Bitcoin Forum
December 06, 2016, 08:04:45 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: Merged Mining is NOT ready and should be stopped until it is  (Read 4016 times)
shads
Full Member
***
Offline Offline

Activity: 224


View Profile WWW
October 07, 2011, 06:42:09 AM
 #1

Merged mining is only a few blocks away now and from what I can see the bitcoin side of the equation is grossly underprepared.  I hope I'm wrong but I don't think this is going to go well.  The problem as I see it is that the merged mining community has prepared itself but not given the bitcoin community the tools it needs to adapt.  Let me explain.

Documentation is woefully lacking.  Basically limited to a wiki page a some scattered discussion threads on the namecoin forum.  Miners don't need to do anything but bitcoin pools that wish to adopt merged mining need to implement the spec one way or another.  Currently the only way to do this is to use vinced's patched bitcoind and namecoind along with the python merged-mining-proxy.  This proxy has to sit between the poolserver (e.g. pushpool, poolserverj or custom) and the bitcoind.  This represents both a potential bottleneck and an extra point of failure and to my knowledge has never been tested on a load greater than about 50GH.  The alternative is for pools to implement the merged mining spec themselves.  This is easier said than done.  The spec is NOT documented anywhere.  The limit of the documentation that I can find is this:

Parent blockchain (AKA bitcoind)

In order to verify that a client has done hashing on the namecoin blockchain it is nessecary to add a proof that the work has been done on the bitcoin blockchain to the namecoin blockchain. In order to make it possible to have this proof created by the bitcoin blockchain the bitcoind needs a patch. This patch makes it possible for the bitcoind to cryptograhically sign, that there there was work submitted to the namecoin blockchain as well.

Status: Patch ready for testing. Need to ask if it gets implemented to stock bitcoind
[edit] Auxiliary Blockchain (AKA namecoind)

The namecoind now needs to accept the cryptographically signed proof from the bitcoind.

Status: Patch ready for testing. Proposed for block 24000 (but still not agreed upon).
[edit] merged-mine-proxy

This is the daemon all miners connect to. The daemon itself knows how to connect to the bitcoind and the namecoind checking if shares are valid for one of its downstream blockchains. Furthermore it requests getworks from the parent (which must be patched, see above) and delivers it to the miners.

Status: Ready for testing. Implemented using python.


There is also an article in the bitcoin wiki by Mike Hearn that predates the MM implementation by many months but it at best it explains the general principles using quite a different example and none of the specifics.

For a protocol that involves talking to multiple chains, building merkle trees of block hashes in a barely defined order with extra fields that are defined but not actually used in the implementation, extracting merkle branches, parsing undocumented contructs of headers, parts of coinbase transactions and merkle branches and somehow using this to validate aux blocks where the parent block may not even exist... This documentation woefully inadequate, bordering on pathetic.

If someone wants to make an alternative implementation they have no choice but to pore over the source code in two different languages.  I've done this, it took me days to get a grip on it and if weren't for ArtForz's considerable help it would have taken a lot longer.  Even having done that there are a number of ambiguous issues that leave implementation decisions to be guessed at.  Luke-jr has submitted a partial solution but has also pointed out that it's partial because there is no documentation for the spec.  I have attempted to seek clarifications on the spec myself both on the namecoin forum and their IRC channels only to be told that only one person really knows how it works and they are hardly ever online.  The few answers I was able to get simply confirmed that source code can be relied upon as a guide only, there are details of the spec that are ambiguous in the reference implementation and cannot be definatively determined using it as the only reference.

So the bottom line is that every pool that wants to adopt merged mining currently has no choice but the use the untested merged-mining-proxy black box.  So the MM spec is not documented and frankly I think it's grossly irresponsible of the people pushing MM to have let it get this far without doing so well in advance of the cutover block. Without hours/days of poring over the code (in 2 different languages) it's essentially a black box. The provided solution creates a number of problems and the sensible thing to do would be for pools to simply refuse to touch it until that situation is addressed.  

Unfortunately merged mining has created a situation where adopting the change is very difficult to resist. Once one pool adopts it, it puts enormous pressure on the others to follow as the miners will flock to the pool that offers free extra Xcoins for the same mining effort. End result is we have a mass of pools struggling to cope with new xcoind patches and an unproven python point of potential failure/bottleneck and no one available to support it except the merged mining developers who seem to be online for about 10 mins/week.

At best the provided tools (merged-mining-proxy) should be considered a reference implementation of a spec that doesn't exist. I predict there will be grief for pools and miners over the next few weeks/months.  The best case outcome I can think of is that the extra NMCs mined tank the namecoin market and miners lose interest.

If there are problems and 1/2 pools roll back it still doesn't create any incentive for the MM crowd to step up and do the job properly because their motive is increase hash power on the NMC network and they will have acheived that whatever the consequences to bitcoin mining might be...

IMHO merged mining should be stopped until this situation is properly addressed.

PoolServerJ Home Page - High performance java mining pool engine

1LezqRatQz7MeNoCVziYwcdwtqeEbvrdAq - http://payb.tc/shads

Quote from: Matthew N. Wright
Stop wasting the internet.
1481054685
Hero Member
*
Offline Offline

Posts: 1481054685

View Profile Personal Message (Offline)

Ignore
1481054685
Reply with quote  #2

1481054685
Report to moderator
1481054685
Hero Member
*
Offline Offline

Posts: 1481054685

View Profile Personal Message (Offline)

Ignore
1481054685
Reply with quote  #2

1481054685
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481054685
Hero Member
*
Offline Offline

Posts: 1481054685

View Profile Personal Message (Offline)

Ignore
1481054685
Reply with quote  #2

1481054685
Report to moderator
1481054685
Hero Member
*
Offline Offline

Posts: 1481054685

View Profile Personal Message (Offline)

Ignore
1481054685
Reply with quote  #2

1481054685
Report to moderator
1481054685
Hero Member
*
Offline Offline

Posts: 1481054685

View Profile Personal Message (Offline)

Ignore
1481054685
Reply with quote  #2

1481054685
Report to moderator
urstroyer
Full Member
***
Offline Offline

Activity: 143



View Profile
October 07, 2011, 08:23:53 AM
 #2

I totally agree with shads.

I won't take a huge risk on my pool and for all existing miner, if we don't know about any side-effects right now.

If any pool will take the risk with existing software right now, please consider:

- merged mining right now comes with higher risk for your whole backend
-> is the software ready for highloads?
-> what about stale rates?
-> are your pool member ready to accept necessary downtimes and outages?

Maybe i'll regret it, but right now, i'am happy not beeing an early adaptor, even if some people see advantages of merged mining right now and are willing to change a pool.

It's my responsibilty as a pool operator to offer a solid and stable pool service and if i would rush into merged mining right now, i cannot guarantee for it.

shads
Full Member
***
Offline Offline

Activity: 224


View Profile WWW
October 07, 2011, 08:54:14 AM
 #3

The key problem though is that you and other more cautious pool ops will come under a lot of pressure to adopt.  There will be pools that do straight away and it's a compelling argument for a miner.  Mine BTC only or switch pools and get bonus NMC at no extra cost.

The usual scenario would be if a new tool is crap no one will use it until it's not crap.  The producer of the tool is strongly incentivized to fix it fast.  In this case there's a strong incentive to use it even if it is crap.  The producer of the tool has no incentive to fix it until they damn well feel like it.




PoolServerJ Home Page - High performance java mining pool engine

1LezqRatQz7MeNoCVziYwcdwtqeEbvrdAq - http://payb.tc/shads

Quote from: Matthew N. Wright
Stop wasting the internet.
EhVedadoOAnonimato
Hero Member
*****
Offline Offline

Activity: 616



View Profile
October 07, 2011, 10:02:19 AM
 #4

Would somebody care to explain me why people chose to implement Namecoin with an independent chain, instead of a dependent chain?

By dependent chain I mean a chain in which, to produce a block, all you need to do it sign it with the generation address of the bitcoin block you just mined.
This way there would be no need for "merged mining" at all. The only disadvantage I see is that, in the best case, a dependent change can only grow in the same pace as bitcoin's chain. Grow in the number of blocks I mean. And, well, for a naming service, that doesn't look a problem to me.

Another design choice of namecoin I don't understand is the existence of coins. Couldn't bitcoin themselves be used for transferring names, by using contracts?

I clearly don't know much about namecoin and thus don't understand why such choices were made. Probably there was a good reason.... is such reason written somewhere?

Thanks
Luke-Jr
Legendary
*
Offline Offline

Activity: 2086



View Profile
October 07, 2011, 02:33:48 PM
 #5

Just to be clear, my merged mining implementation AIUI will require two very minor changes to bitcoind, and will not put the Bitcoin mining operation at risk. While the coinbase component (part of Coinbaser, ready for pulling on GitHub) is complete and obviously simple, the part that is lacking is resubmitting new shares to the side MM managing daemon. This is also a fairly trivial change, but without specs to build the external MM manager and test it, I am not comfortable with calling this piece "done".

twobits
Sr. Member
****
Offline Offline

Activity: 336

Firstbits: 1a6taw


View Profile
October 07, 2011, 05:35:31 PM
 #6

Merged mining is only a few blocks away now and from what I can see the bitcoin side of the equation is grossly underprepared.  I hope I'm wrong but I don't think this is going to go well.  The problem as I see it is that the merged mining community has prepared itself but not given the bitcoin community the tools it needs to adapt.  Let me explain.

The thing is,  it  is not like it has come out of the blue,  it has taken many months to get here.  People had been complaining about how long it has taken to get here.  The bitcoin community does not need to be prepared.  They can just continue to ignore it if they want to.

Quote
Documentation is woefully lacking.  Basically limited to a wiki page a some scattered discussion threads on the namecoin forum.  Miners don't need to do anything but bitcoin pools that wish to adopt merged mining need to implement the spec one way or another.  Currently the only way to do this is to use vinced's patched bitcoind and namecoind along with the python merged-mining-proxy.  This proxy has to sit between the poolserver (e.g. pushpool, poolserverj or custom) and the bitcoind.  This represents both a potential bottleneck and an extra point of failure and to my knowledge has never been tested on a load greater than about 50GH.  The alternative is for pools to implement the merged mining spec themselves.  This is easier said than done.  The spec is NOT documented anywhere.

Yep,  lack of a single place to find most information and much of anything to read that is not code is a real pain. 

Vinced's patched *coind stuff is not the only way to do this.  sacarlson's multicoin-exp has supported merged mining since June now, and he used to post hoping to get others helping to learn and test things out with it.   He has even set up a number of test chains for merged mining.  Almost no one joined in.  People have chosen mostly to ignore mm.

I do agree the documentation should be a lot better.   It is very fragile it seems, and you do need to read code to tweek it at all.  The thing is though it is an OSS project, and they need to do things to get people involved as they are always short man power.   I am tearing my hair out trying to figure out why merged mining works with these two chains, but fails when I add this third chain, but not another third chain etc..     I really wish it was easier to set up.   It is not going to get to be so though with the current man power on it.   

Quote
Status: Ready for testing. Implemented using python.


Ready for testing.. but then how many actually have help them with the testing?

Quote
So the bottom line is that every pool that wants to adopt merged mining currently has no choice but the use the untested merged-mining-proxy black box.  So the MM spec is not documented and frankly I think it's grossly irresponsible of the people pushing MM to have let it get this far without doing so well in advance of the cutover block. Without hours/days of poring over the code (in 2 different languages) it's essentially a black box. The provided solution creates a number of problems and the sensible thing to do would be for pools to simply refuse to touch it until that situation is addressed.  

Unfortunately merged mining has created a situation where adopting the change is very difficult to resist. Once one pool adopts it, it puts enormous pressure on the others to follow as the miners will flock to the pool that offers free extra Xcoins for the same mining effort. End result is we have a mass of pools struggling to cope with new xcoind patches and an unproven python point of potential failure/bottleneck and no one available to support it except the merged mining developers who seem to be online for about 10 mins/week.

Yep, that's it...  once it is active,  people will take an interest finally, and help developed the needed infrastructure if they want it.  Since pools will probably need it to stay competitive they will probably be a major source of the manpower or bounties to finally get what is needed done after all these months.

Quote
If there are problems and 1/2 pools roll back it still doesn't create any incentive for the MM crowd to step up and do the job properly because their motive is increase hash power on the NMC network and they will have acheived that whatever the consequences to bitcoin mining might be...

IMHO merged mining should be stopped until this situation is properly addressed.

Thing is,  we have already had months to work on things, and no one has till it is almost here.  I don't see a delay changing that.  People need real coins to mm or they won't , as the lack of interest in the multicoin-exp test chains shows.




Luke-Jr
Legendary
*
Offline Offline

Activity: 2086



View Profile
October 07, 2011, 05:46:22 PM
 #7

Thing is,  we have already had months to work on things, and no one has till it is almost here.
No. We don't get to start working until there is proper protocol documentation in place.

shads
Full Member
***
Offline Offline

Activity: 224


View Profile WWW
October 08, 2011, 12:41:01 AM
 #8

Thing is,  we have already had months to work on things, and no one has till it is almost here.
No. We don't get to start working until there is proper protocol documentation in place.

^^ this

It far from a trivial process to construct a set of mm blocks and even less so to validate them.  Even one small ambiguity in the process means two different implementations quite likely won't accept each others blocks.


PoolServerJ Home Page - High performance java mining pool engine

1LezqRatQz7MeNoCVziYwcdwtqeEbvrdAq - http://payb.tc/shads

Quote from: Matthew N. Wright
Stop wasting the internet.
Gabi
Legendary
*
Offline Offline

Activity: 1050


View Profile
October 10, 2011, 07:15:03 PM
 #9

Well so far it seems to work...

The only risk i see is that you can risk to lose your work on that pool but as long as you accept that risk, i think it's fine
Luke-Jr
Legendary
*
Offline Offline

Activity: 2086



View Profile
October 10, 2011, 07:21:27 PM
 #10

Well so far it seems to work...
Really? What big pools support it? Where is the data showing there's been no loss of Bitcoin blocks/hashpower? I've in fact heard reports to the contrary...

The only risk i see is that you can risk to lose your work on that pool but as long as you accept that risk, i think it's fine
Due to the panic-rush of it going live before being documented, I have deployed my alternate implementation to Eligius. Unlike most pools (ie, those using the proxy), Eligius does not have this risk.

Gabi
Legendary
*
Offline Offline

Activity: 1050


View Profile
October 10, 2011, 07:41:04 PM
 #11

Who cares about "big pools"? I only care about my profit. It's not hard to check what my profit should be and then check if i really have that profit. So far, i have that profit.

So again, who cares about "big pools"? As far as i gain bitcoin and namecoin, it is working for me.

makomk
Hero Member
*****
Offline Offline

Activity: 686


View Profile
October 10, 2011, 10:25:23 PM
 #12

Merged solo mining doesn't seem to be too bad, but pooled is another matter entirely. For example, I can't figure out how you're meant to tell if the aux chain part of a share is stale or not...

Quad XC6SLX150 Board: 860 MHash/s or so.
SIGS ABOUT BUTTERFLY LABS ARE PAID ADS
iopq
Hero Member
*****
Offline Offline

Activity: 644


View Profile
October 14, 2011, 02:37:06 AM
 #13

Well so far it seems to work...
Really? What big pools support it?
slush, apparently

Luke-Jr
Legendary
*
Offline Offline

Activity: 2086



View Profile
October 14, 2011, 02:46:51 AM
 #14

Well so far it seems to work...
Really? What big pools support it?
slush, apparently
He also uses a custom implementation.

DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
October 14, 2011, 02:50:50 AM
 #15

Well so far it seems to work...
Really? What big pools support it?
slush, apparently

Eligius also (well I guess that is a medium pool).
Luke-Jr
Legendary
*
Offline Offline

Activity: 2086



View Profile
October 14, 2011, 02:51:36 AM
 #16

Well so far it seems to work...
Really? What big pools support it?
slush, apparently

Eligius also (well I guess that is a medium pool).
I meant other pools :p

(obviously Eligius is running my custom implementation)

sadpandatech
Hero Member
*****
Offline Offline

Activity: 504



View Profile
October 17, 2011, 06:08:43 PM
 #17

Luke, did you get the msg about the much needed update to namecoind? http://dot-bit.org/forum/viewtopic.php?p=2182#p2182

  That goes for anyone else that may be solo mining or mining thru namecoind at some of the smaller pools as well.....

If you're not excited by the idea of being an early adopter 'now', then you should come back in three or four years and either tell us "Told you it'd never work!" or join what should, by then, be a much more stable and easier-to-use system. - GA
It is being worked on by smart people. -DamienBlack
Luke-Jr
Legendary
*
Offline Offline

Activity: 2086



View Profile
October 17, 2011, 06:11:35 PM
 #18

Luke, did you get the msg about the much needed update to namecoind? http://dot-bit.org/forum/viewtopic.php?p=2182#p2182

  That goes for anyone else that may be solo mining or mining thru namecoind at some of the smaller pools as well.....
I reported it. :p

sadpandatech
Hero Member
*****
Offline Offline

Activity: 504



View Profile
October 17, 2011, 06:14:06 PM
 #19

Luke, did you get the msg about the much needed update to namecoind? http://dot-bit.org/forum/viewtopic.php?p=2182#p2182

  That goes for anyone else that may be solo mining or mining thru namecoind at some of the smaller pools as well.....
I reported it. :p

  LMAO. sorry was getting spam happy with it. Sadly, 2 of the larger ones, Slush and Davinci are both away atm I believe. :/

If you're not excited by the idea of being an early adopter 'now', then you should come back in three or four years and either tell us "Told you it'd never work!" or join what should, by then, be a much more stable and easier-to-use system. - GA
It is being worked on by smart people. -DamienBlack
Luke-Jr
Legendary
*
Offline Offline

Activity: 2086



View Profile
October 17, 2011, 06:28:13 PM
 #20

Luke, did you get the msg about the much needed update to namecoind? http://dot-bit.org/forum/viewtopic.php?p=2182#p2182

  That goes for anyone else that may be solo mining or mining thru namecoind at some of the smaller pools as well.....
I reported it. :p

  LMAO. sorry was getting spam happy with it. Sadly, 2 of the larger ones, Slush and Davinci are both away atm I believe. :/
I know Slush has already upgraded at least.

Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!