Coinbaser branch's new JSON-RPC method

[Luke-Jr]

It is impossible and unreasonable to force everyone to have the exact same system clock.

I think that Lolcust and ArtForz provided a good counter-example by their GeistGeld alt-chain that uses SNTP for time synchronization. The objective is not "exact-same" system clock, but the system clock differences that are within currently acceptable norms: I think it is still one minute in the insurance industry and in something close to one second in the finance industry.

Notice that I didn't say it's impossible/unreasonable to actually have the "exact" (within reason) same clock. I said it's impossible/unreasonable to force it. But anyhow, it is indeed an off-topic discussion for this thread, which has nothing to do with block times. :p

[shads]

I've only got two concerns left with this patch.  Luke's done some good doco for both parts of it.  And now that he's explained how you'd use a bit better I can see this becoming an eventual replacements for most of vinced's implementation.   It is a much cleaner way in the long run.

The remain concerns are:  Does the -coinbase <cmd> enforce either a minimum tx output value or a max number of tx outputs?  If not it should.  The primary use case for this for pools to payout their miners and have invalid blocks dealt with automagically.  It there are no limits it's only a matter of time before some pools start paying out tiny balances to masses of miners.  This could bloat the coinbase a lot unless I'm missing something. 

My second concern is that Luke has made the following comment in the wiki:

Insert exactly one of these headers into the scriptSig of the coinbase on the parent chain. vinced's code always inserts an OP_2, 0xfa, 0xbe, 'm', 'm' in front of it, but you don't have to.

As far as I can see this patch doesn't deal with validation of the aux blocks.  Presumably this will be left to vinced's original code as you'd require an update to all namecoin miners to alter it's validation rules.  So the question is, will the absence of OP_2, 0xfa, 0xbe, 'm', 'm' before the aux header prevent vinced's validation from working?  If so then the rpc call should check this and return an error if it doesn't.

[Luke-Jr]

The remain concerns are:  Does the -coinbase <cmd> enforce either a minimum tx output value or a max number of tx outputs?  If not it should.  The primary use case for this for pools to payout their miners and have invalid blocks dealt with automagically.  It there are no limits it's only a matter of time before some pools start paying out tiny balances to masses of miners.  This could bloat the coinbase a lot unless I'm missing something.

There is nothing anyone can do to stop block-makers from producing tiny coins. Coinbaser does prevent invalid outputs, though-- if the output is not consistent (ie, tries to pay too much, to an invalid address, etc) it fails over the the default behaviour.

My second concern is that Luke has made the following comment in the wiki:

Insert exactly one of these headers into the scriptSig of the coinbase on the parent chain. vinced's code always inserts an OP_2, 0xfa, 0xbe, 'm', 'm' in front of it, but you don't have to.

As far as I can see this patch doesn't deal with validation of the aux blocks.  Presumably this will be left to vinced's original code as you'd require an update to all namecoin miners to alter it's validation rules.  So the question is, will the absence of OP_2, 0xfa, 0xbe, 'm', 'm' before the aux header prevent vinced's validation from working?  If so then the rpc call should check this and return an error if it doesn't.

No, the reason I changed this to say optional is because I am successfully using it on Eligius without that magic prefix. Furthermore, while the name "setworkaux" might be the source of confusion, it can be used for adding any arbitrary data to the coinbase, not just for the particular iteration of merged mining. There can very well be future functionality (like advertising your fee schedule, for example) using the coinbase as well.

[shads]

The remain concerns are:  Does the -coinbase <cmd> enforce either a minimum tx output value or a max number of tx outputs?  If not it should.  The primary use case for this for pools to payout their miners and have invalid blocks dealt with automagically.  It there are no limits it's only a matter of time before some pools start paying out tiny balances to masses of miners.  This could bloat the coinbase a lot unless I'm missing something.

There is nothing anyone can do to stop block-makers from producing tiny coins. Coinbaser does prevent invalid outputs, though-- if the output is not consistent (ie, tries to pay too much, to an invalid address, etc) it fails over the the default behaviour.

True anyone could create a monster coinbase transaction if they had the knowledge to construct a block from scratch and feed it into the network via p2p protocol.  But if you knew enough to do that you'd probably know why it's a bad idea.  A simple command line switch doesn't require any real knowledge of the network to use successfully and that's where the danger lies.  A relatively technically unskilled pool op decides to payout every single miner on every won block.  This is quite feasible and without any enforcement it just makes the problem accessible to people more likely to make the mistake.

My second concern is that Luke has made the following comment in the wiki:

Insert exactly one of these headers into the scriptSig of the coinbase on the parent chain. vinced's code always inserts an OP_2, 0xfa, 0xbe, 'm', 'm' in front of it, but you don't have to.

As far as I can see this patch doesn't deal with validation of the aux blocks.  Presumably this will be left to vinced's original code as you'd require an update to all namecoin miners to alter it's validation rules.  So the question is, will the absence of OP_2, 0xfa, 0xbe, 'm', 'm' before the aux header prevent vinced's validation from working?  If so then the rpc call should check this and return an error if it doesn't.

No, the reason I changed this to say optional is because I am successfully using it on Eligius without that magic prefix.

And those blocks without magic prefix are accepted by the rest of the nmc network?  If so that's my concern dealt with.

*If* a reasonable limit on coinbase outputs could be added then for what it's worth I'd be happy to add my vote in favour on the change.  Without said limits then my vote is in favour of the 'setworkaux' part of the change only.

[Luke-Jr]

And those blocks without magic prefix are accepted by the rest of the nmc network?  If so that's my concern dealt with.

Yes, they are.

*If* a reasonable limit on coinbase outputs could be added then for what it's worth I'd be happy to add my vote in favour on the change.  Without said limits then my vote is in favour of the 'setworkaux' part of the change only.

And what reasonable limit would that be? There are at least legitimate circumstances for having tiny-amount outputs (I always include a single 1 Satoshi output so it shows up on my pool's listtransactions). I suppose there could be a hard-cap on total count of outputs, but what should that be? Maybe 1% of the current block size limit? That gives 294 outputs max at 1 MB (the current hard-coded block size limit)-- an average of ~0.17 BTC each.

[shads]

*If* a reasonable limit on coinbase outputs could be added then for what it's worth I'd be happy to add my vote in favour on the change.  Without said limits then my vote is in favour of the 'setworkaux' part of the change only.

And what reasonable limit would that be? There are at least legitimate circumstances for having tiny-amount outputs (I always include a single 1 Satoshi output so it shows up on my pool's listtransactions). I suppose there could be a hard-cap on total count of outputs, but what should that be? Maybe 1% of the current block size limit? That gives 294 outputs max at 1 MB (the current hard-coded block size limit)-- an average of ~0.17 BTC each.

Well I can see two approaches to determining limit.  Both have issues but just because it's tricky to enforce a limit sensibly isn't a good reason to allow open slather...

1/ Tx outpoint size is relatively constant.  Yes it can vary but for 99% of use cases it will the same type of script which is the same length.  So from simple easy to enfore rule that would probably be the better option.  I notice eligius already has some notion of a payout queue and I thought that might be somehow related to this issue.  I was quite suprised when I started looking a eligius coinbases to find the number of outputs was always far far less than what I'd guess yr total miners would be.  That's actually what got me thinking about this as a potential problem.  When I saw that I thought you were limiting it for that very reason.

2/ min tx value would be easier from a pool ops point of view.  And whilst the size of the outputs would be more variable it would at least prevent extreme accidental abuse.

Those are the options I can think of... not sure which is the best or if there's a better one or if I'm even worrying about a non-issue.  Hoping someone else will step up with an opinion...

[Luke-Jr]

*If* a reasonable limit on coinbase outputs could be added then for what it's worth I'd be happy to add my vote in favour on the change.  Without said limits then my vote is in favour of the 'setworkaux' part of the change only.

And what reasonable limit would that be? There are at least legitimate circumstances for having tiny-amount outputs (I always include a single 1 Satoshi output so it shows up on my pool's listtransactions). I suppose there could be a hard-cap on total count of outputs, but what should that be? Maybe 1% of the current block size limit? That gives 294 outputs max at 1 MB (the current hard-coded block size limit)-- an average of ~0.17 BTC each.

Well I can see two approaches to determining limit.  Both have issues but just because it's tricky to enforce a limit sensibly isn't a good reason to allow open slather...

1/ Tx outpoint size is relatively constant.  Yes it can vary but for 99% of use cases it will the same type of script which is the same length.  So from simple easy to enfore rule that would probably be the better option.  I notice eligius already has some notion of a payout queue and I thought that might be somehow related to this issue.  I was quite suprised when I started looking a eligius coinbases to find the number of outputs was always far far less than what I'd guess yr total miners would be.  That's actually what got me thinking about this as a potential problem.  When I saw that I thought you were limiting it for that very reason.

The original reason for that was simply to help miners avoid a lot of tiny payouts they have to pay large fees to spend.

2/ min tx value would be easier from a pool ops point of view.  And whilst the size of the outputs would be more variable it would at least prevent extreme accidental abuse.

As I said, valid values can be as low as 1 satoshi...

[shads]

*If* a reasonable limit on coinbase outputs could be added then for what it's worth I'd be happy to add my vote in favour on the change.  Without said limits then my vote is in favour of the 'setworkaux' part of the change only.

And what reasonable limit would that be? There are at least legitimate circumstances for having tiny-amount outputs (I always include a single 1 Satoshi output so it shows up on my pool's listtransactions). I suppose there could be a hard-cap on total count of outputs, but what should that be? Maybe 1% of the current block size limit? That gives 294 outputs max at 1 MB (the current hard-coded block size limit)-- an average of ~0.17 BTC each.

Well I can see two approaches to determining limit.  Both have issues but just because it's tricky to enforce a limit sensibly isn't a good reason to allow open slather...

1/ Tx outpoint size is relatively constant.  Yes it can vary but for 99% of use cases it will the same type of script which is the same length.  So from simple easy to enfore rule that would probably be the better option.  I notice eligius already has some notion of a payout queue and I thought that might be somehow related to this issue.  I was quite suprised when I started looking a eligius coinbases to find the number of outputs was always far far less than what I'd guess yr total miners would be.  That's actually what got me thinking about this as a potential problem.  When I saw that I thought you were limiting it for that very reason.

The original reason for that was simply to help miners avoid a lot of tiny payouts they have to pay large fees to spend.

2/ min tx value would be easier from a pool ops point of view.  And whilst the size of the outputs would be more variable it would at least prevent extreme accidental abuse.

As I said, valid values can be as low as 1 satoshi...

Well I just don't think encouraging/facilitating tiny payouts is a good idea... perhaps this is the way to go: "Maybe 1% of the current block size limit? That gives 294 outputs max at 1 MB (the current hard-coded block size limit)"

It's more tricky for pool ops to calculate but it is a sensible number.

[Mike Hearn]

And those blocks without magic prefix are accepted by the rest of the nmc network?  If so that's my concern dealt with.

That's weird. What is the prefix for, if Namecoin doesn't actually need it?

[Luke-Jr]

And those blocks without magic prefix are accepted by the rest of the nmc network?  If so that's my concern dealt with.

That's weird. What is the prefix for, if Namecoin doesn't actually need it?

It was originally proposed as a way to prevent multiple MM hashes in the same coinbase. After a few people (myself included) made a fuss over the inelegance of it, I presume it was changed to the current (not too much better) method of forcing a specific slot in the aux merkle tree.

[Mike Hearn]

I don't understand either approach to be honest. The alt chains wiki page laid out how that's supposed to work. You tie your aux block to the coinbase hash with a merkle branch to the value in the coinbase transaction and an index into that coinbase scriptSig. There's no need for magic markers or some kind of "uber-merkle-tree" for all aux chains.

Maybe there's something I missed, after all I haven't actually implemented this.
 

[John Tobey]

Maybe there's something I missed, after all I haven't actually implemented this.

This thread touches on it.  I'm curious what you'd think of it.

Why is it so hard to simply disallow a master-chain block from "vouching for" more than one block in each aux chain? :/

Mhmm, If I understand it correctly this might be undesirable because it might be (and is common) that you solve several AUX chain blocks and have one master-block vouching for them, right? This problem would only be solved if the difficulty of both blockchains draw levels.

This was solved in the current code by requiring each chain to appear in a specific (pseudo-random) spot in the aux chain merkle tree, based on the chain ID.  So for a specific chain ID, you would only be able to "solve" one aux chain block per parent block.

It's good to see you all evaluating the details of the implementation and I appreciate the feedback.  The more eyes on this the better.

There is a more flexible solution to the problem.

The problem is that, with MM as first implemented, a miner could cheat the proof-of-work system by putting more than one Namecoin block header into one Bitcoin block.  This "cheating" would not cause a problem if the Namecoin blocks all had the same previous-block hash: one or another would win the race, as happens when competing blocks enter the network.  The problem would arise if a linked series of Namecoin blocks all existed in one Bitcoin block's aux Merkle trees.  The miner could work on them all at once, getting more from his hashes than difficulty warrants.

Vince's solution is to constrain the paths by which Namecoin blocks may be embedded in Bitcoin blocks.  This works, but it complicates the picture for miners and chain designers in a future with many merged-mining chains.

A better solution for new chains is to prevent the existence of linked series of aux chain blocks which lack proof of work.  You might do that by requiring each aux chain block to refer (perhaps via its own coinbase or a new header field) to the "solution" (target-meeting hash of parent chain block) as well as the hash of the previous block in the aux chain.  In Bitcoin, these two are the same, namely hashPrevBlock.  Force blocks to refer not just to the previous block in the chain but to the proof of work on that block.  A series of aux blocks all being worked on would not make sense, because the second block in the series would have to reference the first block's solution, so the first block would have to have been solved.

[Mike Hearn]

Yeah, that solution does look better. It'd be good to put that on the Alternative Chains wiki page too.

[John Tobey]

Yeah, that solution does look better. It'd be good to put that on the Alternative Chains wiki page too.

Thanks, will update in my Copious Spare Time.  

(off-topic, sorry) I think also Luke-Jr's proposal

Why is it so hard to simply disallow a master-chain block from "vouching for" more than one block in each aux chain? :/

could use elaboration.  I take it to mean, have the aux chain software enforce uniqueness of its work-proving hashes.  I think that would work too and is a bit more elegant than sticking the value in the aux chain.

[Luke-Jr]

(off-topic, sorry) I think also Luke-Jr's proposal

Why is it so hard to simply disallow a master-chain block from "vouching for" more than one block in each aux chain? :/

could use elaboration.  I take it to mean, have the aux chain software enforce uniqueness of its work-proving hashes.  I think that would work too and is a bit more elegant than sticking the value in the aux chain.

I think the latter is a better solution: it doesn't require looking up the hash in all the past blocks.

[John Tobey]

Yeah, that solution does look better. It'd be good to put that on the Alternative Chains wiki page too.

Done.  New section; created merged mining page as redirect to Alternative Chains.

[maaku]

Couldn't one of the rules of the aux-chains be to reject proof-of-works which have more than one aux-header in the Merkle tree for the same aux-chain?

[Mike Hearn]

Right, I didn't think Lukes proposal through properly. It is simpler so I updated the wiki page to say that.

[John Tobey]

Right, I didn't think Lukes proposal through properly. It is simpler so I updated the wiki page to say that.

As I would have done, had not Luke himself preferred the other way.  No biggie, I think they're both fine, and the wiki page links here for the truly interested.

[Luke-Jr]

Back on topic, it's about time for 0.6 merging, and coinbaser is still cleanly mergable. Could we get the few people who voted against it to change their votes, or at least explain them?