BIP proposal: automated compliance check and block template exchange

[grau]

I seek to gather opinions to evolve a BIP targeting automated compliance check across versions and implementations of the protocol.

The aim is to support two, I think compatible, uses cases:

1. Automated validation of a new release or implementation by challenging it with the help of an accepted one.

2. Ability to validate to-be-mined block templates through different implementations at a major mining side to ensure block templates the work will be committed on will be accepted by several versions and implementations.

I believe the protocol should be a challenge and response. The challenge could include blocks valid for a chain (that will likely be an on-the-fly generated test chain) and/or templates to extend that test chain or the production chain. The challenged node would answer with accept/reject message identifying in case of rejection the reason.

Please contribute your ideas to evolve the requirement, then the definition of this.

[1715323809]

1715323809

[1715323809]

1715323809

[1715323809]

1715323809

[grau]

The compliance protocol could include existing and new messages such as
- version, whereby a new service indicator tells if the node supports compliance check.
- a new message type that starts a test chain also identifying its genesis block
- a new message that extends the block message identifying the genesis it belongs to. The block might be solved, having sufficient POW or not since template.
- the challenged node would use a new message to tell acceptance or rejection of received blocks (templates).

[Luke-Jr]

This is basically what BIP 23 Proposals do... you send the node your proposed block (before solving the proof of work) and it does all the validity checks and confirms it's valid. For long chains of multiple proof-less blocks, I suppose it might need some kind of extension, but that strikes me as sort of a "testnet mode" more than part of the protocol for exercising it.

[grau]

For long chains of multiple proof-less blocks, I suppose it might need some kind of extension, but that strikes me as sort of a "testnet mode" more than part of the protocol for exercising it.

Yes this is needed on top of BIP23 to catch non-compliance involving chain reorgs or double-spends.

[Mike Hearn]

Matt has written a tool that feeds blocks received from one node to another and ensures they stay in lockstep. It's here:

http://code.google.com/r/bluemattme-bitcoinj/source/detail?r=9a38bc965a2559f4f873a3337b815cc34a19f05b&name=fullverif

Oddly, it seems to have got unmerged somehow. I'll fix that later.

The problem with "reject blocks that aren't accepted by both implementations" is what happens when somebody else does accept them. If 30% of the network are running in dual mode, and 70% aren't, then you haven't gained anything if the consensus of both implementations are required to accept a block because I can still fork the 30% onto a side chain and cut the cost of a double spend by a third. So there has to be one winner that dominates, and that has to be the majority implementation.

If you have one winner that dominates, then unfortunately the incentives change such that you won't find chain splitting bugs. If I find one and I'm malicious, then I have no incentive to use it and reveal the issue via the parallel-impl test nodes. Rather, I'd wait until some big chunk of the network have made the re-implementation the dominant (or only) codebase on their nodes, and then I'd split.

[Luke-Jr]

The idea isn't to reject blocks based on this criteria, but to avoid making blocks that other nodes reject.

My plan for Eligius is:
1. Randomly ask one of (bitcoind 0.6, bitcoind 0.8, bitsofproof, BitcoinJ) for a block template
2. Optional: Make any block changes here
3. Send the proposed block to all of (bitcoind 0.6, bitcoind 0.8, bitsofproof, BitcoinJ)
4. If any of them reject the proposal, log it (with details) and report to relevant developers; fail back to bitcoind 0.6 proposal

[grau]

So there has to be one winner that dominates, and that has to be the majority implementation.

Yes the majority will overwhelm the minority no change to that. The motivation for this also include use cases like:
1. You do not know in a transition period if the new version is already majority
2. regression testing of pre-releases

If I find one and I'm malicious, then I have no incentive to use it and reveal the issue via the parallel-impl test nodes. Rather, I'd wait until some big chunk of the network have made the re-implementation the dominant (or only) codebase on their nodes, and then I'd split.

A testing protocol makes testing easier, so chances are less a malicious one is the only one who finds the split and can exploit it.

I plan to make my implementation available for testing to anyone through this without requiring to downland install and bootstrap one.

[grau]

The idea isn't to reject blocks based on this criteria, but to avoid making blocks that other nodes reject.

My plan for Eligius is:
1. Randomly ask one of (bitcoind 0.6, bitcoind 0.8, bitsofproof, BitcoinJ) for a block template
2. Optional: Make any block changes here
3. Send the proposed block to all of (bitcoind 0.6, bitcoind 0.8, bitsofproof, BitcoinJ)
4. If any of them reject the proposal, log it (with details) and report to relevant developers; fail back to bitcoind 0.6 proposal

The value you add hereby will further differentiate your service and ensure those committing work to your templates will not waste their electricity.