Generation of Test Blocks for the Production Network.

[MatthewLM]

Hello. For the testing of my block-chain validation code, I am needing some blocks which allow me to test various cases. For instance a block where everything is valid except for the timestamp.

The problem is, to test these without disabling the PoW checks, the blocks need to have valid hashes. I'm making this topic to ask if anyone would like to help generate some of these blocks. Once created these blocks should hopefully be useful for many bitcoin implementations.

Matt Corallo has a program named pulltester which may help with this, though I'll leave him the opportunity to explain that in more detail, if he wishes.

I'll list here the blocks I know I'll need to test, though there may be more blocks which will be helpful (inc. for other implementations). A lot of the tests can be done on the validation functions without needing separate blocks for each case so it may seem many things are missing.

All of these blocks should be valid except for the things mentioned. This includes a valid hash.

Blocks after the genesis block with the following problems (ie. At height 1):

* Block with zero transactions.
* Block with incorrect merkle root.
* Block with first transaction not a coinbase.
* Block with non-final coinbase transaction
* Block with zero outputs
* Block with a size of 1000001 bytes.
* Block with bad coin-base script size (< 2 or > 100).

Blocks after the 100 miner test blocks which are made with this code: https://github.com/bitcoin/bitcoin/blob/master/src/test/miner_tests.cpp

* A block with a mixture of sigops (inc. P2SH) with 20000 sigops (This is a valid one).
* Same but with 20001 sigops.
* Block spending coinbase from height 2.
* Block with non-coinbase transaction that is not final.
* Block with transaction spending output of a transaction earlier inside that block (Valid).
* Block with transaction spending output of a transaction later inside that block.
* Two inputs referencing same output in two different transactions in a block.
* Block with transaction with invalid input script.
* Block with input referencing output that does not exist.
* Block with coinbase value equal to block reward plus several transaction fees (Valid).
* Block with coin base value equal to block reward plus several transaction fees plus one.
* Block with timestamp 1231006525.

I also need to test some obscure parts of the block-chain re-organisation.

This can be done with a branch which goes off the branch created with the 100 test blocks and becomes the main chain. Then several blocks can be added to that branch so that the branch with the 100 blocks can be extended without becoming the main chain. Then another branch is made of this branch (Not an extension) which becomes the main branch.

So to begin with there will be these blocks

Code:

A: 0 --> 1
B:   `-> 1 … 99 --> 100

Next a new main chain is need that branches of the previous main chain:

Code:

A: 0 --> 1
B:   `-> 1 … 99 --> 100
C:              `-> 100 --> 101

Then this new branch is extended to make room for extension of the other branch.

Code:

A: 0 --> 1
B:   `-> 1 … 99 --> 100
C:              `-> 100 --> 101 --> 102

Now the other branch is extended:

Code:

A: 0 --> 1
B:   `-> 1 … 99 --> 100 --> 101 --> 102
C:              `-> 100 --> 101 --> 102

Then the new main chain is made:

Code:

A: 0 --> 1
B:   `-> 1 … 99 --> 100 --> 101 --> 102
D:             \                `-> 102 --> 103
C:              `-> 100 --> 101 --> 102

What that tests, is reorganisation in the case where the new main chain makes use of a parent branch with previous validation data and which requires further validation. In this case block B101 has not been completely validated before the reorganisation because complete validation is only done on blocks that enter the main chain or when there is a reorganisation. That's just one part of the code that needs testing.

It would also be good to have more blocks with an invalid block for B101 as that would ensure an invalid block in a parent branch whilst doing complete validation would be caught. B101 would need to be invalid in a particular way. Give B101 an invalid coinbase output value. That would work.

So what is needed is C100, C101, C102, B101, B102, D102, D103, where B is the branch using the miner test blocks.

And also another set of B101, B102, D102, D103 with an invalid B101.

Sorry that this is confusing.

I also need to test the branch trimming code but I won't figure that out until I've actually coded it. 

Thanks for anyone willing to help. If anyone needs any further information please ask.

[Matt Corallo]

As mentioned, I wrote a little tool to test block validation/reorg code to compare my full node-implementation in bitcoinj to that of the reference client which turns out to be fairly good at testing compliance with existing spec of just about any fully verifying bitcoin node which implements the important parts of the P2P protocol.

You can find it at: http://jenkins.bluematt.me/pull-tester/files/ (its currently used as a part of pull-tester to test new pulls as they come in to the reference client).
Its currently using an alt-chain so that it can dynamically generate blocks with a tiny diff.  Its on my get-to-it-eventually TODO list to actually mine the blocks giving them valid POW such that they work on a regular client, but its only a few-line patch that any client should be able to implement, so I feel no rush...(though it would always be nice if someone else stepped up and did it )

You can see the list of tests it performs at http://jenkins.bluematt.me/pull-tester/files/FullBlockTestGenerator.java.  It hits many of the cases you listed (though it has a predetermined list of forks it tests instead of doing tons of forks at a time).  It does, however, hit 100% line coverage of reachable lines in block verification on the reference client, so, despite missing a few cases, it does a pretty good job (if anyone wants to write in the case, Id be very happy to merge them).  

I would expect any full node implementation to test using at least:
data-driven tx/script tests at https://github.com/bitcoin/bitcoin/tree/master/src/test/data and the pull tester above (just start a patched node on localhost and run the jar).