Correct header data for block 1

[MatthewLM]

Hello. I have some code that calculates a hash for a block. It was working perfectly sometime before, but now for some reason I cannot get a correct hash for block 1. I can calculate the correct hash for the genesis block, but not block 1. I've taken what I've thought is the correct header data for block 1 and I've tried to hash it manually using a technique that works for the genesis block and it seems that the header data is incorrect. Can anyone find a problem with this data? I might be going crazy as I can't find it:

Code:

// VERSION
0x1, 0x0, 0x0, 0x0,
// PREV BLOCK HASH
0x6f, 0xe2, 0x8c, 0xa, 0xb6, 0xf1, 0xb3, 0x72, 0xc1, 0xa6, 0xa2, 0x46, 0xae, 0x63, 0xf7, 0x4f, 0x93, 0x1e, 0x83, 0x65, 0xe1, 0x5a, 0x8, 0x9c, 0x68, 0xd6, 0x19, 0x0, 0x0, 0x0, 0x0, 0x0,
// MERKLE ROOT
0x98, 0x20, 0x51, 0xfd, 0x1e, 0x4b, 0xa7, 0x44, 0xbb, 0xcb, 0x68, 0xe, 0x1f, 0xee, 0x14, 0x67, 0x7b, 0xa1, 0xa3, 0xc3, 0x54, 0xb, 0xf7, 0xb1, 0xcd, 0xb6, 0x6, 0xe8, 0x57, 0x23, 0x3e, 0xe,
// TIME
0x61, 0xbc, 0x66, 0x49,
// TARGET
0xff, 0xff, 0x0, 0x1d,
// NONCE
0x1, 0xe3, 0x62, 0x99

Thanks.

[koin]

for some reason I cannot get a correct hash for block 1..

for block 1 or for the genesis block (block 0)? https://bitcointalk.org/index.php?topic=119530

[MatthewLM]

Hi koin. I am referring to the block with the index 1, aka. the 2nd block.

http://blockexplorer.com/b/1

This is driving me slightly insane. :-)

[kjj]

Double check the endianness of your prevhash.  In the genesis block, it is all zeros, meaning that you still get the right result even if you have it backwards.

Just a guess.

[MatthewLM]

Double check the endianness of your prevhash.  In the genesis block, it is all zeros, meaning that you still get the right result even if you have it backwards.

Just a guess.

That's a good guess kjj. Unfortunately I tried reversing the previous block hash and it still failed.

[etotheipi]

Perhaps it's a single-bit error!  I documented my experience with this here.  It was maddening, to be able to confirm 1.5 million tx hashes, and have exactly one fail.  It took me a while, but I eventually narrowed it down.  I was able to manually compare the tx from a different source to the one in my blkfile. 

[MatthewLM]

Thanks for the link Alan, as it's interesting.

But I did not obtain the block data from the block database from the satoshi client. I've created the data manually. It was working before, so at some point I accidentally changed something, I just do not know what. Does anyone know how to dump block byte data from the block database for the satoshi client? Doing that would be very useful to compare the data.

As for bit errors, I've received mixed messages as to whether or not error correction is important at the application level. To be safe, in any case, I plan to implement some error correction to my database software. Before I do that I want to get the full validator unit tests working. The error correction will probably be something simple at first, such as hamming code.

[kjj]

Does anyone know how to dump block byte data from the block database for the satoshi client?

https://bitcointalk.org/index.php?topic=119530.msg1291088#msg1291088

[MatthewLM]

I managed to get a hex dump of the raw block data. The blockchain data storage is more simple than I thought it would be. The blocks are stored with the network identifier ("magic bytes") followed by the block length and then the block. This is the result:

Code:

(null):cbitcoin matt$ hexdump -C -n 80 -s 301 /Users/matt/Library/Application\ Support/Bitcoin/blk0001.dat 
0000012d  01 00 00 00 6f e2 8c 0a  b6 f1 b3 72 c1 a6 a2 46  |....o......r...F|
0000013d  ae 63 f7 4f 93 1e 83 65  e1 5a 08 9c 68 d6 19 00  |.c.O...e.Z..h...|
0000014d  00 00 00 00 98 20 51 fd  1e 4b a7 44 bb be 68 0e  |..... Q..K.D..h.|
0000015d  1f ee 14 67 7b a1 a3 c3  54 0b f7 b1 cd b6 06 e8  |...g{...T.......|
0000016d  57 23 3e 0e 61 bc 66 49  ff ff 00 1d 01 e3 62 99  |W#>.a.fI......b.|

And indeed when I compared the data I saw there is one incorrect byte in the merkle root. I think I remember what I did. I replaced all the lowercase hexadecimal literals with uppercase ones and must have made this mistake and forgot about it. It works now.

Does anyone know how to dump block byte data from the block database for the satoshi client?

https://bitcointalk.org/index.php?topic=119530.msg1291088#msg1291088

Thanks, though I managed to figure it out.

[Peter Todd]

FWIW I wrote some Python code that recreates the raw hex block header, the part that is hashed to create a block hash, from the output of the RPC interface getblock:

Code:

import struct
import hashlib

bitcoin_header_format = struct.Struct("<i 32s 32s I 4s I")

def serialize_block_header(block):
    """Serialize a block header from the RPC interface"""
    return bitcoin_header_format.pack(
        block['version'],
        unhexlify(block['previousblockhash'])[::-1],
        unhexlify(block['merkleroot'])[::-1],
        block['time'],
        unhexlify(block['bits'])[::-1],
        block['nonce'])

def calculate_block_hash(raw_block_header):
    hexlify(hashlib.sha256(hashlib.sha256(raw_block_header).digest()).digest()[::-1])

Note the [::-1] to the byte order to change the endianness in a few places.