Title: Methods of decoding a bitcoin block stored in windows Post by: supermankid on March 27, 2022, 05:53:01 PM Hello everyone,
Started doing some research on bitcoin blocks, when i run a node the blocks are stored on => ...\BTC\blocks (windows) I opened with hex viewer and I could indeed see the infamous"chancelor..." in the genesis block. How can the transaction history be decoded from this block. Secondly the blocks are not 1MB they are like 127MB. How do block explorer or a normal person decode this information from these blocks? Thanks in advance. Title: Re: Methods of decoding a bitcoin block stored in windows Post by: n0nce on March 27, 2022, 06:56:31 PM Hello everyone, You need all blocks to get full transaction history. ;)Started doing some research on bitcoin blocks, when i run a node the blocks are stored on => ...\BTC\blocks (windows) I opened with hex viewer and I could indeed see the infamous"chancelor..." in the genesis block. How can the transaction history be decoded from this block. If you mean decoding the block and such, honestly the best tool is just using Bitcoin Core which you seem to have installed. But if you want to decode individual blk files, I found this small program (https://gist.github.com/ysqi/62820e5955a3e3b56efb8b396585f44a) that might do the trick! python-bitcoin-blockchain-parser]This repository (https://github.com/alecalve/python-bitcoin-blockchain-parser) actually looks a bit better to me, since it has more features and it's Python 3-based so you won't need to compile it! For doing it manually, I like to follow along code such as the one from the link above. As you can see, there is this hex code that you should be able to find in your .dat file, which separates blocks. If you count the bytes between these separators, it should be always around 1MB. Code: (https://github.com/alecalve/python-bitcoin-blockchain-parser/blob/d763ca6dc2357cdf2e885f3e6790eeab1aeaa0a8/blockchain_parser/blockchain.py#L30) BITCOIN_CONSTANT = b"\xf9\xbe\xb4\xd9" In line 46 (https://github.com/alecalve/python-bitcoin-blockchain-parser/blob/d763ca6dc2357cdf2e885f3e6790eeab1aeaa0a8/blockchain_parser/blockchain.py#L46), in get_blocks(), you can see how one blk....dat file is split up and the raw bytes of each block are returned. For decoding that block manually, you can have a look into the codebase of the above project or refer to the Block structure (https://en.bitcoin.it/wiki/Block#Block_structure) according to Bitcoin Wiki. Here's the block header (https://en.bitcoin.it/wiki/Block_hashing_algorithm) and the corresponding Bitcoin Core code is found here (https://github.com/bitcoin/bitcoin/blob/2f0f056e08cd5a1435120592a9ecd212fcdb915b/src/primitives/block.h#L20). Code: (https://github.com/bitcoin/bitcoin/blob/2f0f056e08cd5a1435120592a9ecd212fcdb915b/src/primitives/block.h#L20) // header Secondly the blocks are not 1MB they are like 127MB. That's because one such file contains more than one block.How do block explorer or a normal person decode this information from these blocks? Thanks in advance. Title: Re: Methods of decoding a bitcoin block stored in windows Post by: ABCbits on March 28, 2022, 12:23:28 PM Secondly the blocks are not 1MB they are like 127MB. How do block explorer or a normal person decode this information from these blocks? Thanks in advance. I'd like to mention Bitcoin protocol doesn't specify how you store blockchain on your device. If you use different full node software (Bitcoin Core isn't the only one), it's possible they store blockchain differently. If you mean decoding the block and such, honestly the best tool is just using Bitcoin Core which you seem to have installed. But if you want to decode individual blk files, I found this small program (https://gist.github.com/ysqi/62820e5955a3e3b56efb8b396585f44a) that might do the trick! python-bitcoin-blockchain-parser]This repository (https://github.com/alecalve/python-bitcoin-blockchain-parser) actually looks a bit better to me, since it has more features and it's Python 3-based so you won't need to compile it! Those script haven't updated in a while, so i have concern it can't parse taproot transaction/block correctly. Bitcoin Core is safer option unless OP willing to test those script and also offer to return raw hex of the block. Code: getblock [HASH_OF_THE_BLOCK] 0 Title: Re: Methods of decoding a bitcoin block stored in windows Post by: n0nce on March 28, 2022, 12:34:17 PM If you mean decoding the block and such, honestly the best tool is just using Bitcoin Core which you seem to have installed. But if you want to decode individual blk files, I found this small program (https://gist.github.com/ysqi/62820e5955a3e3b56efb8b396585f44a) that might do the trick! python-bitcoin-blockchain-parser]This repository (https://github.com/alecalve/python-bitcoin-blockchain-parser) actually looks a bit better to me, since it has more features and it's Python 3-based so you won't need to compile it! Those script haven't updated in a while, so i have concern it can't parse taproot transaction/block correctly. Bitcoin Core is safer option unless OP willing to test those script and also offer to return raw hex of the block. Code: getblock [HASH_OF_THE_BLOCK] 0 Title: Re: Methods of decoding a bitcoin block stored in windows Post by: supermankid on March 28, 2022, 04:08:03 PM Thanks for all your valuable insights. My ₿ journey gets more interesting everyday. A little late but I guess better late than never...
Title: Re: Methods of decoding a bitcoin block stored in windows Post by: ymgve2 on March 29, 2022, 02:43:15 AM One important thing to note is that those .blk files do not necessarily contain the blocks in the correct order, as they are written to the files as they are received, and during bootstrapping and syncing, lots of blocks are coming all at once. The files also might contain orphaned blocks.
|