|
Title: natural order of blocks in blk-files? Post by: amaclin on October 29, 2016, 11:29:15 AM Is there a way to force client 0.13.1 to store blocks in blk-files in their "natural order"?
Title: Re: natural order of blocks in blk-files? Post by: 2112 on October 29, 2016, 05:19:13 PM Is there a way to force client 0.13.1 to store blocks in blk-files in their "natural order"? Did you try using "-connect" to force talking to just one peer?I haven't tried 0.13.1, but with some older versions that resulted in "reproducible ordering", I don't recall if it was "natural order". Title: Re: natural order of blocks in blk-files? Post by: gmaxwell on October 31, 2016, 12:16:31 AM Is there a way to force client 0.13.1 to store blocks in blk-files in their "natural order"? No. Not reliably, but you can use the script in contrib/linearize to get a linearized copy.You shouldn't generally assume that you can parse Bitcoin's internal datastructures, they're not a supported interface and may change from version to version. Title: Re: natural order of blocks in blk-files? Post by: amaclin on October 31, 2016, 07:57:17 AM Is there a way to force client 0.13.1 to store blocks in blk-files in their "natural order"? No. Not reliably, but you can use the script in contrib/linearize to get a linearized copy.Title: Re: natural order of blocks in blk-files? Post by: pepethefrog on November 03, 2016, 08:07:58 AM Is there a way to force client 0.13.1 to store blocks in blk-files in their "natural order"? No. Not reliably, but you can use the script in contrib/linearize to get a linearized copy.You shouldn't generally assume that you can parse Bitcoin's internal datastructures, they're not a supported interface and may change from version to version. That is good to know. Pepe has seen some altcoins that have added a 4 byte index to the block headers to number the blocks according to their "height". This way you can atleast know (during the linear parsing of the chain) that you stay "on track". Title: Re: natural order of blocks in blk-files? Post by: amaclin on November 03, 2016, 08:30:55 AM That is good to know. This is not a problem. Blocks are linked together with their hashes. No need to check the height field (and this field doesn't exists in early blocks)Pepe has seen some altcoins that have added a 4 byte index to the block headers to number the blocks according to their "height". This way you can atleast know (during the linear parsing of the chain) that you stay "on track". Checking "staying on track" is very easy. Some pseudocode: Code: void process ( const Block& b ) The problem is "what to do with the block loaded from blk-file if it is not possible to link it right now to its parent, because the parent is below in blk-file" Title: Re: natural order of blocks in blk-files? Post by: pepethefrog on November 03, 2016, 09:23:04 AM That is good to know. This is not a problem. Blocks are linked together with their hashes. No need to check the height field (and this field doesn't exists in early blocks)Pepe has seen some altcoins that have added a 4 byte index to the block headers to number the blocks according to their "height". This way you can atleast know (during the linear parsing of the chain) that you stay "on track". Checking "staying on track" is very easy. Some pseudocode: Code: void process ( const Block& b ) The problem is "what to do with the block loaded from blk-file if it is not possible to link it right now to its parent, because the parent is below in blk-file" Yes ofcourse, but since the blocks are already in the blk-files, we can assume that the full bitcoin node has already checked and validated their hashes. Pepe thought that this thread is about whether you can assume that the blocks inside the blk-files are stored in linear order or not? Now, if they are not, it will require you to either know the index structure to be able to quickly look up the position of the next block, or, to have a numbered index in the block header of the block to quickly check the linear progression. Looking up a number is way faster than calculating and comparing hashes. Just Pepe's 2 cents Title: Re: natural order of blocks in blk-files? Post by: amaclin on November 03, 2016, 10:03:27 AM Now, if they are not They are not.Quote , it will require you to either know the index structure to be able to quickly look up the position of the next block, Too difficult for single-pass parserFor example, the block order is ... 99, 100, 101, 200, 102, 103, ... 199, 201, 202, 203... The easiest way for blockchain parser is to keep in memory the blocks which can not be added to a by asking bitcoin client save them in right order. OK, it is not possible today. I've already written a code for linearization-on-the-fly. The topic can be closed. |