blk00000.dat strange format

[userkoen]

Hi, i installed the bitcoin core today but also one week ago and downloaded some blk*.dat files for learning purpose.

The blk*.dat files downloaded last week seem ok to me (for example blk00000.dat hex data is starting with f9beb4d9 1d010000 01000000 ... etc and f9beb4d9 marking each block throughout the file .dat )

but when i look into the blk00000.dat file i downloaded today (tested this on 2 computers)  i see strange series of bytes in the beginning of the file not starting with the f9beb4d9 etc ...and nowhere in the file thies magicbytes appear    )  Could this be an encoding thing ? Any clues ? I have no idea what has been changed on my side ? Just injstaleld the same software twice. I would have expected an identical blk00000.dat.
thx for any replies ! reg k

ps .. the blk00000.dat file downloaed today starts with thie binary sequence .. (which i totally dp not recognize)
02 5C 0E E9 C2 DF AD 9E  FA E2 BA 30 DF DE AD 9E
FB E2 BA 30 DF DE AD 9E  FB E2 BA 30 DF DE AD 9E
FB E2 BA 30 DF DE AD 9E  FB E2 BA 30 E4 7D 40 63
81 99 A8 82 A5 19 81 A0  9C 94 35 51 A0 16 B6 5D
73 68 EB 02 E5 41 15 34  B0 FC E4 7A F6 75 F2 D7
04 1D BA 2D C2 72 86 E2  FA E3 BA 30 DF DF AD ... etc

[ABCbits]

but when i look into the blk00000.dat file i downloaded today (tested this on 2 computers)  i see strange series of bytes in the beginning of the file not starting with the f9beb4d9 etc ...and nowhere in the file thies magicbytes appear    )  Could this be an encoding thing ? Any clues ? I have no idea what has been changed on my side ? Just injstaleld the same software twice. I would have expected an identical blk00000.dat.

IIRC there are at least 2 different reasons of this behavior.
1. Bitcoin Core obfuscate the data using XOR operation, with randomly generated key.
2. Bitcoin Core doesn't guarantee block is written into storage drive in deterministic/sequential order.

[userkoen]

hi ok, but the files (the whole blockchain) downloaded last week (all blk*.dat) with same core sw are all prefectly 'readable' without undoing the xor ...  In the sense that they start with the magic bytes / version etc    Why would that be the case .. ?

I know that he actual blocks are not perfectly in sequential order in the blk*.dat files but thats no issue.

I see inded an xor.dat file..

thx reg k

[ABCbits]

hi ok, but the files (the whole blockchain) downloaded last week (all blk*.dat) with same core sw are all prefectly 'readable' without undoing the xor ...  In the sense that they start with the magic bytes / version etc    Why would that be the case .. ?

It could happen due to one of these reasons,
1. Using Bitcoin Core with version below v28.0.
2. Start download blockchain with Bitcoin Core with version below v28.0.
3. Use Bitcoin Core v28.0 or newer, but manually add blocksxor=0 on your Bitcoin Core configuration before starting download blockchain.

More information, https://learnmeabitcoin.com/technical/block/blkdat/, XOR section.

[userkoen]

Ok, found it! .. today i used core 28.1 which gives xor'ed dat files (default in 'local' directory on windows) ... also 28.0 will give you xor'ed dat files...

and i think last week i used 27.2 (defaults to 'roaming' directory on windows)..

As we speak i re-installed core 27.2 and the blk*.dat files are not xor'ed on disk but directly useable..

Thks ! reg koen

hi ok, but the files (the whole blockchain) downloaded last week (all blk*.dat) with same core sw are all prefectly 'readable' without undoing the xor ...  In the sense that they start with the magic bytes / version etc    Why would that be the case .. ?

It could happen due to one of these reasons,
1. Using Bitcoin Core with version below v28.0.
2. Start download blockchain with Bitcoin Core with version below v28.0.
3. Use Bitcoin Core v28.0 or newer, but manually add blocksxor=0 on your Bitcoin Core configuration before starting download blockchain.

More information, https://learnmeabitcoin.com/technical/block/blkdat/, XOR section.

thx, just found out the same ! reg k (gues used 27.2 last week .. cant really remember ..