why every blk00000.dat is different in every wallet?

[bigphoenixman]

I am a new one studying bitcoin source. I think that maybe writing a blockchain parser is first step. I realize that every blk00000.dat is different.  I got 2 different blk00000.dat.  I compared the files as below.
http://chuantu.biz/t6/152/1511169018x-1566660940.png
You can see the Magic Number, Size, Version, and the PreHash are different.
I had read https://github.com/bitcoin/bitcoin/issues/6613, but I am still confused.
If there is a key to XOR to make difference, why the blocks before do not XOR? And obviously, Magic Number, Size, Version do not XOR, which field will be XOR?

Thanks a lot!!!!
 

[achow101]

Bitcoin Core downloads and stores in blocks in the order that they are received, not the order that they are in the blockchain. Bitcoin Core downloads multiple blocks simultaneously, so it is likely that blocks will be received and written to disk in a different order from the actual blockchain and in a different order from node to node.

[aestrum]

Content is XORed with random key, so that antivirus software does not trigger.

[achow101]

Content is XORed with random key, so that antivirus software does not trigger.

The block files themselves are not XORed with the random key; the databases are.

[aestrum]

Yes, right, I confused it with chainstate db, which is XORed.

[jnano]

Wouldn't it make sense to normalize the block storage format?
That would make it practical to do various operations between computers, like fill missing data or compare/verify.

Why XOR the chainstate data or (index?) database files?

[achow101]

Wouldn't it make sense to normalize the block storage format?
That would make it practical to do various operations between computers, like fill missing data or compare/verify.

The block storage and file formats are normalized. What you can't do is ensure that every single node has exactly the same data in exactly the same order. This is because during normal operation, orphan blocks will cause nodes to receive different blocks in different orders. Some nodes might see a block which another node does not see at all. Or they might see two different blocks and write them in the same position. Because of orphans, it is not possible to make all nodes have exactly the same data in exactly the same order.

During initial sync, having to write the blocks to disk in blockchain order rather than order received will result in a slower sync.

Why XOR the chainstate data or (index?) database files?

To prevent antivitus software from messing with the database files because sometimes they contain virus signatures because virus signatures are embedded into the blockchain. Having the database files corrupted will require the entire database to be rebuilt which will take a long time.

[jnano]

Well, maybe it could have an optional flag to do store blocks in final blockchain order, or to trigger a one-time normalization pass on existing data.
And if orphan blocks are needed, keep them separate from the main chain.

I thought the AV-triggering data would be in the block files. Or what did you mean by databases?
Is the chainstate data XORed as well? I thought that's just the global UTXO balance, with no arbitrary data.

[bigphoenixman]

Bitcoin Core downloads and stores in blocks in the order that they are received, not the order that they are in the blockchain. Bitcoin Core downloads multiple blocks simultaneously, so it is likely that blocks will be received and written to disk in a different order from the actual blockchain and in a different order from node to node.

Thank you