|
Title: Do I have a corrupt blk00000.dat? Post by: JuxtaposeLife on August 01, 2025, 02:34:44 AM I've been running a full node for awhile now, with no issues. It has been synced and reliably keeping up with new blocks. I was attempting to more quickly traverse the local data history without RPC using blockchain_parser.blockchain to ingest into a new utxos table for analysis, but whenever it tried to index the raw data the CPU spiked to almsot 100% for hours with no progress. I think I discovered the problem is that it's hung up on the first block
~/.bitcoin/blocks/blk00000.dat starts with: Code: 00000000 3d 11 2a 31 ab c4 8b 90 |=.*1....| that's not right, is it? Is there a way to force an update of just that block... Curious if others have a better suggestion for how to run through the data. I'm also wondering how my initial block got like this? Edit: Doing some more investigating, it seems all of the blocks (at least the first 5) start with thsoe same bytes. Just checked the entire folder and ever single file blk*.dat starts with the magic string 3d112a31 I think this means I need to redownload/sync my entire history? Still confused how this could happen... has anyone ever run into this before? Appreciate and insights that help me avoid having to go through this resync in the future. fwiw, I'm running Bitcoin Core version v28.99.-0 on Ubuntu 24.04.1 LTS Title: Re: Do I have a corrupt blk00000.dat? Post by: nc50lc on August 01, 2025, 07:27:24 AM blockchain_parser.blockchain may not be designed to correctly parse the new XOR'd blocks of Bitcoin v28.0+
Check its repo if there are reported issues related to it. Once you confirm that it's the issue, you may restart with a fresh data directory but make sure that blocksxor=0 is set in your config file or use it as a command line arg. Title: Re: Do I have a corrupt blk00000.dat? Post by: JuxtaposeLife on August 02, 2025, 01:36:44 AM Appreciate that tip (I've added blocksxor=0), I also discovered I wasn't running a stable release of 28, so I'm updating to 29.0
Downloading and syncing again. Should fix it. Title: Re: Do I have a corrupt blk00000.dat? Post by: Cricktor on August 02, 2025, 01:47:10 PM Wouldn't it be faster to "un-XOR" existing blockchain data files if some of your tools can't cope with non-zero XORed block files, instead of repeating an IBD from scratch with blocksxor=0?
The file xor.dat in Core's blocks folder stores the used obfuscation 8-byte XOR value. It shouldn't be very difficult to write a tool to do the un-XORing. I haven't tried it myself... |