Rescan fails with old wallet

[jungle]

I have a synced-up blockchain, Bitcoin Core starts with no issues with an empty wallet, the same blockchain (which I keep rsynced) has no issues on a linux box running bitcoind. When I replace the empty wallet.dat file with my old wallet.dat file that has some bitcoin in it, the client rescans on startup and fails. It worked fine with the exact same wallet file last year. I suspect there's a corrupt block but don't understand why neither bitcoind nor bitcoin-core have no issue when starting with empty wallets. Also, I don't see a way to fix the corrupt block without downloading the blockchain from scratch, which would probably take months. I searched the net for the error, found zilch.


Bitcoin Client Software and Version Number: Bitcoin Core version v0.20.0
Operating System: MacOS version 10.13.6
System Hardware Specs: Macbook Pro 2013 2.6 Intel Core i5 8GB RAM 4TB usb drive
Description of Problem: Rescan fails with old wallet
Any Related Addresses: None
Any Related Transaction IDs: None
Screenshot of the problem: None
Log Files from the Bitcoin Client: https://pastebin.com/SfBWPcKt

[jackg]

So the Linux box version is working fine? Have you got the full blockchain there? If so, you c I uld try backing up your current data folder, replace it with the Linux box version and try again (with your same wallet dat) and try rescanning it to see if that'll work.

[jungle]

So the Linux box version is working fine? Have you got the full blockchain there? If so, you c I uld try backing up your current data folder, replace it with the Linux box version and try again (with your same wallet dat) and try rescanning it to see if that'll work.

Yes, I have the full blockchain on the linux box, and also rsynced to the Mac, so it's in both places. Both the bitcoind running on the linux box and the Mac bitcoin-core client have no problem with the blockchain when starting with an empty wallet (I just delete the wallet.dat file and restart jye client), so in principle, as far as I know, the blockchain should be ok.

But when I start the client on the Mac with my old wallet.dat file, it rescans the blockchain starting from block 470713 and fails before reaching the current height.

This never happened before, with the same exact setup and wallet.dat file. The only thing I can think of is that the blockchain got somehow corrupted, but I don't understand why neither client seems to notice with an empty wallet. Also, I don't know how to fix it. Downloading the whole thing from scratch is only a desperate last resort option given how long it takes. There must be a better option.

[jackg]

But if it launches on the virtual os fine then you should be able to. Just copy acrosd the data from there and your mac version will work if noth are fully synced without having to reindex (backup both chains first though if you can).

[jungle]

There's no virtual OS. I have a linux box and a Mac. Two computers.

1. On the linux box I run bitcoind (this is the bitcoin node) and it keeps the linux box copy of the blockchain constantly up to date.

2. Every once in a while I copy the blockchain from the linux box to the Mac. Since copying everything is very slow, I use a tool called rsync that compares files in the two locations and copies or deletes anything it needs to ensure they are identical. This takes a few minutes. So the Mac copy of the blockchain is identical to the linux copy. In case anyone is wondering, I don't rsync the blockchain while the bitcoind daemon is updating it. I have a daily crontab process on the linux box that shuts down the daemon, makes a backup of the blockchain (again using rsync) on the same disk, and restarts. The Mac rsyncs from the backup, so it's in a consistent state.

3. In other words, the Mac has a perfect copy of the blockchain that runs on the linux box.

4. Both machines run the bitcoin node without issues on this blockchain with empty wallet.dat files that have no transactions on them.

5. When I replace the empty wallet on the Mac with a wallet that has transactions on it, bitcoin-core rescans starting at block 470713 and fails.


Next steps that I'm trying or going to try:

1. Rescan the entire blockchain. It's already on the way, took a few hours but it's about to finish.

2. Try that wallet on the linux box.

I'll report any progress here.

[LoyceV]

I suspect there's a corrupt block but don't understand why neither bitcoind nor bitcoin-core have no issue when starting with empty wallets.

I would do a full rescan with the empty wallet. See if that gives a similar error.

Question: did you shut down Bitcoin Core (on both systems) before (and during) rsync?
You answered this already while I was typing.

Also, I don't see a way to fix the corrupt block without downloading the blockchain from scratch

That is correct.

which would probably take months.

Depending on your hardware and internet speed, it should take much less time than that.

The only thing I can think of is that the blockchain got somehow corrupted, but I don't understand why neither client seems to notice with an empty wallet.

Try rescanblockchain.

Also, I don't know how to fix it. Downloading the whole thing from scratch is only a desperate last resort option given how long it takes. There must be a better option.

If it's damaged, redownloading is the way to go.

[jungle]

I would do a full rescan with the empty wallet. See if that gives a similar error.

I was already doing that, but with the non-empty wallet. It just finished. And what do you know, it worked.

No idea why this happened. I guess I need to learn more about what rescan does.

I compared a copy of the wallet.dat file from before the rescan with the one after the rescan, and they are different. So I'm storing a fresh copy of the file in my backup.

Case closed. Thanks for helping.

[keychainX]

I would do a full rescan with the empty wallet. See if that gives a similar error.

I was already doing that, but with the non-empty wallet. It just finished. And what do you know, it worked.

No idea why this happened. I guess I need to learn more about what rescan does.

I compared a copy of the wallet.dat file from before the rescan with the one after the rescan, and they are different. So I'm storing a fresh copy of the file in my backup.

Case closed. Thanks for helping.

all old wallet.dat files change in size after a rescan if they have not been scanned for a while