Offline resync issues - Error: Failed to rescan the wallet during initialization

[blade_house888]

Bitcoin Client Software and Version Number:
Bitcoin Core version v0.21.0 (release build)

Operating System:
Live CD: Ubuntu 14.04.3 LTS, x86_64-little_endian-lp64

System Hardware Specs:
Intel Core i5-2520M 2.5GHz
8G ram
no internal storage
external ssd 500GB (with bitcoin data files)
external flash drive 8GB (with wallet.dat file)

Description of Problem:
I am trying to load a wallet.dat on an offline node using a copy of the blockchain I have on an external, and getting an error on rescan.  My goal is to use the offline node with my cold wallet to create a transaction using the blockchain data from the external that I will then bring to an online node to broadcast.

I start bitcoin-qt with a datadir and walletdir set, and also the --rescan flag.  Rescanning took ~4hours (tried twice with same result), then I get an error and it exits.

Log Files from the Bitcoin Client:
https://dpaste.org/pNpv

Error from the log:

Code:

2021-04-01T13:25:45Z [default wallet] Still rescanning. At block 675765. Progress=0.995148
2021-04-01T13:26:15Z [default wallet] AddToWallet <snip my tx id - it's the right txid for my coins>  
2021-04-01T13:26:45Z [default wallet] Still rescanning. At block 676606. Progress=0.997801
2021-04-01T13:27:25Z [default wallet] Rescan completed in        15039816ms
2021-04-01T13:27:25Z Error: Failed to rescan the wallet during initialization
2021-04-01T13:28:20Z Shutdown: In progress...

Any ideas?

[DaveF]

With the standard make a backup of the wallet.dat file.
Try starting with -salvagewallet see if that works.

If not take a look at pywallet: https://bitcointalk.org/index.php?topic=34028.0 and see if you can pull your private keys.

As always make several backups and never work with the original file.

-Dave

[blade_house888]

With the standard make a backup of the wallet.dat file.
Try starting with -salvagewallet see if that works.

If not take a look at pywallet: https://bitcointalk.org/index.php?topic=34028.0 and see if you can pull your private keys.

As always make several backups and never work with the original file.

-Dave

Thanks for the reply.  It looks like salvagewallet has been removed: https://bitcoincore.org/en/releases/0.21.0/:

The -salvagewallet startup option has been removed. A new salvage command has been added to the bitcoin-wallet tool which performs the salvage operations that -salvagewallet did. (#18918)

I guess  I'll have to use the bitcoin-wallet tool with that - after looking at that man page it states salvage is experimental :/

So you think this is most likely a corrupt wallet file?

[HCP]

Offline setups aren't really meant to operate with a copy of the blockchain... you'd supposed to just create the unsigned transaction on your online machine creating the PSBT... then transfer the PSBT (Which includes all the necessary info) to the offline machine, and use the offline machine wallet to simply sign the transaction using the private keys.

It's possible that the offline node is attempting having issues as it might have an incomplete copy of the chain and so it thinks it cannot rescan properly.

Why are you trying to use the blockchain data on the offline machine?

[blade_house888]

Why are you trying to use the blockchain data on the offline machine?

My thought was to rescan the blockchain on my offline machine with my cold wallet so I could use qt to create the transaction, then transfer the transaction to my online node to broadcast.  However, I should look into PSBT - is it pretty easy to use in this fashion?  Also, do you know if PSBT would work if the cold wallet is from ~2013, and the online machine creating the PSBT is current core?

[HCP]

Also, do you know if PSBT would work if the cold wallet is from ~2013, and the online machine creating the PSBT is current core?

Is the offline wallet running the same version of Bitcoin Core as the online machine? If so, then it should work fine with PSBT. If not... then the offline machine likely won't be able to understand what the PSBT is (depending on how old the offline version of Bitcoin Core is)

The age of the wallet.dat file itself likely won't matter... as it will be "upgraded" when it is loaded into the newer version of Bitcoin Core. It's the version of the Bitcoin Core application itself that is the important factor here.

[blade_house888]

Also, do you know if PSBT would work if the cold wallet is from ~2013, and the online machine creating the PSBT is current core?

Is the offline wallet running the same version of Bitcoin Core as the online machine? If so, then it should work fine with PSBT. If not... then the offline machine likely won't be able to understand what the PSBT is (depending on how old the offline version of Bitcoin Core is)

The age of the wallet.dat file itself likely won't matter... as it will be "upgraded" when it is loaded into the newer version of Bitcoin Core. It's the version of the Bitcoin Core application itself that is the important factor here.

Yes it will be the same version - my offline machine is just a live cd boot of ubuntu, then I unpack the latest core transferred via flash drive.

[blade_house888]

you'd supposed to just create the unsigned transaction on your online machine creating the PSBT...

How exactly do you do this from the online machine with out the cold wallet?

[HCP]

Looking into this in more detail, it seems that attempting to use Bitcoin Core in an Online/Offline "air gap" setup could be quite problematic... mainly due to the fact that Bitcoin Core uses "hardened" key derivation. This means you can't simply import an "xpub" into your online wallet and generate the addresses as required.

Instead, you have to import addresses manually into a watching-only wallet on the "online" machine. To create the watching-only wallet you select the "disable private keys" option when creating the wallet.

This stack exchange has a fairly good description of the process (using 0.21.0 and PSBTs): https://bitcoin.stackexchange.com/a/101595


Another option would be using Electrum with the added bonus that it allows you to create watching only wallets from "xpubs", so you don't need to manually import all the addresses. You just create the watching only wallet using the master public key (xpub) and it will be able to automatically generate all the addresses from your offline wallet.

Alternatively, it might be possible to create a non-hardened offline wallet in Bitcoin Core by utilising the "descriptor" functionality that was recently added... I've not had much luck getting it set up though.

[NotATether]

I start bitcoin-qt with a datadir and walletdir set, and also the --rescan flag.  Rescanning took ~4hours (tried twice with same result), then I get an error and it exits.

~snip

Error from the log:

Code:

...
2021-04-01T13:27:25Z Error: Failed to rescan the wallet during initialization

Have you ever aborted the rescanning process yourself? Because that's where the above error comes from.

Also, do you know if PSBT would work if the cold wallet is from ~2013, and the online machine creating the PSBT is current core?

PSBT's have been added in 0.17 and that particular version was released in December 2018. So both nodes obviously need to be running 0.17+. Any wallet should work as long as you start Bitcoin Core with -upgradewallet option or just type the upgradewallet command in the Console once your wallet file is loaded. It's going to upgrade your wallet file to the latest format that your version of Core is using.