An approach to recover change addresses from old wallet.dat backups

[nestex_one]

We made this little guide that may help people recover funds lost to change addresses, from their old wallet.dat backups

https://trade.nestex.one/docs/recover_old_wallet.pdf

You don't actually need to move funds to test if this methodology works! You can also 'test' the premise noted in the guide by creating a wallet (labelled 'wallet A' as per the guide), backing it up, and then generating many more addresses and now you have a 'wallet B'. Then restore 'wallet A' on a completely different computer and you should have the private keys that match 'wallet B', which means recovery complete!

- If you do share this guide, please do so under the CC BY license and try to use this PDF as-is.
- If it does add value we'd like to hear back from you! No we don't need a donation. Just tell me it worked for you, that's my happiness.

The intent here is to help people out. There is no commercial perspective here.

Also, many thanks to bias for guiding me with the right way & place to share this. Here's a big THANK YOU to an awesome member of the bitcointalk community!

We made this little document that may help people recover funds lost to change addresses, from their old wallet.dat backups

https://trade.nestex.one/docs/recover_old_wallet.pdf

Hope this helps someone!

IMO, anything that can help out anyone is always welcome. Especially if we are talking about such a serious issue. Maybe it would be good to make an additional post in another section of the forum that has more visibility. Like Beginners & Help, or Development & Technical Discussion for BTC.

[odolvlobo]

If change addresses are not automatically recovered from a backup, that seems like a serious bug/shortcoming in Bitcoin Core to me.

A couple of questions.

Remember you MUST have the public keys available to you (you can find this in the explorer, ...

You write that you need the public keys, but the instructions never use them. When you wrote "public key", did you mean "address"?

How do you find the public keys (or perhaps addresses) generated by the wallet in an explorer?

[BitMaxz]

It seems you're a bit confused. What do you mean by this: "Works for Bitcoin-like UTXO QT wallets only"? Why UTXO is included? that's unspent transactions. If you're talking about a wallet, then it must be Bitcoin QT or Bitcoin Core.

I think you'd better change or correct the guide from your PDF file.

And the thing is, why would you import a  public key just to recover the change address? The change addresses should be recorded on wallet.dat and can be able to recovery by dumping all keys from your wallet.dat using pywallet.

If you have an unencrypted wallet.dat file by using Notepad, you should be able to find those change addresses under the pre-generated keys reference code below, and next to that it will show you the private key and addresses.

Code:

# These are pre-generated keys ready for future transactions (either for receiving or change).

If you were talking about a corrupted wallet.dat, I think the only solution to recover the keys from that wallet is to salvage the wallet.

[nestex_one]

If change addresses are not automatically recovered from a backup, that seems like a serious bug/shortcoming in Bitcoin Core to me.

A couple of questions.

Remember you MUST have the public keys available to you (you can find this in the explorer, ...

You write that you need the public keys, but the instructions never use them. When you wrote "public key", did you mean "address"?

How do you find the public keys (or perhaps addresses) generated by the wallet in an explorer?

Public Key = Wallet Address for this context. The instructions do indeed use them in point #4

The REASON you should have this on hand is so you can be clear how much $$ you have in those.

To find your change address, you can open the explorer of your coin and track the change addresses used when you sent money from your original wallet address (which you know because it's present in Wallet 'A').

And the thing is, why would you import a  public key just to recover the change address? The change addresses should be recorded on wallet.dat and can be able to recovery by dumping all keys from your wallet.dat using pywallet.

If you have an unencrypted wallet.dat file by using Notepad, you should be able to find those change addresses under the pre-generated keys reference code below, and next to that it will show you the private key and addresses.

If you were talking about a corrupted wallet.dat, I think the only solution to recover the keys from that wallet is to salvage the wallet.

You're speaking of a different scenario. If you have Wallet B and you're able to dump keys or recover them you don't need to use this method. Please review the scenario I've mentioned in the PDF, it emphasizes that you have addresses for which the private key was not backed up and is not recoverable. Or the latest wallet.dat is lost (disk failure or suchlike)

[achow101]

This guide makes no sense. There is no need to call importaddress for every change address; all of the change addresses are already, or will be, in the wallet after keypoolrefill. All that will probably happen is that you confuse users because any of the change that is detected via the import will be shown as watchonly first, until the private key is generated.

If you have a backup of your wallet, all you have to do is perform a rescan. Hell, most of the time you don't even need to explicitly rescan because the wallet will rescan by itself when it sees it is not up to the chain tip. You literally just have to load it and wait.

[nestex_one]

This guide makes no sense. There is no need to call importaddress for every change address; all of the change addresses are already, or will be, in the wallet after keypoolrefill. All that will probably happen is that you confuse users because any of the change that is detected via the import will be shown as watchonly first, until the private key is generated.

If you have a backup of your wallet, all you have to do is perform a rescan. Hell, most of the time you don't even need to explicitly rescan because the wallet will rescan by itself when it sees it is not up to the chain tip. You literally just have to load it and wait.

You're talking about a different case where you have a backup of the latest wallet. Which doesn't match the scenario I've stated.

I did come across this issue for an altcoin and had to do this in order to have the wallet retain valid addresses. Rescan (starting from an old wallet) and Salvage (on the corrupted wallet in my case) did not yield results.

However importing the addresses in advance before keypoolrefill resulted in the right outcome and I did recover all of it.

But if it doesn't make sense to you feel free to not use it! Thanks for taking the time to check it out

[achow101]

You're talking about a different case where you have a backup of the latest wallet. Which doesn't match the scenario I've stated.

No, I'm talking about a backup of a HD wallet, made at any time.

I have, in fact, either written or reviewed all of the Bitcoin Core wallet code.


This guide feels very much "do a bunch of magic and it magically works", without understanding why any of it actually works.

I would expect that, if rescan did not work, a keypoolrefill followed by a rescan would be enough. Importing anything should be completely unnecessary. I also want to point out that importing your change addresses will result in breaking change detection.

[nestex_one]

I have, in fact, either written or reviewed all of the Bitcoin Core wallet code.

Fair enough, and I can confirm I've not encountered this scenario on Bitcoin Core or Litecoin or any of the more popular (and thus better maintained?) crypto wallets.

One specific usecase where keypoolrefill and rescan did not result in an outcome was with a lesser known altcoin named 'Shibacoin'. When I had a wallet break it basically 'lost' all funds that were on change addresses. Repeated keypoolrefills and rescans didn't do much better.

However importing the change address where the funds were sitting (as checked from the explorer) brought results.

I didn't completely understand your point about 'change detection' breaking by using this method. Can you elaborate?

[achow101]

I didn't completely understand your point about 'change detection' breaking by using this method. Can you elaborate?

Determining whether an address is change is done by some data being missing from the address book. Explicitly importing a change address will set that data, which means that the address will no longer be considered change. This results in some transactions being shown in a confusing manner where change is shown as a new incoming output rather than elided as change.