Bitcoin Core wallet.dat restore shows wrong amount.

[Tanquen256]

I have a VM with Bitcoin Core Wallet and forgot the password. I have a backup of the DAT file that was from before I encrypted it. I setup a new install and used the DAT and it has most of the transactions and once it synced and download all the block chain it shows all the transactions but that there are only 0.013... and the VM is also synced but shows more coins. All the same transactions are listed but the amount left is not the same on the old DAT restore. Are there other files than the DAT needed?

_____________________________________________________________

Sorry, not all the same transactions are listed in both. The old wallet is missing the last few transactions and is a about 4 months older than the last transaction in the encrypted wallet.

[1714009615]

1714009615

[1714009615]

1714009615

[1714009615]

1714009615

[1714009615]

1714009615

[1714009615]

1714009615

[HCP]

When the wallet.dat gets encrypted... for safety reason, it resets the seed/keypool, generates a bunch of new keys/addresses and then "archives" the old keys/addresses (marks them as inactive so they don't get "used" when you generate a new address to receive or for change).

That might explain why your old (pre encryption) backup is not showing all the transactions... new keys/addresses would have been generated after the encryption was set and coins have been received on the new addresses that the old wallet.dat does not have.

However, it's interesting that you say both wallets show exactly the same transactions? But a difference balance? I could only guess that it's because you sent some coins... and the old pre-encryption wallet does not have the change address from the new post-encryption wallet... so, it thinks that both UTXOs created were going to external wallets.

You could probably confirm that by using the getaddressinfo command (on the console in Bitcoin Core) with the "change" addresses shown in the most recent send transactions on both instances of Bitcoin Core and checking the "ismine" value... it will probably say "true" on the VM (with the encrypted version of the wallet.dat)... and "false" on the old unencrypted version (as it doesn't have those keys/addresses).

[Tanquen256]

I can try but I thought I saw something say the console is locked as well.

Would there be anyway to fix the old (pre encryption) backup so I can send the remaining coins or are they lost?

The said part is I think I have had to do this before and I guess I got lucky using the old DAT file before. It was a number of years ago and my memory is shot.

I saw something about running a rescan command of some kind after restoring the DAT but it seems like the Core Wallet is doing that anyway when it see a new DAT file. It takes an hour or more on the logo screen saying "Rescanning...".

[HCP]

I can try but I thought I saw something say the console is locked as well.

The console should not be "locked". The only things you won't be able to do without the passphrase is anything requiring the decryption of the keys (like signing transactions/messages or dumping the wallet keys etc)

Would there be anyway to fix the old (pre encryption) backup so I can send the remaining coins or are they lost?

Unfortunately no... even if you added a passphrase to an old copy of the unencrypted wallet.dat, it would just end up generating a different set of keys/addresses.

I saw something about running a rescan command of some kind after restoring the DAT but it seems like the Core Wallet is doing that anyway when it see a new DAT file. It takes an hour or more on the logo screen saying "Rescanning...".

Correct. It will have already rescanned the wallet.


As this point, your best option would be to try and recover your wallet passphrase using something like: https://github.com/3rdIteration/btcrecover/

Or engaging the services of Dave @ wallet recovery services.
refer: https://bitcointalk.org/index.php?topic=240779
and: http://walletrecoveryservices.com/

[Tanquen256]

Thanks for the info. I thought I was just locking access to the wallet not the coins to the wallet as well.

I thought I remembered the pass phrase but nothing is working.

My memory is so bad now I'm not sure what I did.

I think I thought having a (pre encryption) backup would work if I forgot the VMs pass phrase and maybe it would have if I made a backup without encryption after each transaction.

I found a newer backup that is scanning now but it is missing the last few transaction that the VM shows. So I guess it will also show the incorrect amount.

[OgNasty]

Wallets like Electrum have the option in settings to not use change addresses. This helps make backup a lot more simple, but you give up a certain amount of financial privacy in doing so. Still, change addresses are a tricky thing for people to comprehend and I would advise new users who don’t comprehend the way change addresses work to take advantage of features that will keep your coins from ending up at change addresses. Even if you manage your change addresses properly, you’d have to constantly be backing them up in order to keep from losing your funds in the event of a hard drive crash.

[HCP]

I'm not sure about the old old versions of Bitcoin Core/QT... but the newer ones definitely carry a warning to let you know that when you add encryption to a wallet.dat, as soon as the encrypted wallet is "used", old unencrypted backups will become "useless":


I hope it isn't a huge loss

Wallets like Electrum have the option in settings to not use change addresses. This helps make backup a lot more simple, but you give up a certain amount of financial privacy in doing so. Still, change addresses are a tricky thing for people to comprehend and I would advise new users who don’t comprehend the way change addresses work to take advantage of features that will keep your coins from ending up at change addresses. Even if you manage your change addresses properly, you’d have to constantly be backing them up in order to keep from losing your funds in the event of a hard drive crash.

That isn't really an issue with a wallet like Electrum... all addresses (receive and change) are recoverable from the seed. So, as long as you still have the seed, you can recover everything.

The same applies to a wallet like Bitcoin Core... the issue here is that the OP effectively changed the "seed" by adding encryption to the wallet (rendering old unencrypted backups useless)... and then lost/forgot the passphrase to the wallet... effectively removing their access to the seed/private keys required to access the change addresses.

This isn't really a "change" address issue, per se... it's a lost passphrase problem.

[Tanquen256]

Not a huge loss and I don't recall a warring like that but it was about 3 years ago.

Again I could have sworn I'd recovered from a forgotten password this way before so I thought I was ok having an unencrypted backup.

You can't take it with you and I was struggling to think of something to buy with it anyway.

Just makes me mad that I can remember a few days ago anymore let alone 3 years ago. I should have put the pass word someplace and so on.

[nc50lc]

I'm not sure about the old old versions of Bitcoin Core/QT... but the newer ones definitely carry a warning to let you know that when you add encryption to a wallet.dat, as soon as the encrypted wallet is "used", old unencrypted backups will become "useless":

Not a huge loss and I don't recall a warring like that but it was about 3 years ago.

Bitcoin core version 0.11.1 from June 2017 (almost 4 years ago) had the a similar warning, only the non-important part is different:


That warning will be displayed after you've encrypted your wallet, maybe you missed that.

[Tanquen256]

...
You could probably confirm that by using the getaddressinfo command (on the console in Bitcoin Core) with the "change" addresses shown in the most recent send transactions on both instances of Bitcoin Core and checking the "ismine" value... it will probably say "true" on the VM (with the encrypted version of the wallet.dat)... and "false" on the old unencrypted version (as it doesn't have those keys/addresses).

How do you do this part?

getaddressinfo (need to find the last change address and put here?)

When I try getaddressinfo by it's self it says "Method not found (code -32601)"

Looking at the last transaction I don't see a "change" addresses or "ismine" value.

I feel fairly sure the wallet on the VM is good as the amount is still the same after it synced up the last few years.

[BitMaxz]

How do you do this part?

getaddressinfo (need to find the last change address and put here?)

When I try getaddressinfo by it's self it says "Method not found (code -32601)"

Looking at the last transaction a "change" addresses or "ismine" value.

I feel fairly sure the wallet on the VM is good as the amount is still the same after it synced up the last few years.

getaddressinfo wouldn't work without adding address at the end you should look like this when you type the command on the console

Code:

getaddressinfo bc1qptg5vvtmwwgn96qgwgalpflqjvxxmvlwmu8529

The address there is my BTC as sample change address the result sample would be like this

Code:

 {
  "address": "bc1qptg5vvtmwwgn96qgwgalpflqjvxxmvlwmu8529",
  "scriptPubKey": "00140ad146317b739132e808723bf0a7e0930c6db3ee",
  "ismine": false,
  "solvable": false,
  "iswatchonly": false,
  "isscript": false,
  "iswitness": true,
  "witness_version": 0,
  "witness_program": "0ad146317b739132e808723bf0a7e0930c6db3ee",
  "ischange": false,
  "labels": [
  ]
}

Since my address is generated from Electrum and not bitcoin core ismine result is false as you can see above.

Sample this image below it was generated from my bitcoin core wallet.



As you can see ismine is true it means that address is connected to my core wallet.

Try to add the change address at the end and let see if the result is true or not. If it shows true then you have a chance to get the private key of the change address where your BTC missing.

[HCP]

getaddressinfo (need to find the last change address and put here?)

Correct... just try ALL of the output addresses associated with the last "send" transaction (or two) that you see in your "old" unencrypted backup wallet.dat

If my theory is correct... both addresses (for each send transaction) will show "ismine: false" when you use the getaddressinfo command on your old unencrypted wallet.dat, which would indicate that it is missing the "change" address keys.

Then try using getaddressinfo on the "encrypted" VM wallet.dat using those same output addresses. Again, if my theory is correct, one of the output addresses should show "ismine: true" to indicate that the encrypted wallet does have the "change" address keys.

It doesn't really help you recover the keys... but at least it will explain why your transactions are the same, but your balances are different... and it will confirm that you'll be unable to recover all the coins using your old unencrypted wallet.dat.

[Tanquen256]

What would be the correct address to use on blockchain.com to and make sure the encrypted wallet is correct?

____________________________________

So I use this command with my addresses from the last few transactions addresses.

getaddressinfo bc1qptg5vvtmwwgn96qgwgalpflqjvxxmvlwmu8529


Then I can see if ismine is true.


"Try to add the change address at the end and let see if the result is true or not. If it shows true then you have a chance to get the private key of the change address where your BTC missing."

How is this part done? "Try to add the change address at the end" And this is all on the unencrypted wallet?

____________________________________

Sorry I'm not great at this stuff. When I look at the last few transactions there is a To address and a Transaction ID and they don't work with the getaddressinfo command.

[HCP]

ok... so, what you want to do it find the most recent "send" transaction that is in the "encrypted" wallet... then right click the transaction and select "copy transaction id":



Once you have that TransactionID, you can lookup the transaction on a blockexplorer like blockchain.com, blockchair.com, blockstream.info, blockcypher.com etc... For this example, I'll use blockchain.com. Lets say that our TransactionID is: 8868a53fdf818ce4dda050b9e5992d9a61e3fe3da754b68b521e48acc94406bb

We put that into blockchain.com and we see this:


We can see on the right hand side, the "output" addresses:
- 1ETCkqqVd4CTffmZJyL3R18eGbVb44EXbU
- 3EEJFjZURxShNr2AoJtbfcvCB749yzP7LP

What you want to do is check all of the output addresses that you see for your transaction using the getaddressinfo command... on both the unencrypted and the encrypted wallet.dat's...

So, for our example transaction, we want to use the following commands on both the old and new wallet.dat's:

Code:

getaddressinfo 1ETCkqqVd4CTffmZJyL3R18eGbVb44EXbU

Code:

getaddressinfo 3EEJFjZURxShNr2AoJtbfcvCB749yzP7LP

If my theory is correct, on the old unecrypted wallet.dat... all of the output addresses will say "ismine:false"

like this:

Code:

getaddressinfo 1ETCkqqVd4CTffmZJyL3R18eGbVb44EXbU

{
  "address": "1ETCkqqVd4CTffmZJyL3R18eGbVb44EXbU",
  "scriptPubKey": "76a914938e43764d6068ff631f3d98d582d4e9e718c88a88ac",
  "ismine": false,
  "solvable": false,
  "iswatchonly": false,
  "isscript": false,
  "iswitness": false,
  "ischange": false,
  "labels": [
  ]
}

and this:

Code:

getaddressinfo 3EEJFjZURxShNr2AoJtbfcvCB749yzP7LP

{
  "address": "3EEJFjZURxShNr2AoJtbfcvCB749yzP7LP",
  "scriptPubKey": "a914898c095b5b02d8a3f736f6cf872c5f97c5eccefc87",
  "ismine": false,
  "solvable": false,
  "iswatchonly": false,
  "isscript": true,
  "iswitness": false,
  "ischange": false,
  "labels": [
  ]
}

However, when you check the addresses on the "new" encrypted wallet.dat... one of those addresses will display as "ismine: true" (and most likely "ischange: true" as well)... like this:

Code:

getaddressinfo 3EEJFjZURxShNr2AoJtbfcvCB749yzP7LP

{
  "address": "3EEJFjZURxShNr2AoJtbfcvCB749yzP7LP",
  "scriptPubKey": "a914898c095b5b02d8a3f736f6cf872c5f97c5eccefc87",
  "ismine": true,
  "solvable": false,
  "iswatchonly": false,
  "isscript": true,
  "iswitness": false,
  "ischange": true,
  "labels": [
  ]
}

[Tanquen256]

Thanks, that is great! Now I'm using the output address but I still get the same error.

Method not found (code -32601)

I'm using the debug window and the Console tab.

Is the getaddressinfo  command not available in v0.16.0?

________________________________________________

Ok, I have v0.21.1 installed on another VM and the command works now.

Trying them out now.

_________________________________________________

Do I need the block chain data to run the commands?

When I tried to switch the wallet I got an Open wallet failed error.

Wallet loading failed. Prune: last wallet synchronization goes beyond pruned data. You need to -reindex (download the whole blockchain again in case of pruned node)

That will take a few days.

_______________________________

Ok, on V21 they both say "ismine": false but that is with out the blockchain.

Do I really need to setup two VMs with v21 and download blockchain for each wallet?

[HCP]

When I tried to switch the wallet I got an Open wallet failed error.

Wallet loading failed. Prune: last wallet synchronization goes beyond pruned data. You need to -reindex (download the whole blockchain again in case of pruned node)

oh... you're running a pruned wallet... that could potentially be problematic with attempting to "sync" (ie. rescan) a wallet.dat file... Bitcoin Core can't scan blocks it doesn't have on disk... so if your node is pruned, it can't scan the pruned blocks... hence the error message about needing to "reindex" (and redownload the missing blocks).

However, using the getaddressinfo should still identify which address you have the private keys to... regardless of whether the wallet.dat is properly synced (rescanned) or not... and regardless of whether or not the node is pruned... and regardless of whether or not the node is fully synced (ie. all blocks downloaded/indexed)

Thanks, that is great! Now I'm using the output address but I still get the same error.
...
Ok, on V21 they both say "ismine": false but that is with out the blockchain.

How many output addresses where in the last send transaction? One... or two?

If there is only one, then it's likely an external address... so try going back to the next "Send" transaction... and find the output addresses from that and check those... you're really trying to find the most "send" transaction that has at least two output addresses.

[Tanquen256]

Is there a way to un-prune the wallet? Just let a new install of the Core Wallet download all the blockchain without a wallet loaded?

How to I verify the amount in encrypted wallet is correct with blockchain.com?

The last transaction has two output address. It's a little confusing as the last day on the unencrypted wallet I had some sent transactions that I was having an issue sending they and the receiver later returned one. The ID and amount are the same in both but the date/time stamp differs by 4 hours or so.

Unencrypted Wallet
Both outputs have "ismine": false.

Encrypted Wallet
First output has "ismine": false.

Second output has "ismine": true with more info like, "desc" and "pubkey".


BitMaxz had said there was a chance to update the Unencrypted Wallet? But I'm not sure what he is asking to do here.

"Try to add the change address at the end and let see if the result is true or not. If it shows true then you have a chance to get the private key of the change address where your BTC missing."

[LoyceV]

Is there a way to un-prune the wallet? Just let a new install of the Core Wallet download all the blockchain without a wallet loaded?

Correct. Just unselect Prune, and restart Bitcoin Core. It will take a while to download, verify and store 400 GB.

[Tanquen256]

How to I verify the amount in encrypted wallet is correct with blockchain.com?

I tried using the getaccountaddress "" command but when I put the output into blockchain.com I get this for the BTC address and BCH address:

This address has transacted 0 times on the Bitcoin blockchain. It has received a total of 0.00000000 BTC ($0.00) and has sent a total of 0.00000000 BTC ($0.00). The current value of this address is 0.00000000 BTC ($0.00).

Has it really been zeroed out somehow?

[BitMaxz]

BitMaxz had said there was a chance to update the Unencrypted Wallet? But I'm not sure what he is asking to do here.

"Try to add the change address at the end and let see if the result is true or not. If it shows true then you have a chance to get the private key of the change address where your BTC missing."

No, I didn't say that, I was actually focused on getaddressinfo command this command only use to get the info of the address and to know if your wallet.dat holds its private key.


If you can publicly post the whole transaction history here both from your new wallet and from the wallet in your VM then maybe we can analyze why you get 0.013 remaining balance after you restore it with the latest version.


What actually your purpose? Do you want to get the whole balance the same as what you see from your wallet in VM?
Or your purpose is to withdraw them all and transfer them to another wallet?

If you want to transfer them to another wallet then you can just copy all addresses from your wallet in VM(With the correct balance). Click the "window" and then "receiving addresses" it must show all of your used addresses including the change address. Collect them and put them to notepad and then check them one by one on http://blockchain.com/ to check each of these addresses has balance and calculate them in total.

If the total balance is correct after you calculate it there is a problem with your old wallet.dat file backup there might be a missing address on that wallet after you imported it to the latest bitcoin core. Possible in that time you might be generated a new receiving address after the backup and the missing BTC is in newly generated address.


If all suggestion above doesn't work the only solution that I can see here is to brute-force the wallet.dat file where the BTC balance is correct.
There is a tool called BTCrecover that you can use to brute-force the wallet file you can find it from here https://github.com/gurnec/btcrecover
Or take the service of http://walletrecoveryservices.com/