Old wallet.dat in latest core debug info help please.

[morbius55]

When I load an old wallet into Bitcoin Core, it changes from 96kb to over 600kb and only shows one legacy format key and the rest are new keys(presumably a thousand). I also expected the wallet file to be encrypted but it isn't. The wallet seems to load okay without throwing a corruption message out but does not show any old keys etc. I have shown four screenshots here of the debug log, from starting up with this wallet file. Thanks.
https://imgur.com/a/tjdKISW

[BitMaxz]

It seems the wallet.dat become corrupted after imported it to the latest version of bitcoin core.
Anyway, do you still have the old backup copy?

If you still have the old copy make a new copy and try to recover your old public keys and private keys using pywallet you can follow the guide from this link below.

- [GUIDE] Recover your deleted keys

or you can just dump private keys using this command

Code:

pywallet.py --dumpwallet --wallet=J:\Dump\wallet.dat > walletdump.txt

If it's encrypted then add this

Code:

--passphrase=yourwalletpassword

It should look like this

Code:

pywallet.py --dumpwallet --passphrase=yourwalletpassword --wallet=J:\Dump\wallet.dat > walletdump.txt

Change the path location under --wallet= if where your wallet.dat copy.

Now it should show all of your private keys.

Now you can sweep or import it in your latest bitcoin core if you still want to use the latest bitcoin core.

[morbius55]

It seems the wallet.dat become corrupted after imported it to the latest version of bitcoin core.
Anyway, do you still have the old backup copy?

If you still have the old copy make a new copy and try to recover your old public keys and private keys using pywallet you can follow the guide from this link below.

- [GUIDE] Recover your deleted keys

or you can just dump private keys using this command

Code:

pywallet.py --dumpwallet --wallet=J:\Dump\wallet.dat > walletdump.txt

If it's encrypted then add this

Code:

--passphrase=yourwalletpassword

It should look like this

Code:

pywallet.py --dumpwallet --passphrase=yourwalletpassword --wallet=J:\Dump\wallet.dat > walletdump.txt

Change the path location under --wallet= if where your wallet.dat copy.

Now it should show all of your private keys.

Now you can sweep or import it in your latest bitcoin core if you still want to use the latest bitcoin core.

I can;t understand how it even used the wallet without causing an error. Not sure why there ends up being one old style address in there.

[nc50lc]

-snip-

The logs seem normal to me aside from some font issues and "Unknown wallet record: 1" probably because it's an old wallet file, usually not an issue.
Are you sure that it's the correct wallet.dat or the latest backup of it? Because an outdated copy of a non-HD wallet (since you said its old) won't be able to recover the newer keys generated by the active instance of it.

The size is probably because when it's loaded in the latest Bitcoin core, its keypool was pre-loaded with thousand more keys, while the old version only contains hundreds.
The address format, it's because Bitcoin core will generate native SegWit by default but it won't change what's already used by the wallet.
For the encryption suddenly removed, it strongly suggest that it must be an older backup or a totally different wallet.dat than what you're expecting.

[HCP]

It would appear the old wallet.dat only had one key in it... I'm not sure if the old versions of Bitcoin Core always had a keypool of 100, or if it originally just generated keys/addresses on the fly.

It's also possible the wallet.dat was modified to delete keys... and/or it is actually a "recovered" wallet.dat from your many attempts at using PyWallet that only had 1 legacy key/address loaded in it.

Once you opened it with the latest version, it would have been loaded up with a new keypool of 1000 (which would have defaulted to the bech32 addresses) which would explain why the filesize changed.

Opening a "non-encrypted" wallet with a new version of Bitcoin Core does not automatically encrypt the wallet... you would need to explicitly set a passphrase using the "Settings -> Enncrypt wallet" option in Bitcoin Core. Also, note that "encrypting" a Bitcoin Core wallet does not actually encrypt the entire contents of the wallet (like Electrum does). ie. it is not a full file encryption... it is only the private key data that actually gets encrypted.

What makes you think the wallet.dat should have been encrypted?

[morbius55]

It would appear the old wallet.dat only had one key in it... I'm not sure if the old versions of Bitcoin Core always had a keypool of 100, or if it originally just generated keys/addresses on the fly.

It's also possible the wallet.dat was modified to delete keys... and/or it is actually a "recovered" wallet.dat from your many attempts at using PyWallet that only had 1 legacy key/address loaded in it.

Once you opened it with the latest version, it would have been loaded up with a new keypool of 1000 (which would have defaulted to the bech32 addresses) which would explain why the filesize changed.

Opening a "non-encrypted" wallet with a new version of Bitcoin Core does not automatically encrypt the wallet... you would need to explicitly set a passphrase using the "Settings -> Enncrypt wallet" option in Bitcoin Core. Also, note that "encrypting" a Bitcoin Core wallet does not actually encrypt the entire contents of the wallet (like Electrum does). ie. it is not a full file encryption... it is only the private key data that actually gets encrypted.

What makes you think the wallet.dat should have been encrypted?

Pywallet confirmed that it is encrypted and that the passphrase is correct. This wallet file is not a pywallet recovered file, it is a wallet.dat recovered from the hard drive with recuva and showed that it originally came from the usual appdata/roaming/ bitcoin file path on the drive. I have shown some screenshots below which includes one of the wallet info command in the daemon bitcoin-wallet utility(this is before bitcoin core has done it's thing with the wallet). I have shown a hex view of the wallet and highlighted the address that the wallet dumps show, and also the address I'm after(strange that this address is in an encrypted wallet, yet core can pull it out of there with it's private key and the resulting wallet is unencrypted). Another strange thing is the dates of the addresses(2013). Cheers.
https://imgur.com/a/8WxPaJj

[HCP]

Is the hex view from before or after it has been upgraded by Bitcoin Core?

Interestingly... the "1Cni" address that you show actually has a date of 2016... So, I'm wondering if this wallet was already "updated" at some point AFTER 2013.

I suspect the old keys from 2013 were just never used, and now that the wallet was upgraded, they were given "bc1" addresses in the wallet dump (as that is what Bitcoin Core defaults to now).

You could possibly test this, by taking a copy of the original wallet.dat that has not been opened in a new version of Bitcoin Core... and then loading it into Bitcoin Core, after starting Bitcoin Core using the -addresstype="legacy" and -changetype="legacy" options set (or addresstype="legacy" and changetype="legacy" in bitcoin.conf).

Then check the dumpwallet output to see if the 2013 keys are labelled with "1"-type legacy addresses instead of bc1 addresses.

[morbius55]

Is the hex view from before or after it has been upgraded by Bitcoin Core?

Interestingly... the "1Cni" address that you show actually has a date of 2016... So, I'm wondering if this wallet was already "updated" at some point AFTER 2013.

I suspect the old keys from 2013 were just never used, and now that the wallet was upgraded, they were given "bc1" addresses in the wallet dump (as that is what Bitcoin Core defaults to now).

You could possibly test this, by taking a copy of the original wallet.dat that has not been opened in a new version of Bitcoin Core... and then loading it into Bitcoin Core, after starting Bitcoin Core using the -addresstype="legacy" and -changetype="legacy" options set (or addresstype="legacy" and changetype="legacy" in bitcoin.conf).

Then check the dumpwallet output to see if the 2013 keys are labelled with "1"-type legacy addresses instead of bc1 addresses.

The hex view is before being loaded into core. I have seen both a 2016 and 2013 date shown in 2 different wallet dumps for that legacy address, last image first line.
Something to note is that i haven't done a rescan while this wallet has been loaded into core. Would that have a different outcome regarding what keys/addresses it would show. Still don't understand the encrypted wallet thing though.

[HCP]

In any case... if you can see the private key for the address you're after... what is the issue? Can't you just go and import this key into a wallet of your choosing and recover the funds?

Or are you just trying to work out the technical workings of Bitcoin Core and how it goes about updating wallet files?

[morbius55]

In any case... if you can see the private key for the address you're after... what is the issue? Can't you just go and import this key into a wallet of your choosing and recover the funds?

Or are you just trying to work out the technical workings of Bitcoin Core and how it goes about updating wallet files?

It only shows the 1cni address, not the other address that is next to it in the hex view. That is the one with the 32btc. Cheers.

[HCP]

Ahhhh, in that case, I suspect that the "1L5s" address might have actually been in the old wallet as a "watching only" address... and the wallet didn't contain the private key for it.

It's also possible that the file was just damaged and the private key data was unable to be extracted.

[morbius55]

Ahhhh, in that case, I suspect that the "1L5s" address might have actually been in the old wallet as a "watching only" address... and the wallet didn't contain the private key for it.

It's also possible that the file was just damaged and the private key data was unable to be extracted.

Most likely it's the latter, as he is even less bitcoin savvy than me, hence the mess he has created. If I placed the file in the pywallet folder, do you think I could dump it there, even though it isn't a "recovered" wallet?

[HCP]

PyWallet should dump any valid wallet.dat... regardless of whether or not it was created by PyWallet, recovered by PyWallet, recovered by data recovery software or created by Bitcoin Core.

There isn't real "marker" that would indicate to the script that it was a "Recovered" wallet or not... As long as it is not corrupted and in a valid format, PyWallet should be able to read and dump it.

[morbius55]

So the plot thickens. I tried dumping it with pywallet without a passphrase, and it did, and said it is unencrypted. When the wallet is examined for info using the bitcoin-wallet.exe in daemon, it says it is encrypted. I have also used a passphrase with pywallet and it told me passphrase is correct I think the wallet file maybe has a passphrase assigned to it, but the mechanism for using it is maybe damaged. So anyway, the cni address is shown as the default key, and nearly 200 other legacy keys are dumped with private keys(which don't show up when loaded into bitcoin core) and the key I need is not present. The address of interest is just shown near the bottom with the cni address, but it's private key isn't anywhere to be seen So I thought well, the wallets unencrypted, time to bring out winhex and run the old 0201010420 search. Sure enough it found all the markers for the private keys and I tried one on bitaddress(offline of course) and It happened to be the cni address and it all tallied up. I found this excellent post on stack exchange which tells you a few markers to search for, which I tried out and and confirmed all of the ones relevant to an unencrypted wallet. I searched the key(00 01 03 6b 65 79) marker and found that 0201010420 followed it, exactly the same amount of bytes away, for every key in the wallet But, and it's a big but(no pun intended) their is one key that has some of it's bytes zeroed out including the preceding 0201010420 marker but I do know it's position by counting it's distance from the key marker My thought is that maybe a script could be written by someone to guess the missing hex characters(could or couldn't be possible in a reasonable time frame I assume). Or search for chunks of what I can see of the key on the whole hard drive with winhex, which I would assume will be a long process, but I can use various size chunks of the remaining hex traces and they're are consecutive runs of bytes that should be pretty unique, even throughout a whole hard drive. Anyway, sorry to ramble on but that is where I am with it now. Cheers.
https://bitcoin.stackexchange.com/questions/41447/filesystem-is-corrupt-how-to-find-wallet-dat

[HCP]

Right... so you managed to recover the private key for your CNI address then? If so, that's great news!

As for trying to bruteforce a key with missing HEX characters, it's technically possible, but the big question is how many characters are missing? If it's more than 8-10 characters missing is going to take quite a long time... 9 missing characters would be: 68719476736 possible combinations... 10 missing characters is: 1099511627776 possibilities... and it just keeps growing exponentially.

[morbius55]

What does pywallet use for it's search, is it purely the 0201010420 sequence or other markers as well? Cheers.

[HCP]

I believe these are the markers that it uses: https://github.com/jackjack-jj/pywallet/blob/master/pywallet.py#L165

[malevolent]

It would appear the old wallet.dat only had one key in it... I'm not sure if the old versions of Bitcoin Core always had a keypool of 100, or if it originally just generated keys/addresses on the fly.

It's also possible the wallet.dat was modified to delete keys... and/or it is actually a "recovered" wallet.dat from your many attempts at using PyWallet that only had 1 legacy key/address loaded in it.

Once you opened it with the latest version, it would have been loaded up with a new keypool of 1000 (which would have defaulted to the bech32 addresses) which would explain why the filesize changed.

Bitcoin Core always had a keypool of 100 until version 0.15 in late 2017 when it was increased tenfold. As far as I know, Bitcoin-Qt/bitcoind also always came with a default keypool of 100.