Restoring a deleted wallet file

[daniel g]

Due to a highly unlikely failure of all my encrypted backups, I can't access my wallet.dat. I have tried to restore a deleted (unencrypted) version of the wallet file from my hard drive using recovery software, but without success.

Is there any chance of restoring the deleted wallet.dat on a block-by-block basis? If so, what information would help me recognize the blocks and put them in the correct order? Of course, it would be enough to recover the private key, if that's somehow simpler than restoring the whole file.


In case you think you might be able to help me restore my TrueCrypt volumes, here is what happened:

Two years ago, I created two True Crypt containers and placed my wallet.dat into the first and several relatively unimportant files into the second. After checking the integrity and the password of both several times, I deleted all copies of the wallet, and copied both TrueCrypt volumes to four physically separate drives (my hard drive, two external hard drives and a flash drive). Every few weeks I would check both volumes on my hard drive and they would open fine. About a year ago, after not checking for several months, I noticed that the first volume gave me the error message, "Incorrect password or not a TrueCrypt volume", when trying to open it. The second volume still opens fine. I have used almost identical passwords on both (varying in a single digit), therefore I am certain that I have entered the correct password at least once. I tried the Restore Volume Header function and it didn't work. I have no extra backup of the header. I tried opening all copies from all hard drives on two separate computers and received the same error message in all cases. I use Windows 7, and I believe the volumes were created with TrueCrypt 7.0a.

Trying to make sense of this as a layperson I see three possible scenarios of what might have happened:

1. All five copies became corrupted independently.
2. The same damaging operation was performed on each copy as I tried to mount them (perhaps by Windows).
3. An error was present in the original volume in "seed form" that later corrupted all its copies (don't know if that's possible).

My first question is: What do you think happened to the volume and how did it happen to four separate copies?

My second question is: Is there any way to repair or restore the original volume?


Thanks a lot for any help or ideas.

[1715098122]

1715098122

[1715098122]

1715098122

[CIYAM]

I think out of your 3 scenarios only #2 really makes sense (unless you think your TrueCrypt software had some sort of time bomb in it).

Did you actually verify that each of your backed up copies actually worked (am not talking about the original but the copies themselves)?

The problem with recovering deleted files is how quickly you do it - the more usage that has happened between deleting and trying to recover then the less chance there is of recovering anything much at all.

In regards to finding the private keys (for an unencrypted wallet) you just need to locate some hex bytes (you'll need to search the forum for that - I think Mike Caldwell has helped people with this before).

And btw - why didn't you back up to a CD-R/DVD-R (not trying to rub in salt but something that really should have been done)?

[HighInBC]

I have not trusted hard drives since my raid5 array had a double failure.

I use a combination of cloud storage and printouts. Both encrypted to good passwords.

[daniel g]

Thanks for the replies.

I am pretty sure I tested several of the copies before I deleted the original wallet file.

I don't like the idea of having an unencrypted wallet on a CD/DVD. I would have prefered a paper wallet. What I really should have done, however, is using a brain wallet, but at the time I created the wallet I didn't find any information on that option.

Ian, you said that I "just need to locate some hex bytes". Is it also possible to locate them on the hard drive after the unencrypted wallet has been deleted? (I am assuming it's impossible to locate them in a corrupted TrueCrypt volume.) I did a brief search for "hex bytes" and "Mike Caldwell", but didn't find anything that seemed immediately helpful (also, my technical understanding is quite limited). If you could specify this a bit more or point me in the right direction, I'd be very grateful.

[CIYAM]

The search on the forum sucks - a quick google came up with these:

https://bitcointalk.org/index.php?topic=56655.0
https://bitcointalk.org/index.php?topic=22697.0

Maybe one of those might help (or perhaps send Mike a PM).

[daniel g]

Awesome, thanks a lot!

[w1R903]

Awesome, thanks a lot!

Daniel, if (and only if!) someone can tell you a non-destructive way to make a copy of your restored copy of the wallet.dat, it would be worth a try to run pywallet on it:

https://github.com/jackjack-jj/pywallet

Then run:

python pywallet.py --datadir='/your/data/dir' --dumpwallet

Note that the datadir will be something like "%APPDATA%\Bitcoin" if you're on Windows.

But before you do any of this, make sure you figure out a way to safely make a copy without doing any additional damage to the file.  I don't know enough about the Windows file system to tell you how to do this safely (any Windows devs here know how?).  Maybe one of the EaseUS recovery utilities would help?

pywallet has helped a lot of people in the past dump the private keys of a (slightly) damaged wallet.  If this doesn't work, some of the more advanced berkelydb data dump tools might help.  PM me if you succeed in safely making a copy and need help.

[daniel g]

Thanks, w1R903. I'll be sure to PM you if I need additional help.

From what I understood, the first step is to try to locate the private keys with a hex editor. The second would be to compile them into a wallet.dat using pywallet. I hadn't read enough about that second step to realize that there is a danger of further damaging the file. I thought that once (and if) I recover the private key, that's all the data I need.

To be safe, I'll have a friend who is an IT professional talk me through this. I definitely appreciate any advice on software and/or procedure, though!

[w1R903]

Thanks, w1R903. I'll be sure to PM you if I need additional help.

From what I understood, the first step is to try to locate the private keys with a hex editor. The second would be to compile them into a wallet.dat using pywallet. I hadn't read enough about that second step to realize that there is a danger of further damaging the file. I thought that once (and if) I recover the private key, that's all the data I need.

To be safe, I'll have a friend who is an IT professional talk me through this. I definitely appreciate any advice on software and/or procedure, though!

Daniel, if I were you -- and assuming you have enough in that wallet for this to be worth your while -- I'd find someone skilled in Windows data recovery.  Maybe your friend understands this.  Get him/her to make a non-destructive copy of the file.  I just don't know enough about Windows file systems to tell you to just make a copy -- I'm worried that this could further damage the file.

But once someone who understands data recovery on Windows helps you to make a copy, you can play around all you want with pywallet, berkelydb's data dump tool, and a hex editor until you (hopefully) find your keys.  Note that pywallet cannot "compile" wallet.dat files (unless it's recently added this functionality).  Instead, pywallet reads wallet.dat files and "dumps" out the private keys to the screen in json format (along with some other data).

Here's a thread that talks about recovering data with a hex editor, thanks to John Tobey's expertise.  He also includes a brief Perl script to help recover deleted wallet.dat, but it only works on Linux partitions.

Good luck.

[jackjack]

Note that pywallet cannot "compile" wallet.dat files (unless it's recently added this functionality). 

I'm not sure what you mean by 'compile', but Pywallet can import csv files full of private keys for a few weeks
More on topic, it can also read the partitions to find deleted keys/wallets, for more than a year
OP might give it a try

[daniel g]

Sorry, by "compile" I meant "restore the private keys to a format usable by a Bitcoin client", i.e. make private keys usable. I will try to avoid using technical terms I don't understand.

I am still trying to figure out the best way to go about a recovery step-by-step. Is pywallet able by itself to find the deleted wallet/keys or would I have to run a separate utility beforehand?

[w1R903]

OP might give it a try

Which is why I recommended it.  

And I put "compile" in quotes because this was the term the OP used, apparently in reference to creating a new wallet.dat file based on the output read from an old one ("compile them into a wallet.dat using pywallet").  I knew that creating a new wallet.dat using the data read using pywallet hasn't traditionally been possible, but since I knew you were working on it recently, I didn't want to rule out a new feature categorically, however unlikely.  I'm not unfamiliar with Python, pywallet,  berkeleydb, or the wallet.dat format.

[jackjack]

Sorry, by "compile" I meant "restore the private keys to a format usable by a Bitcoin client", i.e. make private keys usable. I will try to avoid using technical terms I don't understand.

I am still trying to figure out the best way to go about a recovery step-by-step. Is pywallet able by itself to find the deleted wallet/keys or would I have to run a separate utility beforehand?

Pywallet does tends to be practical: you run it (no separate utility, and no depency for recovery mode I believe), it finds the keys, then it create a usable wallet.dat

 

OP might give it a try

Which is why I recommended it. 

And I put "compile" in quotes because this was the term the OP used, apparently in reference to creating a new wallet.dat file based on the output read from an old one ("compile them into a wallet.dat using pywallet").  I knew that creating a new wallet.dat using the data read using pywallet hasn't traditionally been possible, but since I knew you were working on it recently, I didn't want to rule out a new feature categorically, however unlikely.  I'm not unfamiliar with Python, pywallet,  berkeleydb, or the wallet.dat format.

It's possible now
I know it works on Linux but I don't remember anyone testing it on windows