Bitcoin Forum

I have been doing some tinkering around, thinking about other people's wallet disasters, and believe I have come to the following conclusion...

If you have lost your wallet.dat for whatever reason (deleted it, formatted your drive, file corruption, etc.) it's possible that it may still be lurking on your computer.  If so, recovery is no longer purely theoretical.  With a little knowledge of what to search for, you can use a hex editor to potentially find usable remnants of your wallet.dat file and get back your bitcoins, even if the original file isn't fully recoverable.

So here goes...

If you can use a hex-editor to do a sector-by-sector search/edit on your entire hard drive, then search your entire hard drive for occurrences of the following byte sequence:

01 03 6B 65 79 41 04...........

the middle four of these bytes represent the string "keyA" in ASCII.

Each time this byte sequence occurs, a Bitcoin private key is probably stored nearby, about 180 bytes later.  The 32-byte private key is the only thing you need to recover your bitcoins!... as long as you find the right one(s).

Approximately 180 bytes after this sequence, you may find the byte sequence 04 20 (hex).  These two bytes seem to precede every private key (the 0x20 suggests a length of 32 bytes).  If you find this sequence, the thirty-two bytes that come after 04 20 are the private key representing a Bitcoin address and might be the private key that recovers some of your lost bitcoins!  Your wallet will have numerous private keys (at least one hundred, due to the pre-allocation of keys)... get as many as you can find.  Carefully search the sectors adjacent to any sector containing the "keyA" sequence above.  Then yell for help!  (But don't share the private keys in public, unless you want to give away your wallet.)

An example of a hex editor that can scan an entire disk volume for specific byte sequences for Windows is WinHex.  In WinHex, use Tools, Open Disk (F9), and choose the disk you want to scan.  Scanning a full disk can take hours.  WinHex must "run as administrator" to be able to scan a physical disk.  Someone please recommend a good way to do this in Linux, preferably with a known Live CD, if possible.  Also, any time you are scanning a disk for potentially lost data, you should NEVER boot the disk you're searching - always boot from another disk and install the target disk as secondary.

Would this work on SSDs? I heard they don't let you physically scan the flash memory.

Yes, you can do it on SSD's, it's just that your chances of success will be somewhat lower.  When you scan a disk in this manner, you are simply doing a sector-by-sector read of the entire disk, which will always succeed (in the sense that the disk can be read end-to-end without running into an error or a protest from the disk)... but it just may not turn up any lost bitcoins.

Say bye-bye to this method as soon as wallets are encrypted on disk.

It still works if you can decrypt the archive (or the entire disk) before doing the search. So mount the encrypted volume, then do the search.

Wouldn't help if the file alone was encrypted (say, with GPG or similar).  Still, even if that's implemented in the client, I hope such a feature would be optional, it might contain a warning such as: Encrypting your wallet may help thwart theft of your wallet file, though it also diminishes chances of recovery on accidental file deletion.

I was talking about encrypted wallet as implemented by the bitcoin client, which is a strongly demanded feature these days. Once deleted, it's lost forever. Keep backups!

If bitcoin use encrypted wallets, copying the wallet.dat several times becomes possible without danger

Casascius, may I suggest considering to offer this as a service?

You already have the skill and knowledge to do a search for the wallet data, and are relatively trusted, so others might prefer to send their hard-drives to you in the event of an accidental deletion of a wallet than do it themselves.

Wouldn't a windows system restore bring back a deleted wallet file?

This is now pretty much automated:
http://forum.bitcoin.org/index.php?topic=25091.0

I think in some cases that would work. Windows doesn't create restore points all the time, so I think there's a high chance that the data that a person deletes will not have been in existence when the latest restore point was created.

By the way, looking at your profile pic makes me want to drink a smoothie.


Thanks for that. This should be mentioned in the bitcoin wiki if it already isn't.

Is it expected that the private key would come way after the public key?  I'm doing a brute force search of my wallet file, using my crypto library to check whether each sequence of 64 consecutive bytes is on the secp256k1 elliptic curve (meaning it's a public key), then searching for a 32-byte sequence that gives you this public key when you interpret it as the private key.  What I am finding is that (1) this is really slow, (2) not every public key has an associated private key in the wallet file and (3) the private keys I'm finding are located about 200-300KB past where the public key was found.

This works under the assumption that both keys are encoded in big-endian and the public keys are encoded as [0x04 [X] [Y]] and private keys are encoded as [0x0420 [secretInt]].  But it doesn't work well enough to be confident that I'm extracting everything--why am I finding so many public keys that don't have matching private keys?   Could keys be stored in different endiannesses?  are some of them not preceded by '0x0420'?

I'd like to see the keys stored in a flat file that makes key recovery a million times easier.  The keypairs can be encoded as constant-length binary strings, just like the headers are serialized.  The private keys can be encrypted individually in this flat file, without encrypting the entire file.  Instead, you just convert the 256-bit private-keys to a 256-bit encrypted-private-key before writing it to file.  Then your wallet file still works for tracking transactions (since the public keys/addresses are still in plaintext), but the private key will be useless without transforming it back to the unencrypted bitstring.

Are you searching just wallet.dat or your entire hard drive?  The block chain database is going to be loaded with numerous public keys - belonging to other people, of course.

Each record containing a key in wallet.dat also contains the ASCII text "key" nearby as well.  (Because wallet.dat can hold multiple kinds of records, and this is how the software knows that this record is a key.)

I'm only searching the wallet.dat file.  I wanted to get my keys into a flat file for fun, maybe come up with a way to convert between the wallet.dat and wallet.flat.txt, which would make manual key management and recovery easier.

I figured out that the other keys, I believe, are addresses to which I've sent coins before... the ones that show up under the "Sent" addresses tab in the client. 

However, it does seem that there are multiple instances of your public keys in the file.  One of my public keys (for which I have the private key) shows up 10+ times.  Others show up two or three times.  I guess the wallet file holds transaction information, in addition to the keys themselves.

So, I have created this script which pulls out every public and private key in your wallet and stores them into flat files.  The "keylistpub.txt" will contain every public key in the wallet file whether it's yours or not, and "keylistpair.txt" contains a list of just the keys for which there is an associated private key in the wallet, and includes both.  This script should avoid duplicate keys in the output files.

http://dl.dropbox.com/u/1139081/extractKeys.tar.gz (http://dl.dropbox.com/u/1139081/extractKeys.tar.gz)

Keep in mind, this utility is going to create a new file on your computer with your private keys.   Handle with care! 

Now, I want to create something that converts the flat file back into a wallet.  This will allow someone to extract keys from multiple wallets, combine them into a single file, and the create a merged wallet.  The output file format looks like this:

(don't get too excited, I have mangled the private keys)

-- The first string is the address, which should be fairly obvious. 
-- The PubKey is two 32-byte numbers (x,y), which correspond to a point on the secp256k1 elliptic curve (the ECDSA curve used by the bitcoin network)
-- The PrivKey is literally a random 32-byte number, which gives the public key when you multiply the generator point by this number.  Yes, a private key is just a random number.  As such, there is no way to identify whether a string of digits is a private key, without having a public key to compare to.  Or rather, every 256-bit number is a private key, so a "private key" is only meaningful in the context of a public key point (x,y).   (all hex numbers are encoded in BigEndian)


Anyone want to help converting pub/priv keypairs into a wallet file?  I believe it can be done with the bsddb package in Python, but I haven't gotten it to work, myself, yet.

Joric's pywallet is the tool you need
However, I modified it a bit and it's more practical

Basically I added the possibility to import a key:
 - with its label
 - in a wallet not named 'wallet.dat'
 - as a reserve key (the hidden ones not shown in bitcoin adress book)

Here are both:
https://github.com/jackjack-jj/pywallet
https://github.com/joric/pywallet



Edit: I now read your entire message
I made a script too to backup the keys of a wallet into a new one automatically using my pywallet, without writing keys to the hdd
You may take a look: http://forum.bitcoin.org/index.php?topic=31418.0

This tool doesn't have to backup to the HDD, it's just that there's no other place to put it, at the moment.  I wanted this for my own use, to have a human-readable list of keys -- for backup or for easy input into other scripts/programs I want to write to do things with the keys without having to understand the wallet.dat format.   

As I expected, I'm not the first person to extract keys from a wallet, but I did want to recreate the wallet without the transaction history.  Sounds like pywallet will get me there.

04 20

 ;)

Pywallet can't import hex keys yet, please wait a few hours

Not necessary, I have no problem converting hex to binary on-the-fly.  I just gotta dig into pywallet code (eventually) and figure out how to get what I want out of it.

Actually it's not even binary, it's a pywallet format
I just finished the code, I run some tests and push the commit

Done, it's much more useful with hex support anyway
If you want to use it, grab the last version and run './pywallet.py  --importprivkey blahblahblah --importhex', with --datadir and --wallet maybe

Is this only for a single private key?  Can I supply a file with a list of private keys instead?  I started to try to figure out the mechanism by which the code tracks the keys and writes them to the wallet file, but so far I've been unsuccessful.  I also didn't try very hard...

Yes, only for a single key
I plan to add json file support soon though

Hello, I need some help please.

So, I followed the instructions on the first post, and I found many occurences of the byte series "01 03 6B 65 79 41 04" for "keyA" on my disk.
I don't know what to do now, the first post tells about private keys, but what about the public keys? How can I find the public-private key combinations? Or is there a way to recover the wallet.dat file as a whole?
Also there are texts like "pool", "wallet.dat", "addr.dat", "blkindex.dat" in between random characters around that area.

Thanks!

I accidentally formatted the wrong drive which had my wallet on it!

I was able to recover everything using Reclaime http://www.reclaime.com/library/

Interestingly enough, I recovered more data than the original drive held. The recovery even included files that I had deleted prior to my stupid self inflicted disaster.

I already had a backup of my drive but it was six months old and didn't include my current wallet. Now I back up every night and I back up the backup! Terrabyte drives are cheap compared to what I thought I had lost forever.

just wanted to say thanks, thanks, thanks casascius, I was able to recover a lost wallet with your info.  I thought I'd add some helpful steps to automate it.  Not 100% automated, but saved a bunch of time and automated enough for me.
Here are the tools I used:
pywallet.py https://github.com/jackjack-jj/pywallet (https://github.com/jackjack-jj/pywallet) Save this in the folder/drive where you will be doing the recovery from. ie 1.2.1
Notepad++ http://notepad-plus-plus.org/ (http://notepad-plus-plus.org/)  If you're good with sed or for then you can probably get away without this, but this is a great editor 10/10 would recommend for just about anything.
MultiBit https://multibit.org/ (https://multibit.org/) Multibit allows you to import private keys, that's why I chose it.
WinHex http://www.x-ways.net/winhex/ (http://www.x-ways.net/winhex/)

My 1st step suggestion would be to try testdiskhttp://www.cgsecurity.org/wiki/TestDisk_Download (http://www.cgsecurity.org/wiki/TestDisk_Download).  This program has no gui, but I have had much success with it in the past.  Unfortunately for me this time, the file it recovered had already been overwritten, and had spam data in it.

I created an image of my hard drive with WinHex.  I had the luxury of having another disk.  Any time you are doing data recovery your best bet is to not touch.  Ideally don't even boot off of the drive with the data you are recovering.  I was booted off of the disk I recovered from, so it is possible.  I didn't have a huge amount of BTC, so loosing them would have been a bummer, but not worth the hassle of moving the drive to another machine for me.  If you don't have an extra disk with enough space to store the partition you are trying to recover, this won't work for you.  Anyway on to the guide.  I did this on Windows 8.1, but should work on any version of windows past XP.  Also I'm writing this after the fact, but I'll try not to miss any steps.  I just figured it would have saved me some time.

To paste in the command window you have to right click and choose paste, ctrl+v does something else.  If you don't get a menu when you right click, click on the upper window icon, and Edit->Paste will be there.

1) Image the hard drive with WinHex.  This will take a while, took 3-4 hours on my computer iirc
  1.1)File->Create Disk Image...
    1.1.1)Edit Disk Window will open, select the partition where your wallet was.
  1.2)Next the Create Disk Image will open
    1.2.1)In Path and File name, choose a seperate disk to back up to, I don't know what the assigned automatically would do.  My choice was R:\Drive C.whx
    1.2.2)Scope:Sectors, put 0 in the first box
    1.2.3)Compression, choose none
    1.2.4)Split image into segments of, I left this at the default of 4096MB

from here on out we will be working in a cmd window.  Start->Run cmd doesn't need to be admin, but you'll need write access on the drive.  And Notepadd++.  Don't close either after a step, we'll be switching back and forth between them.
The drive i dumped to was R:\ from step 1.2.1.

2) scan the dumps with pywallet.py
  2.1)  Get a list of the dumps
for me this was
3) Create a password input file pass.txt, this will automate pywallet so you don't have to type in a password to use for the recovery wallet, and passwords to try on the dumps.  Save this in the same folder where you did the dumps.  Use notepad, notepad++, echo whatever you are most comfortable with, but it must be a plain text editor.  No word or whatever else.  Extra lines won't matter, but if you don't have enough, pywallet.py will be prompting you for them.  If pywallet is not running all of them automatically, and is asking for input, then you don't have enough empty lines in your pass.txt
Mine looked like this, because my original wallet did not have a password on it.
4) Open wallet recoveryStep1.cmd in notepad++.  Notpad++ allows column selection and editing, that is the main feature we will be using.  A little bit of search and replace also.  Column selection works by holding shift+alt
File will look like this
but many more dumps, mine went up to 233
  3.1) build the beginning of each line. Open Replace with ctrl+f, then select the replace tab
    3.1.1) findwhat =    3.1.2) replace with =    3.1.3) Search mode = Regular expression
  3.2) build the end of each line.  Open Replace with ctrl+f, then select the replace tab
    3.2.1) findwhat =    3.2.2) replace with =    3.2.3) Search mode = Regular expression
  3.3) Delete the last line if if looks like this, ie has no recov_device
 3.3) Save the file, it should now look like this
4) Create the WalletRecovery directory. pywallet will fail if it doesn't exist
4) run your generated cmd file. in the command window you openned earlier.  This will take a while, but not as much time as 1)
5) You should now have files in your WalletRecovery directory, the important ones are recovered_wallet_*.dat, hopefully you have some with a size > 32KB, this means pywallet.py has found something.
6) Build a script to extract the keys
  6.1)  6.2) open walletrecoveryStep2.cmd in notepad++, should look like this
 6.3) Open Replace with ctrl+f, then select the replace tab
    6.3.1) findwhat =    6.3.2) replace with = here's mine
   6.3.3) Search mode = Regular expression
  6.4) build the end of each line.  Open Replace with ctrl+f, then select the replace tab
    6.4.1) findwhat =    6.4.2) replace with =    6.4.3) Search mode = Regular expression
  6.5) Delete the last line if it has not wallet path in it ie7) Extract the keys 8) Open keys.txt in notepad++.  Should Look Something like this.  I had some which were 1 character either longer or shorter, don't remember, and I don't know if this matters or not, but I just used all that it made.
 8.1) Open Replace with ctrl+f, then select the replace tab
    8.1.1) findwhat =    8.1.2) replace with = empty
    8.1.3) Search mode = Regular expression
  8.2) Open Replace with ctrl+f, then select the replace tab
    8.2.1) findwhat =    8.2.2) replace with = Change this date so that it is earlier than your last transaction.  There is a space after the date
    8.2.3) Search mode = Regular expression
  8.3) Convert the line endings, Edit->EOL Conversion->Unix/OSX Format
  8.3) Save the file as Keys.key
9) Import the keys into multibit, Tools->Import Private Keys
  9.1) Choose Import File, select keys.key
  9.2) Click Import private keys
10) Profit, you can now either transfer these to a new wallet, move it to your preffered client.  I transferred mine out.
  10.1) I went back and changed the dates on the key file to an earlier date.  and then re-imported them, I don't know if this is needed or not.  I used a later date originally, because I didn't want it to have to sync up with as much.  After you found you wallet, you can then just do a send transaction and send them to a newly generated wallet.

ouch, I have no clue, but I would assume this will only work with a magnetic storage device.  If you were able to image the iphone storage somehow, or gain block level access you could continue on from the steps after having the image created.

Hi there,

Complete newbie trying a treasure hunt, luckily found your post, Thank You!

So I am scanning the hard drive with WinHex: should it look like this https://imgur.com/a/mrEeL (https://imgur.com/a/mrEeL) ?

I basically selected the drive, pressed "F9" and pasted, pressed "CTRL+F", and pasted "01 03 6B 65 79 41 04". Looking good?

Cheers!

I want to know is this a sit from where we can recover our lost bitcoins. And if so how is it possible. Because the past experience shows that when someone hacked or stole your bitcoin it will never be replaced and recover. But if it is possible due to this site it will be appreciable and I will specially warm welcome to this site. The controller should keep all the securities very tight and up to date.

Hi there guys,

Any reference to professionals providing this service?

Cheers

Is this method still can work to our forgoten private key? if this method still can work was so dangerous to all of us, because easy for hacker to hack our pc or laptop then take out the wallet.dat files to encrypted.

The best technique to do this is to decrypt the whole disk in order to do the search properly. This  also works fine on SSD.

Hi Guys,

Now I'm trying to solve the issue with corrupted wallet.dat for syscoin.
Maybe at first I'll very short describe the situation:
- I was trying to setup SYSCOIN masternode on Allnodes.com
- I was following the instruction:
1. Open your wallet (MAKE SURE IT IS THE 4.1.3 LATEST VERSION).
2. Go to the “Settings” menu and choose “Options”. In the Main tab enable the “Show Masternodes Tab” option and press “OK”.
3. Go to Syscoin Core root directory (location of wallet.dat) and open masternode.conf file with any basic text editor. This file may already contain # as the first item in the lines. These lines are comments and can be left in the file (to open masternode.conf file on MacOS click on the file, then select the “Open file with” option and choose “TextEdit” application there).
4. Insert that special configuration string you received from Allnodes in masternode.conf file below other lines.

 And at point 4, I made very big mistake (by acciden or my stupidity) I opened wallet.dat instead of masternode.conf.
Now the wallet.dat is corrupted.

I tried to recover the private key back using pywallet which was desrcibed by jackjack. Python founds tousands of keys, but unfortunatelly any of them after import don't show syscoin in syscoin core wallet.

Do you know what I'm doing wrog or there is no possibility to get the key and syscoin back?

Thank you in advance

What if I’m completely technologically illiterate (and a complete idiot) who acquired some BTC on a dinosaur smart phone in 2008 or 2009 as a novelty item, at the time (think Coke points or Marlboro rewards 😆) AND I don’t have access to the old (Yahoo) email address, the phone is gone and I don’t remember the wallet (because it was all practically worthless then) and I abandoned that email and last name? Screwed??? Dude, I had like 23 of them. I’m dead! Anyone have suggestions or should I just shoot myself now?

Yes... screwed...

You need at least one of the following:
- A wallet file (and any associated password)
- The private keys
- A seed phrase/recovery phrase (aka 12/24 word seed mnemonic)

It sounds like you don't have any of those things... or any way to recover any of those things. So, yes... screwed. :-\