how I rescued my wallet.dat

marilyn4325 (OP)

Full Member

Offline

Activity: 229
Merit: 134

how I rescued my wallet.dat

December 23, 2017, 07:28:23 PM
Last edit: January 24, 2018, 09:25:49 AM by marilyn4325

Merited by ABCbits (11), Jet Cash (10), suchmoon (4), LoyceV (4), BitMaxz (1)

Two years ago I formatted my harddisk and installed Windows 10 on it. Before this I did a backup, but unfortunately the backup was broken. So I lost my wallet.dat, with a few Bitcoins in it. I could restore some files with RStudio, and I had older backups for the rest, but seemed to be that the latest wallet.dat was already overwritten, and I frequently add new addresses. So I gave up, not a big deal, maybe $200 lost. But I didn't use the harddisk and bought a new one.

Fast forward to December 2017: Now a few Bitcoins is some serious money, so I decided to give it another try. I tried any option I could find in RStudio, checking the dozens of filesystems it reported after scanning it for hours (only a few where valid from previous installations), but I couldn't restore it. Ok, this needed some more work.

My assumption was, that the file headers were broken, so I wrote a small C program myself, which scanned the whole harddisk for the wallet.dat signature (testing for the first 16 bytes). The filesystem was NTFS, which has 4k sector sizes and a file starts always at sector start, if I understand it correctly, which makes things easier. Also usually if there is enough space, contiguous sectors are used to save a file. My hope was that somewhere I could find old version of the wallet.dat, but not too old that the new keys were missing.

This is the very simple and straightforward scan program I hacked together:

Code:

#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>
#include <string.h>

uint8_t buf[4096];
char filename[1000];

uint8_t search[] = {
    0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x62, 0x31, 0x05, 0x00
};

int main(int argc, char** argv)
{
    uint64_t pos = 0;
    FILE* f = fopen("/dev/sdd", "rb");
    FILE* w = NULL;
    int walletNumber = 0;
    int walletIndex = 0;
    uint64_t max = 1000000000000ULL;
    while (1) {
        int c = fread(buf, 1, 4096, f);
        if (c != 4096) break;
        if (!w) {
            if (memcmp(search, buf, 16) == 0) {
                sprintf(filename, "wallet%i.dat", walletNumber++);
                walletIndex = 0;
                w = fopen(filename, "wb");
                printf("found: %" PRIu64 "\n", pos);
            }
        }
        if (w) {
            fwrite(buf, 1, 4096, w);
            walletIndex++;
            if (walletIndex == 256) {
                fclose(w);
                w = NULL;
            }
        }
        pos += 4096;
    }
    fclose(f);
    return 0;
}

I used it on Linux as my host system and the old harddisk was visible as /dev/sdd (you can see this with dmesg). I compiled it with "gcc -O3 scan.cpp -o scan" and started it with "sudo ./scan", and a few hours later (it was a 1 TB harddisk) I got a wallet0.dat to wallet9.dat, each 1 MB in size (it doesn't matter if there is crap after the wallet data). This was a nice start Grin

Then I tried to copy it to a wallet.dat of a current Bitcoin installation, but most of the time it said the wallet was corrupt, once it even crashed at start and when it said it could salvage some information, no keys were in it.

My rescue was https://github.com/joric/pywallet This program could decode all files and output it in JSON format. It needs the wallet.dat in a bitcoin-qt installation in the .bitcoin directory. I knew one of my old addresses, so I wrote a script which did test all files (actual key changed) :

Code:

for i in $( ls wallet*.dat ); do
    echo item: $i
    cp $i .bitcoin/wallet.dat
    ./pywallet.py --dumpwallet --datadir=.bitcoin | grep -i 12QDRXssT63Pv5KTGBN2kyAvfLW3s7jxBs
done

The output looked like this:

Code:

item: wallet0.dat
item: wallet10.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet11.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet1.dat
WARNING:root:encrypted wallet, specify password to decrypt
item: wallet2.dat
item: wallet3.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet4.dat
Traceback (most recent call last):
  File "./pywallet.py", line 1706, in <module>
    main()
  File "./pywallet.py", line 1683, in main
    read_wallet(json_db, db_env, True, True, "")
  File "./pywallet.py", line 1556, in read_wallet
    parse_wallet(db, item_callback)
  File "./pywallet.py", line 1287, in parse_wallet
    for (key, value) in db.items():
bsddb.db.DBPageNotFoundError: (-30986, 'BDB0075 DB_PAGE_NOTFOUND: Requested page not found')
item: wallet5.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet6.dat
WARNING:root:encrypted wallet, specify password to decrypt
            "addr": "12QDRXssT63Pv5KTGBN2kyAvfLW3s7jxBs", 
item: wallet7.dat
WARNING:root:encrypted wallet, specify password to decrypt
item: wallet8.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.
item: wallet9.dat
ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.

So the address was in wallet6.dat, success! I then used "/pywallet.py --dumpwallet --datadir=.bitcoin --password=mysecrectpassword > keys.txt" and I got all my keys back. In the bitcoin client I could import it with importprivkey (don't forget the "false" parameter as the last parameter, to avoid rescanning after each import, if you import multiple keys) and after the final rescan, I got my Bitcoins back. One day work for like 2 Bitcoins, which I already sold, that's a nice hourly rate Cool

Maybe this will help some other people as well. In case you rescue a lot of Bitcoins, I would really love it if you would send me some to 1ieKggPzp2DfroFBNie4ib77kHKNbJMkw.

Marilyn wishes you a merry Christmas, a merry Christmas, And a happy New Year!

marilyn4325 (OP)

Full Member

Offline

Activity: 229
Merit: 134