Is it possible to recover an encrypted corrupt wallet if I have the passphrase?

[stuffses]

I found my old Bitcoin wallet that was encrypted about one year ago. Unfortunately it seems to be corrupted since the old (at time of wallet's encryption) and new versions of Bitcoin Core are unable to open it.

All my passwords are kept in a secure Keepass database, so I still have the wallet's pasphrase.

Opening the wallet file in a text editor (gedit) shows mostly binary (I think) data, but a few interesting words scattered around. Such words include "blockindex", "ckey", and the names of some transactions.

I've tried using the latest version of pywallet (https://github.com/jackjack-jj/pywallet) from the command line, but always receive this message:

Code:

ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.

Bitcoin is definitely not running, and I've tried the same commands with a fresh wallet without issues.

There are lots of very old (2+ years) posts claiming this is impossible, but I'm hoping things have changed since then. I haven't seen any recent posts on the subject.

Any ideas?

[btchris]

I've tried using the latest version of pywallet (https://github.com/jackjack-jj/pywallet) from the command line, but always receive this message:

Code:

ERROR:root:Couldn't open wallet.dat/main. Try quitting Bitcoin and running this again.

I'd start with this:

Open pywallet.py in a text editor that can deal with Unix-style lines endings (I'm a fan of Notepad++ if you're on Windows) and search for that error message. Shortly before it, you'll see this:

Code:

	try:
		r = db.open(walletfile, "main", DB_BTREE, flags)
	except DBError:
		r = True

Replace the "r = True" with "raise", being careful to keep the same level of indentation, and then re-run the --dumpwallet command. This should hopefully result in a more useful error message.

[stuffses]

I got the more detailed error:

Code:

Traceback (most recent call last):
  File "./pywallet.py", line 5028, in <module>
    read_wallet(json_db, db_env, determine_db_name(), True, True, "", options.dumpbalance is not None)
  File "./pywallet.py", line 2442, in read_wallet
    db = open_wallet(db_env, walletfile, writable=FillPool)
  File "./pywallet.py", line 2000, in open_wallet
    r = db.open(walletfile, "main", DB_BTREE, flags)
bsddb.db.DBInvalidArgError: (22, 'Invalid argument -- BDB2509 the log files from a database environment')

I assume the last line is most important. It mentions a log file, but I couldn't find anything like that in the folders being used.

[btchris]

I got the more detailed error:

Code:

bsddb.db.DBInvalidArgError: (22, 'Invalid argument -- BDB2509 the log files from a database environment')

Here's the full error message, which python truncates to just the last line:

Code:

BDB2506 file <filename> has LSN #/#, past end of log at #/#
BDB2507 Commonly caused by moving a database from one database environment
BDB2508 to another without clearing the database LSNs, or by removing all of
BDB2509 the log files from a database environment

Theoretically this should only happen if the wallet.dat file is copied out of the data directory while Bitcoin is still running or immediately following a crash.

If this is the only problem, running bitcoin-qt with the -salvagewallet option (with the wallet.dat to recover in the data dir) should be able to recover most if not all of the wallet.

[stuffses]

Not looking that good. Here is the output from debug.log after using salvagewallet:

Code:

2014-07-20 19:23:43 Bitcoin version v0.9.2.1-g354c0f3-beta (Thu, 19 Jun 2014 09:51:15 +0200)
2014-07-20 19:23:43 Using OpenSSL version OpenSSL 1.0.1h 5 Jun 2014
2014-07-20 19:23:43 Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
2014-07-20 19:23:43 Default data directory /home/ben/.bitcoin
2014-07-20 19:23:43 Using data directory /home/ben/.bitcoin
2014-07-20 19:23:43 Using at most 125 connections (1024 file descriptors available)
2014-07-20 19:23:43 Using 4 threads for script verification
2014-07-20 19:23:43 Using wallet wallet.dat
2014-07-20 19:23:43 init message: Verifying wallet...
2014-07-20 19:23:43 CDBEnv::Open : LogDir=/home/ben/.bitcoin/database ErrorFile=/home/ben/.bitcoin/db.log
2014-07-20 19:23:44 Renamed wallet.dat to wallet.1405884223.bak
2014-07-20 19:23:44 CDBEnv::Salvage : Database salvage found errors, all data may not be recoverable.
2014-07-20 19:23:44 Salvage(aggressive) found no records in wallet.1405884223.bak.
2014-07-20 19:23:44 Initialization result: 0
2014-07-20 19:23:44 Requesting shutdown
2014-07-20 19:23:44 Running Shutdown in thread
2014-07-20 19:23:44 Shutdown : In progress...
2014-07-20 19:23:44 StopNode()
2014-07-20 19:23:44 Shutdown : done
2014-07-20 19:23:44 Shutdown finished
2014-07-20 19:23:44 Shutdown result: 1
2014-07-20 19:23:44 Stopping thread
2014-07-20 19:23:44 Stopped thread

And db.log:

Code:

Program version 4.8 doesn't match environment version 5.3
file unknown has LSN 3831/10114800, past end of log at 1/1076
Commonly caused by moving a database from one database environment
to another without clearing the database LSNs, or by removing all of
the log files from a database environment
wallet.1405885221.bak: DB_VERIFY_BAD: Database verification failed

I've also tried opening the wallet with the version of Bitcoin that I thought was used to create it. No luck there either.

[coinsolidation]

Code:

Program version 4.8 doesn't match environment version 5.3

It looks like the wallet was created (or modified) with a version of bitcoin that was built to use berkeley 5.3, and you are trying to salvage it using the built in 4.8

[stuffses]

Code:

Program version 4.8 doesn't match environment version 5.3

It looks like the wallet was created (or modified) with a version of bitcoin that was built to use berkeley 5.3, and you are trying to salvage it using the built in 4.8

What can I do to get a newer version of Berkeley? I've never heard of this before. And why would the latest version of core have an older version than one from a year ago?

[coinsolidation]

Code:

Program version 4.8 doesn't match environment version 5.3

It looks like the wallet was created (or modified) with a version of bitcoin that was built to use berkeley 5.3, and you are trying to salvage it using the built in 4.8

What can I do to get a newer version of Berkeley? I've never heard of this before. And why would the latest version of core have an older version than one from a year ago?

If you are on linux, and say building from source, then the default berkeley is 5.3 for the operating system, and bitcoin uses 4.8 in the compiled executables.

I do not know if this applies to you, but it may well be the problem.

[RedDiamond]

Code:

Program version 4.8 doesn't match environment version 5.3

It looks like the wallet was created (or modified) with a version of bitcoin that was built to use berkeley 5.3, and you are trying to salvage it using the built in 4.8

What can I do to get a newer version of Berkeley? I've never heard of this before. And why would the latest version of core have an older version than one from a year ago?

If you are on linux, and say building from source, then the default berkeley is 5.3 for the operating system, and bitcoin uses 4.8 in the compiled executables.

I do not know if this applies to you, but it may well be the problem.

Yes, download soure, configure --with-incompatible-bdb and compile (assuming you have Berkeley  5.3 installed).

[btchris]

Code:

Program version 4.8 doesn't match environment version 5.3

It looks like the wallet was created (or modified) with a version of bitcoin that was built to use berkeley 5.3, and you are trying to salvage it using the built in 4.8

I have to disagree with coinsolidation... that message doesn't apply to the database version, but rather the environment version which is something else entirely in the complicated Berekeley DB world.... The Bitcoin client typically creates the environment on startup, and deletes it before exiting (except for really old versions of Bitcoin).

pywallet does not delete the environment before exiting, so I suspect that the warning message above is the result of running pywallet, and then running Bitcoin afterwards. In any case, I'm fairly sure it shouldn't cause any problems, and that it's just a warning.

Since you're on linux, you could use the Berkeley DB command line utilities to see if they can do any better. In theory, they try to do the same thing as -salvagewallet, but they're more up to date versions than the 4.8 that ships with Bitcoin, so maybe this will help.

On Debian/Ubuntu, they can be installed with "sudo apt-get install db-util". Then try this:

Code:

db_dump -r wallet.dat
db_dump -r wallet.dat | db_load newwallet.dat

If the first command produces a bunch of output, run the second one to create a newwallet.dat. If the first command doesn't produce output, try it again with a capital -R instead of the lowercase.

If all this still doesn't work, I could come up with some instructions for using pywallet's recovery methods. It would be my last choice though... pywallet doesn't recover the address book, and it's encryption facilities are fairly well broken, so you'd have to use them to create an unencrypted wallet and then deal with the consequences...