Yet another corrupted wallet question...

[odysseus654]

Okay, looks like this is the only place I can ask my questions, let's see how far I can get...

I've been having issues for the last few months with Bitcoin unexpectedly disappearing, over time I would get a couple dozen little BitCoin icons in my tray (that would evaporate whenever I touched them with the mouse, a symptom of a previously-crashed program)

With the newest version (0.4beta) I'm actually starting to get error messages.  This is an example:

Code:

EXCEPTION: 22DbRunRecoveryException       
DbEnv::txn_checkpoint: DB_RUNRECOVERY: Fatal error, run database recovery       
C:\Program Files (x86)\Bitcoin\bitcoin.exe in ThreadIRCSeed()       

I (made a copy and) deleted all the files in my bitcoin directory except for: wallet.dat, bitcoin.conf, server.cert, server.pem and restarted Bitcoin.  It then rebuilt the keychain and got up to about the middle of August and would start getting the same kinds of errors:

Code:

EXCEPTION: 22DbRunRecoveryException       
DbEnv::open: DB_RUNRECOVERY: Fatal error, run database recovery       
C:\Program Files (x86)\Bitcoin\bitcoin.exe in AppInit()       

EXCEPTION: 22DbRunRecoveryException       
DbEnv::open: DB_RUNRECOVERY: Fatal error, run database recovery       
C:\Program Files (x86)\Bitcoin\bitcoin.exe in CMyApp::OnUnhandledException()

Okay, time to break out my VirtualBox running Ubuntu...

Code:

db4.8_dump wallet.dat | db4.8_load wallet-new.dat

(no errors)

Launching BitCoin with this:

Code:

EXCEPTION: 11DbException       
Db::open: Resource device       
C:\Program Files (x86)\Bitcoin\bitcoin.exe in CMyApp::OnUnhandledException()       

Okay, deleted my wallet file, launched bitcoin to create an empty wallet, then shut it down and used the empty wallet as the target of db4.8_load (again, no errors).  Bitcoin says:

Code:

Error loading wallet.dat: Wallet corrupted

At this point I'm feeling less like my wallet is bad and more like I don't know what I'm doing and I need to start diving into source to see what is going on, like excluding one table from the dump or writing a program to wipe the contents of a table or something.  The fact that db4.8_dump gave no obvious errors makes me feel like whatever is going on is happening outside of BDB inside of Bitcoin proper.

The dump file format doesn't make it terribly easy to read, I'm seeing a very long sting of hex digits that I'm guessing I need to somehow decode if I'm to make any heads or tails of this?  Aren't there supposed to be table names somewhere in here?

This being the newbie forum I'm half expecting the usual "run fdisk and try again" response I seem to get so often from customer service.  I prob don't have the number of coins that people here have (although I don't really know what my current balance is right now).  It would be nice if I could come up with some SQL somewhere to copy the keys over and just do a -rescan of everything else...

[1714856520]

1714856520

[1714856520]

1714856520

[1714856520]

1714856520

[1714856520]

1714856520

[odysseus654]

Just a quick update, after surveying the various "bitcointools" out there, I used TheBlueMatt's version and called "fixwallet.py" with default options on my wallet.dat file.  There were no messages displayed during the conversion.

Bitcoin ran again for about an hour and then halted with the following message:

Code:

EXCEPTION: 11DbException       
Db::get: Resource device       
C:\Program Files (x86)\Bitcoin\bitcoin.exe in ProcessMessage()       

It is refusing to start up again with the following message:

Code:

EXCEPTION: 22DbRunRecoveryException       
DbEnv::open: DB_RUNRECOVERY: Fatal error, run database recovery       
C:\Program Files (x86)\Bitcoin\bitcoin.exe in AppInit()       

I guess I'll tinker with it some more, maybe run fixwallet.py a few more times...

[2112]

Check out the content of the file "db.log". It has a plain text messages about what was the problem discovered with the databases.

[odysseus654]

Uff, I really wish this thing had timestamps in it, there's an awful lot of junk in it.  Here's an interesting excerpt though:

Code:

wallet.dat: unable to flush page: 10
txn_checkpoint: failed to flush the buffer cache: DB_RUNRECOVERY: Fatal error, run database recovery
PANIC: fatal region error detected; run recovery
PANIC: fatal region error detected; run recovery
C:\Users\Erik\AppData\Roaming\Bitcoin/database\log.0000000106: log file unreadable: Resource device
PANIC: Resource device
PANIC: DB_RUNRECOVERY: Fatal error, run database recovery
unable to join the environment

[2112]

"region error" means that one of the __db.00? files got damaged. At the moment I can't find the sensible description of "Resource device" error.

No matter what, the most likely cases:

1) bad disk drive (failing SSD?)
2) some real-time antivirus corrupting files

I just read your original post. You corrupted your files yourselves. You used db_dump compiled with different compiler settings which has a different layout of structures in the "region" and "log" files. It is pretty much impossible to compile Berkeley DB the same way as the core development group without using the "gitian" build system that they are using.

[odysseus654]

Yah well I made plentry of copies.  None of the db_dump|db_reload files are currently in service (I kinda noticed that every attempt to use BDB utilities seems to explode on contact, last attempt to use the utilities resulted in a "FATAL: unsupported log version 16" error.

The git checkout of fixwallet.pl seems to work, but it seems to fail after a period of time, after which I need to copy the folder back over to ubuntu, re-run the script, and then copy it back again.  I'm on the third cycle now and it's somewhere in September 10 right now.

[2112]

The git checkout of fixwallet.pl seems to work,

With emphasis on "seems". You should be able to use the db_* utilities provided:

1) you build them with the same build process that the "core developers" use.
2) add DB_CONFIG file into the %APPDATA%\Bitcoin directory that exactly emulates the flags set in the DB_ENV calls in the bitcoin source code.

db_* utilities give fairly descriptive error message about "database environment mismatch" if you don't follow the step (1) correctly. However they are helpless about (2) because the "core developers" decided to hide the database logs in the %APPDATA%\Bitcoin\database subdirectory. Without correct DB_CONFIG file you are pretty much assured corruption, unless the *.dat files were properly flushed and closed by the main bitcoin executable.

[odysseus654]

Okay, sounds like I need to rebuild Bitcoin from source then or something.  I've already found the github directory and I'm guessing ubuntu would be easier to build than msvc (ubuntu seems to always be easier for these platform-independent open-source projects).

I just hope it's easier to build than VirtualBox, I spent a good month and never die get all the Qt prerequisites and compiler patches assembled quite right to make it more than an hour into the process.

Thank you for your help and in pointing me in a (relatively) healthy direction.

[odysseus654]

FWIW Bitcoin just crashed again, which gives me a before+after to be able to describe the entire error message:

Code:

MoveFile: rename C:\Users\Erik\AppData\Roaming\Bitcoin\__db.001 to temporary file: Broken pipe
DeleteFile: C:\Users\Erik\AppData\Roaming\Bitcoin\__db.001: The process cannot access the file because it is being used by another process.


C:\Users\Erik\AppData\Roaming\Bitcoin/database\log.0000000106: log file unreadable: Resource device
PANIC: Resource device
PANIC: DB_RUNRECOVERY: Fatal error, run database recovery
unable to join the environment

However, it looks like BitCoin may still be running (my confirmation count is still climbing???) behind this error message.

I will probably still be looking into building a developer BitCoin, but likely not tonight.  Also ran into QBitcoin as a possible alternative/future client, will be very glad to get away from this BDB junk.

[odysseus654]

Think I'm prob done with this for now.  I did end up creating a developer BitCoin (after sticking an extra "windows:LIBS += -lgdi32" at the end of the .PRO file to satisfy apparent linker dependencies in OpenSSL) and the debugger seems to go a bit off the rails around the first call to dbenv.open() (seems to eventually chain to abort() somehow, debugger stops giving useful info long before that point).  It seems to launch okay if i give it a brand new Bitcoin folder, but not with my current wallet.

It looks like things are pretty much okay as long as I don't push too many buttons.  Will revert to my earlier BitCoin directory and just relaunch the officially-downloaded client every so often.  I don't *really* need to know my balance and it sounds like I still have access to my keys & balance when I need them, but not having it participate in whatever non-miners do all the time kinda makes me feel like i'm leeching or something...

[tlhonmey]

If you launch the client with the -rescan parameter it should go through the block chain and figure out what the balances are for the keys you have.  If it still comes up with nothing then the program listed at https://bitcointalk.org/index.php?topic=25091.0 will go through your whole disk and find anything that looks like a private key.  It seems to do a pretty good job if your wallet wasn't encrypted.

[deepceleron]

It almost sounds like you do have bad sectors on your hard drive and a recertification of the whole disk would be prudent. The best way would be with your hard drive manufacturer's HDD diagnostics (they boot off a CD), and do a full surface test. After that passes, fsck your file system and check for bad blocks in the OS.

After backing up your wallet again(you should have many copies, so that if corruption happened a month ago but it went unnoticed, you can roll back to that), run the tests above. Then delete the entire bitcoin data directory including the log files directory and start Bitcoin up fresh so it creates a new wallet and blockchain from scratch. If you get it working up to where it has the latest block and can receive and send back out test amounts, then replace the wallet.dat with your backup, preferably a copy that was made before any issues started. If it still dies horribly, likely you have some bad hardware like bad RAM or a bad HDD controller corrupting things.