Crash during fresh synchronization

[galmok]

Upon resuming from standby, my bitcoin (windows client 0.14.1-win64) decided that there was a checksum problem with my repository and crashed/exited. I tried -reindex, but that also crashed/failed.

Then I decided to rename the BitCoin data directory and create a new (with the client stopped of course). I copied over peers.dat and wallet.dat files.

I fired up bitcoin and it started to synchronize from the internet, downloading a lot. But eventually, it crashed as well.

The last 3 lines in the debug.log file are:

2017-04-30 13:08:16 UpdateTip: new best=00000000000000b97602ec386d3264dd4d8801e34e987d25f495f8bfc98d463e height=239440 version=0x00000002 log2_work=70.214311 tx=18868874 date='2013-06-03 08:05:10' progress=0.087046 cache=238.3MiB(342458tx)
2017-04-30 13:08:16 Pre-allocating up to position 0x400000 in rev00064.dat
2017-04-30 13:08:16 UpdateTip: new best=00000000000000c7bd084cd484b92fae1dd9c67b308d7350fd3355c12b7c8275 height=239441 version=0x00000002 log2_work=70.214366 tx=18869290 date='2013-06-03 08:20:27' progress=0.087048 cache=238.4MiB(342590tx)

They are similar to the other lines in the log file and probably don't give any help.

The event viewer had this to say:

Faulting application name: bitcoin-qt.exe, version: 0.14.1.0, time stamp: 0x58f87f32
Faulting module name: ntdll.dll, version: 10.0.14393.479, time stamp: 0x5825887f
Exception code: 0xc0000005
Fault offset: 0x000000000006bc29
Faulting process id: 0x3924
Faulting application start time: 0x01d2c1a93075d47f
Faulting application path: C:\Program Files\Bitcoin\bitcoin-qt.exe
Faulting module path: C:\WINDOWS\SYSTEM32\ntdll.dll
Report Id: d79e4785-50fb-414f-901d-3fc488630b9b
Faulting package full name:
Faulting package-relative application ID:

I was able to start bitcoin up again and have it continue, but still, I'd prefer to get a handled error instead of a direct application crash. ;-)

[1714980908]

1714980908

[1714980908]

1714980908

[1714980908]

1714980908

[1714980908]

1714980908

[galmok]

And it crashed again. This time the debug log was more informative:

2017-04-30 13:39:41 LevelDB read failure: Corruption: block checksum mismatch
2017-04-30 13:39:41 Corruption: block checksum mismatch
2017-04-30 13:43:38 AddPortMapping(8333, 8333, 192.168.1.4) failed with code 402 (Invalid Args)
2017-04-30 14:04:37 AddPortMapping(8333, 8333, 192.168.1.4) failed with code 402 (Invalid Args)
2017-04-30 14:06:31 Error reading from database: Database corrupted

Crash location is different also:

Faulting application name: bitcoin-qt.exe, version: 0.14.1.0, time stamp: 0x58f87f32
Faulting module name: bitcoin-qt.exe, version: 0.14.1.0, time stamp: 0x58f87f32
Exception code: 0x40000015
Fault offset: 0x00000000012e8203
Faulting process id: 0x77b8
Faulting application start time: 0x01d2c1b4b9d5a7d9
Faulting application path: C:\Program Files\Bitcoin\bitcoin-qt.exe
Faulting module path: C:\Program Files\Bitcoin\bitcoin-qt.exe
Report Id: 8aa16d81-4345-4032-b538-90ce4501f197
Faulting package full name:
Faulting package-relative application ID:

My pc is otherwise completely stable, except bitcoin wont work properly.

[galmok]

And it of course crashed again due to the first crash causing database corruption. So it seems I can get approx 10% synchronized before database corruptions start occurring. Seeing as I haven't found anyone else with this problem, I can only assume my system is bad. But how do I debug a system that only exhibits problem with bitcoin?

[galmok]

I am now trying to see if it works with the linux binary, running via Bash for Windows 10 (full ubuntu 14 install, running bitcoin-qt using VcXsrv as X-server). I'll report back...

[achow101]

What are the specs of your machine?

Bitcoin Core is very resource intensive and will often make hardware issues much more obvious. It sounds like you have a hardware issue, probably one of your drives is bad. Do a full hardware diagnostic to see if it finds anything wrong with your hardware.

[galmok]

I have run Bitcoin Core for a long time without issues. I have a few hundred GB set aside for Bitcoin and use an i7 with 12GB ram to run it on. It has been very stable until resuming from the latest standby. I have rebooted since and checked disks. No error found nor any stability issues located.

The ubuntu (Bash for Windows 10) attempt is happily running, estimating 8 hours left. Though upnp doesn't work (lacks features in Bash for Windows), I tried port forward, but due to a router firmware bug, this isn't working (Netgear R7000). I'll let it continue until done and then clone the result and try continuing with the windows client. If I am lucky, this will work, but I guess this will cause the database to become corrupted again.

My guess so far is that there is something in the windows client that corrupts the database, probably a bad library.

[galmok]

Well, the linux bitcoin also crashed:

2017-04-30 17:31:39 LevelDB read failure: Corruption: block checksum mismatch
2017-04-30 17:31:39 Corruption: block checksum mismatch
2017-04-30 19:13:55 Error reading from database: Database corrupted

I guess my pc isn't good enough for bitcoin anymore, but how do I get my bitcoins back? The wallet.dat file is only usable with a working bitcoin.

[achow101]

I guess my pc isn't good enough for bitcoin anymore, but how do I get my bitcoins back? The wallet.dat file is only usable with a working bitcoin.

You can have Core be offline and export the private keys to be used elsewhere. Start Core with -connect=0 so it is offline. Then go to Help > Debug Window and click on the console tab. Type

Code:

dumpprivkey <address>

to get the private key of <address>. Do that for all of your addresses and record the private keys down somewhere. Then get a different wallet software that allows you to import private keys and import or sweep your private keys into the new wallet.

[galmok]

I'd rather not trust websites with my wallet. Is there any other bitcoin software that lets me use my wallet from my pc directly?

[achow101]

I'd rather not trust websites with my wallet. Is there any other bitcoin software that lets me use my wallet from my pc directly?

Certainly, and I recommend that you do use a desktop wallet. I recommend that you use Electrum. Any wallet listed here: https://bitcoin.org/en/choose-your-wallet under the "Desktop" category is a good choice for a wallet.

[galmok]

After having tried (and failed) to use a different disk, my conclusion was that it had to be memory error. I ran memtest86 and it actually did find a single bit error. First time I have had a memory fail on me.

I have now mapped the page out in windows and am attempting to run bitcoin again. If I was successful in mapping out all errors, I expect this to have solved the problem.

Now I can only wonder how many files have errors in them due to this memory error... BitCoin fortunately uses checksums, but not much else does. :-/

If BitCoin fails, I'll have a look at the wallet software.

[galmok]

This single bit seems to be the culprit for all my problems.

BitCoin has run for over 9 hours now and seems stable. Synchronizing takes a long time and it is only 49% complete so far. But it is stable.

Thanks for all the help and suggestions.