bitcoin-qt wallet crashes twice a month, database corrupted

[induktor]

Hello.
i've been having these issues for almost a year, but now it is far more frequent.

before version 0.13.0 happened maybe once every two months, now it's twice a month.

i use bitcoin to work, i need the wallet operational, every time it dies, it means that i cannot use it for nearly 30 hours until the --reindex, rescan finishes!, what is it, the same issue happened with 0.12.0, then 0.13.0 , now with 0.13.1.

here is the log section:

Code:

2016-11-16 01:29:48 Corruption: block checksum mismatch
2016-11-16 01:29:48 *** System error while flushing: Database corrupted
2016-11-16 01:29:57 ERROR: ProcessNewBlock: ActivateBestChain failed
2016-11-16 01:29:57 tor: Thread interrupt
2016-11-16 01:29:57 torcontrol thread exit
2016-11-16 01:29:57 msghand thread interrupt
2016-11-16 01:29:57 addcon thread interrupt
2016-11-16 01:29:57 scheduler thread interrupt
2016-11-16 01:29:57 net thread interrupt
2016-11-16 01:29:59 opencon thread interrupt
2016-11-16 01:29:59 Shutdown: In progress...
2016-11-16 01:29:59 StopNode()
2016-11-16 01:29:59 Corruption: block checksum mismatch
2016-11-16 01:29:59 *** System error while flushing: Database corrupted
2016-11-16 01:30:01 Shutdown: done
2016-11-16 01:30:09 

i don't understand why it gets corrupted, my computer works fantastically fine, i have zero problems, i am an IT, i don't use antivirus, i can detect a virus a mile away and remove it without any tool.
i do all kind of heavy work on this machine all the time and never have one single problem, except for the bitcoin wallet that gets corrupted all the time.

specs:
AMD APU A8-7600 on gigabyte F2A88X, 16GB DDR3 1866 kingston black, SSD (boot/bitcoin wallet) Samsung 840 Pro, win 7 64bit.

any idea of what can cause this?, it is not hardware, last crash i even left it overnight doing memory test to see if that was the issue but no, all fine.

i am lost with this issue.
if it is a block that get's corrupted, why do i need to rescan the entire blockchain, if it just can remove and update from the network the damaged block it will be done in minutes, not 30 hours (i can't even imagine how much longer would it take if i used a standard hard drive!).

thanks
sorry for the long post.
indkt.

[achow101]

The database is getting corrupted. This usually indicates some sort of hardware error.

What OS are you using?

[induktor]

The database is getting corrupted. This usually indicates some sort of hardware error.

What OS are you using?

Hi
Windows 7, 64 bit.
i asked around and most people told me that, hardware error, but i cannot reproduce it outside bitcoin-qt i have zero problems with this machine, i have been using it for two years, and i don't even remember one single freeze/haungup.

what are my options?
is there is any way to speed up the --reindex?, because it usually takes about 2 hours, but after the reindex starts a rescan, and the rescan is the process that takes more than a day to complete, and it is only using 20% CPU usage, and disk usage is also 20% so not really taking advantage of the machine capabilities.

[achow101]

Hi
Windows 7, 64 bit.

Double check that Windows defender isn't messing with the blockchain. It tends to do that.

i asked around and most people told me that, hardware error, but i cannot reproduce it outside bitcoin-qt i have zero problems with this machine, i have been using it for two years, and i don't even remember one single freeze/haungup.

Bitcoin Core is very hardware intensive. I suggest that you run full diagnostics on your hardware just to be sure that there isn't some issue.

what are my options?
is there is any way to speed up the --reindex?, because it usually takes about 2 hours, but after the reindex

Yes. You can use the --dbcache option and increase it. It will use more memory but it should also make the reindex much faster.

starts a rescan, and the rescan is the process that takes more than a day to complete, and it is only using 20% CPU usage, and disk usage is also 20% so not really taking advantage of the machine capabilities.

You are incorrect. Rescanning does not happen after a reindex. A rescan takes place at startup during the splash window. After the splash window you see the normal wallet view. Reindexing happens there, and it looks a lot like resyncing the blockchain because of the progress bar at the bottom.

[induktor]

Double check that Windows defender isn't messing with the blockchain. It tends to do that.

AHAAAAA!! that mother F&$*# of defender, disabled, done, it is very likely the issue!, i never thought about it!.

Bitcoin Core is very hardware intensive. I suggest that you run full diagnostics on your hardware just to be sure that there isn't some issue.

Did that the last crash, found nothing, i even swapped memory sticks with the other machine (same brand/type/specs), now, as an additional measure, i lowered the memory speed from 1866 (stock) to 1600Mhz.

Yes. You can use the --dbcache option and increase it. It will use more memory but it should also make the reindex much faster.

thank you!, running now with --dbcache=8192 --reindex
i should probably buy a newer/faster machine anyway   

starts a rescan, and the rescan is the process that takes more than a day to complete, and it is only using 20% CPU usage, and disk usage is also 20% so not really taking advantage of the machine capabilities.

You are incorrect. Rescanning does not happen after a reindex. A rescan takes place at startup during the splash window. After the splash window you see the normal wallet view. Reindexing happens there, and it looks a lot like resyncing the blockchain because of the progress bar at the bottom.

yep, my bad, you are right  , i meant after the reindexing it says processing blocks on disk, and that is the process that takes forever.

i bet that the issue was defender!.
thank you for taking the time to explain and help me 

[Hanslo]

I had a similar problem a couple of times. Now I'm using Norton Firewall after a consultant from Norton helped me to make exception, and 0.13.1 running ok.

But after Windows update I got crash like you. I shut down my computer and startet again. Startet bitcoin-qt wallet and after a couple of minuts it was up and go.

[induktor]

I am still rebuilding (processing blocks on disk) since last night, but the --dbcache=8192 did a HUGE impact on processing speed, now bitcoin-qt is using 8GB of ram and 100% CPU and ssd is at 90% load, and i am just at 5 weeks away from completion, so it would take less then 8 hours (previously was 30 hours without the --dbcache parameter), freaking fantastic!.

btw: i still need to buy a faster computer though i want that number lowered to 4 hours (reindex && processing blocks on disk time).

[induktor]

I had a similar problem a couple of times. Now I'm using Norton Firewall after a consultant from Norton helped me to make exception, and 0.13.1 running ok.

But after Windows update I got crash like you. I shut down my computer and startet again. Startet bitcoin-qt wallet and after a couple of minuts it was up and go.

Hi
usually if bitcoin-qt hangup due to a block corruption, it will work again, as long as it does not attempt to access that block, once that detects the fucked up block it will hangup again, so the only solution is to do a --reindex, which takes a very long time, and my main complain is that why, if it detects a damaged block, it does not redownload it, instead, it forces you to do a full disk reindex which is crazy!.

at least with the hyper-mega tip achow101 give me of use the --dbcache=hughememoryusenumber lowered the processing time a lot, but it still takes a long time (and i know my computer is two - three year old machine, i should get at least a 6th gen intel i5 with pci-e M2 SSD, will do as soon as i can get the money for it that kind of hw is very expensive here in my area )

the other way is to restore from backup, which i can do, i backup the entire blockchain to a 7z file once a month and keep three copies, but i wanted to find another solution to prevent this or solve it quickly.

3weeks behind... almost done!!

[Hanslo]

I had a similar problem a couple of times. Now I'm using Norton Firewall after a consultant from Norton helped me to make exception, and 0.13.1 running ok.

But after Windows update I got crash like you. I shut down my computer and startet again. Startet bitcoin-qt wallet and after a couple of minuts it was up and go.

Hi
usually if bitcoin-qt hangup due to a block corruption, it will work again, as long as it does not attempt to access that block, once that detects the fucked up block it will hangup again, so the only solution is to do a --reindex, which takes a very long time, and my main complain is that why, if it detects a damaged block, it does not redownload it, instead, it forces you to do a full disk reindex which is crazy!.

at least with the hyper-mega tip achow101 give me of use the --dbcache=hughememoryusenumber lowered the processing time a lot, but it still takes a long time (and i know my computer is two - three year old machine, i should get at least a 6th gen intel i5 with pci-e M2 SSD, will do as soon as i can get the money for it that kind of hw is very expensive here in my area )

the other way is to restore from backup, which i can do, i backup the entire blockchain to a 7z file once a month and keep three copies, but i wanted to find another solution to prevent this or solve it quickly.

3weeks behind... almost done!!

Here is how it looks for me:

Code:

2016-11-11 20:34:20 UpdateTip: new best=000000000000000001542e46acf8bd8901baae7dfa47f44950c48b139c88371a height=438440 version=0x20000000 log2_work=85.529235 tx=169959484 date='2016-11-11 20:15:51' progress=0.999995 cache=126.1MiB(62820tx)
2016-11-11 20:34:21 tor: Thread interrupt
2016-11-11 20:34:21 torcontrol thread exit
2016-11-11 20:34:21 scheduler thread interrupt
2016-11-11 20:34:21 addcon thread interrupt
2016-11-11 20:34:21 net thread interrupt
2016-11-11 20:34:21 opencon thread interrupt
2016-11-11 20:34:21 ProcessMessages: advertising address [2001:0:5ef5:79fd:24ba:1f45:a322:8ce1]:8333
2016-11-11 20:34:21 receive version message: /Satoshi:0.13.1/: version 70014, blocks=438444, us=92.221.115.30:59592, peer=5
2016-11-11 20:34:21 msghand thread interrupt
2016-11-11 20:34:21 Shutdown: In progress...
2016-11-11 20:34:21 StopNode()
2016-11-11 20:34:23 Corruption: block checksum mismatch
2016-11-11 20:34:23 *** System error while flushing: Database corrupted
2016-11-11 20:34:41 Shutdown: done

It did'nt reindex.

[mindrust]

I also had the same problem with the core wallet. It gotten corrupted and started to fix itself after i delete a few blocks. It took days to make it work again.

I deleted it and using electrum now.

 I felt like i was part of something while it was working but after it started to crap on me i realized that home users do not need to download 100gb of useless crap. Don't make any sense.

[induktor]

I also had the same problem with the core wallet. It gotten corrupted and started to fix itself after i delete a few blocks. It took days to make it work again.

I deleted it and using electrum now.

 I felt like i was part of something while it was working but after it started to crap on me i realized that home users do not need to download 100gb of useless crap. Don't make any sense.

I don't trust electrum wallet, maybe is my lack of knowledge, or paranoia, but, for me, bitcoin is real money, and i need to be absolutely sure of the reliability of the wallet, that's why i use the official (full node) wallet, i also use the phone (android) wallet for petty change, but for the primary wallet i wanna be sure i can rely on it.

i don't care about 100 gb of data if i know that my money is secure, that i can do transactions reliably whenever i need to, if that takes 100gb, or 200gb or 300gb of ssd storage, i buy a bigger/faster ssd, i can live with that, that's why i did this thread because i was worried that i didn't know why the wallet get screwed up every now and then, thanks to achow101 here, i think he nailed it with the defender messing with the data blocks from time to time, causing the wallet to crash.

probably i should try electrum as a secondary wallet, it's been around a long time and if it has problems we probably already read them in the forum right?

will give it a try  

[mindrust]

I also had the same problem with the core wallet. It gotten corrupted and started to fix itself after i delete a few blocks. It took days to make it work again.

I deleted it and using electrum now.

 I felt like i was part of something while it was working but after it started to crap on me i realized that home users do not need to download 100gb of useless crap. Don't make any sense.

I don't trust electrum wallet, maybe is my lack of knowledge, or paranoia, but, for me, bitcoin is real money, and i need to be absolutely sure of the reliability of the wallet, that's why i use the official (full node) wallet, i also use the phone (android) wallet for petty change, but for the primary wallet i wanna be sure i can rely on it.

i don't care about 100 gb of data if i know that my money is secure, that i can do transactions reliably whenever i need to, if that takes 100gb, or 200gb or 300gb of ssd storage, i buy a bigger/faster ssd, i can live with that, that's why i did this thread because i was worried that i didn't know why the wallet get screwed up every now and then, thanks to achow101 here, i think he nailed it with the defender messing with the data blocks from time to time, causing the wallet to crash.

probably i should try electrum as a secondary wallet, it's been around a long time and if it has problems we probably already read them in the forum right?

will give it a try  

I also had suspicions like you. After i deleted the core wallet, there was a topic popped up in the forum saying that the electrum has trojan. (1/50 of the online antivirus showed that it has a malicious code inside, probably false positive) Deleted electrum went back to the  core wallet. Then it have gotten corrupted again. I was mad again. Back to electrum

[unamis76]

What's your connection? Have you tried limiting the amount of acceptable incoming connections?

[induktor]

I have max connections set to 6
my ISP is shit, ADSL 3 MB download, 512K upload (and that is in a good day!)

I cannot order more speed because i am too far from the DSL base station and lost sync, so i had to go back to 3MB max.
if i choose cable provider (only second option to ADSL) it is 10 Mb down /1Mb up, but every cable company here throttle all P2P traffic, including bitcoin making it useless, it takes 20 times more to update the wallet using cable than using 3MB ADSL (since adsl does not do any P2P throttle).

so, unless i am willing to pay 10x more money to order a professional grade with SLA, i have to survive with ADSL until i move out  

i also have bandwidth capped bitcoin at the router level (mikrotik) so i can browse the web while i am updating the wallet, otherwise it would be impossible.

[unamis76]

All right, so I guess the network isn't the issue. I asked because from my experience, even machines with quite a bit of RAM tend to see bitcoind crashing due to too many connections filling up all the RAM.

[induktor]

All right, so I guess the network isn't the issue. I asked because from my experience, even machines with quite a bit of RAM tend to see bitcoind crashing due to too many connections filling up all the RAM.

Oh yes i read that like a year ago, probably the reason why i have maxconnections=6 hehe,  i think i did have that issue before, not recently though

it is good to know though that it can still be an issue, maxconnections will stay there
thanks!

[Abdussamad]

I can think of a few things that could cause this:

- Last I tried bitcoin core you had to manually shut it down before you shut down the computer i.e. it would not automatically shut down with the OS. If you didn't do this there would be DB corruption
- Like achow101 said DB corruption can be a sign of a hardware problem. In my experience it can be a sign of a failing hard drive.

If I was you I'd definitely make sure bitcoin core was shutting down correctly when you shut down your OS. If you live in an area with power cuts you have to get a UPS or this DB corruption will keep happening.

As for the second point you can run chkdsk to check your hard drive for bad sectors.

BTW electrum is one of the most widely used wallets. You shouldn't be afraid to use it.

[induktor]

Hello
I know that bitcoin-qt doesn't like when you shutdown the machine with the wallet running, i did it once and pay the price hehe

This machine has a SSD Samsung 840 PRO 250GB which is a pretty good ssd drive!, very reliable.
also this machine is connected to a APC SMART 2200XLI with extended battery, so even in a power outage scenario i have hours of operation, and even then, i have a generator (honda inverter, very clean output) hehe (yeah power utility in this country is a joke, you need to have all this crap to be able to work!)  

[Abdussamad]

Hello
I know that bitcoin-qt doesn't like when you shutdown the machine with the wallet running, i did it once and pay the price hehe

This machine has a SSD Samsung 840 PRO 250GB which is a pretty good ssd drive!, very reliable.
also this machine is connected to a APC SMART 2200XLI with extended battery, so even in a power outage scenario i have hours of operation, and even then, i have a generator (honda inverter, very clean output) hehe (yeah power utility in this country is a joke, you need to have all this crap to be able to work!)  

are you indian? anyway if it's not the ssd then it could be ram. run mem test for a couple of hours.

[induktor]

are you indian? anyway if it's not the ssd then it could be ram. run mem test for a couple of hours.

South american (argentina) power utility here is probably far worse than india.

Not ram, i swapped by identical sticks from another machine, run Memtest86 for an entire day, perfect both.
also i have a better way to test memory (sometimes Memtest86+ miss some memory hiccups) using quickpar2
probably the most memory intensive app i know, i create the par2 files for a 4GB iso image for example and
then check the par2 against the file, and if it fails, memory is the problem 99% of the time.

the problem is that quickpar is quite old, and doesn't like windows7 or newer, and also doesn't like more than
3gb of ram, so i have to virtualize it in order to be able to run it, but still the most reliable memory tester i found so far.

my bet is that the primary cause of the issue is what achow101 said in his first post, i think he nailed it. that
defender is from time to time messing with the blocks on disk and fucking things up, i disabled defender, we'll see
but my bet is that defender was the problem.