Core v0.12.1 database error

[1337leet]

Hey guys,

I use Core v0.12.1 as wallet on my external hdd with datadir to the folder on the external hdd. This has worked just fine so far... but some weeks ago it started to make trouble..

I used v0.11.0 and got "error while reading the database" - so I reindext - all worked fine. I closed the wallet and restarted it - same problem again.

So I updated to v0.12.1 and reindexed - all worked fine - I closed it and restarted and I have the exact same error again....

2016-04-29 22:42:29 Bitcoin version v0.12.1 (2016-04-11 13:01:43 +0200)
2016-04-29 22:42:29 InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1
2016-04-29 22:42:29 GUI: "registerShutdownBlockReason: Successfully registered: Bitcoin Core didn't yet exit safely..."
2016-04-29 22:42:31 Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
2016-04-29 22:42:31 Default data directory C:\Users\Nils\AppData\Roaming\Bitcoin
2016-04-29 22:42:31 Using data directory W:\Mining\Bitcoin
2016-04-29 22:42:31 Using config file W:\Mining\Bitcoin\bitcoin.conf
2016-04-29 22:42:31 Using at most 125 connections (2048 file descriptors available)
2016-04-29 22:42:31 Using 4 threads for script verification
2016-04-29 22:42:31 Using wallet wallet.dat
2016-04-29 22:42:31 scheduler thread start
2016-04-29 22:42:31 init message: Verifiziere Wallet...
2016-04-29 22:42:31 CDBEnv::Open: LogDir=W:\Mining\Bitcoin\database ErrorFile=W:\Mining\Bitcoin\db.log
2016-04-29 22:42:31 Bound to [::]:8333
2016-04-29 22:42:31 Bound to 0.0.0.0:8333
2016-04-29 22:42:31 Cache configuration:
2016-04-29 22:42:31 * Using 2.0MiB for block index database
2016-04-29 22:42:31 * Using 32.5MiB for chain state database
2016-04-29 22:42:31 * Using 65.5MiB for in-memory UTXO set
2016-04-29 22:42:31 init message: Lade Blockindex...
2016-04-29 22:42:31 Opening LevelDB in W:\Mining\Bitcoin\blocks\index
2016-04-29 22:42:31 Opened LevelDB successfully
2016-04-29 22:42:31 Using obfuscation key for W:\Mining\Bitcoin\blocks\index: 0000000000000000
2016-04-29 22:42:31 Opening LevelDB in W:\Mining\Bitcoin\chainstate
2016-04-29 22:42:31 Opened LevelDB successfully
2016-04-29 22:42:31 Using obfuscation key for W:\Mining\Bitcoin\chainstate: 099110f582227f13
2016-04-29 22:42:35 LoadBlockIndexDB: last block file = 501
2016-04-29 22:42:35 LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=15, size=13107917, heights=408728...408742, time=2016-04-24...2016-04-24)
2016-04-29 22:42:35 Checking all blk files are present...
2016-04-29 22:42:35 LoadBlockIndexDB: transaction index disabled
2016-04-29 22:42:35 LoadBlockIndexDB: hashBestChain=000000000000000002c17eb95c343027858e592bc427264f2d4113df5bf7d742 height=408742 date=2016-04-24 16:04:06 progress=0.996686
2016-04-29 22:42:35 init message: Verifiziere Blöcke...
2016-04-29 22:42:35 Verifying last 288 blocks at level 3
2016-04-29 22:42:47 No coin database inconsistencies in last 9 blocks (16143 transactions)
2016-04-29 22:42:47  block index           15968ms
2016-04-29 22:42:47 init message: Lade Wallet...
2016-04-29 22:42:47 nFileVersion = 120100
2016-04-29 22:42:47 Keys: 0 plaintext, 215 encrypted, 215 w/ metadata, 215 total
2016-04-29 22:42:47  wallet                  264ms
2016-04-29 22:42:47 init message: Aktiviere beste Blockkette...
2016-04-29 22:42:47 mapBlockIndex.size() = 408745
2016-04-29 22:42:47 nBestHeight = 408742
2016-04-29 22:42:47 setKeyPool.size() = 100
2016-04-29 22:42:47 mapWallet.size() = 24
2016-04-29 22:42:47 mapAddressBook.size() = 15
2016-04-29 22:42:47 init message: Lade Adressen...
2016-04-29 22:42:47 torcontrol thread start
2016-04-29 22:42:47 ERROR: Read: Failed to open file W:\Mining\Bitcoin\banlist.dat
2016-04-29 22:42:47 Invalid or missing banlist.dat; recreating
2016-04-29 22:42:47 Loaded 55626 addresses from peers.dat  247ms
2016-04-29 22:42:47 dnsseed thread start
2016-04-29 22:42:47 upnp thread start
2016-04-29 22:42:47 net thread start
2016-04-29 22:42:47 addcon thread start
2016-04-29 22:42:47 opencon thread start
2016-04-29 22:42:47 init message: Laden abgeschlossen
2016-04-29 22:42:47 msghand thread start
2016-04-29 22:42:47 GUI: Platform customization: "windows"
2016-04-29 22:42:47 GUI: PaymentServer::LoadRootCAs: Loaded  39  root certificates
2016-04-29 22:42:48 receive version message: /Satoshi:0.12.0/: version 70012, blocks=409486, us=134.3.108.26:53155, peer=1
2016-04-29 22:42:48   nActualTimespan = 1209731  before bounds
2016-04-29 22:42:48 GetNextWorkRequired RETARGET
2016-04-29 22:42:48 params.nPowTargetTimespan = 1209600    nActualTimespan = 1209731
2016-04-29 22:42:48 Before: 1806274b  000000000000000006274b000000000000000000000000000000000000000000
2016-04-29 22:42:48 After:  18062776  0000000000000000062776acbefbefbefbefbefbefbefbefbefbefbefbefbefb
2016-04-29 22:42:49 UPnP: ExternalIPAddress = 134.3.108.26
2016-04-29 22:42:49 AddLocal(134.3.108.26:8333,3)
2016-04-29 22:42:49 AddPortMapping(8333, 8333, 192.168.178.33) failed with code 403 (UnknownError)
2016-04-29 22:42:50 UpdateTip: new best=00000000000000000313b4c769cf689877380e0ebfc0b4c65fc42400fdc8097f  height=408743  log2_work=84.543911  tx=124456535  date=2016-04-24 16:43:05 progress=0.996703  cache=5.6MiB(6724tx)
2016-04-29 22:42:51 LevelDB read failure: Corruption: block checksum mismatch
2016-04-29 22:42:51 Corruption: block checksum mismatch
2016-04-29 22:42:59 Loading addresses from DNS seeds (could take a while)
2016-04-29 22:43:04 136 addresses found from DNS seeds
2016-04-29 22:43:04 dnsseed thread exit
2016-04-29 22:46:09 Error reading from database: Database corrupted

[1714892278]

1714892278

[1714892278]

1714892278

[achow101]

This is interesting. You will have to reindex again. However, since this has occurred multiple times, you might actual have hardware errors. I would suggest that you run some diagnostics on your hard drive and see if there are any problems.

[1337leet]

The HDD is fine - I use it for several tasks - never had any Problems.

I reindexed 4 times in the past 6 Weeks and the problem came back 2-3 days later.

The HDD is encrypted - I reindex - after that the wallet syncs and works fine.
Then I close the wallet and unmount the external HDD.
I mount it again and open the wallet again - and I will get the database error again.

But that's exactely how I did it before - and before it worked fine for 2-3 years.

Maybe I try to delete the whole Wallet and reinstall it and just download the blockchain again.

Do I need to save anything else than the wallet.dat file ?

[achow101]

The HDD is fine - I use it for several tasks - never had any Problems.

I reindexed 4 times in the past 6 Weeks and the problem came back 2-3 days later.

The HDD is encrypted - I reindex - after that the wallet syncs and works fine.
Then I close the wallet and unmount the external HDD.
I mount it again and open the wallet again - and I will get the database error again.

But that's exactely how I did it before - and before it worked fine for 2-3 years.

Honestly, then, I'm not sure what is wrong. You should open a issue on the github: https://github.com/bitcoin/bitcoin/issues. Or if you don't want, I can do it for you. You will need to include the debug.log and mention that you are using an encrypted external hard drive.

Also, software changes over time so newer versions may not behave the same way as a version from 2 or 3 years ago.

Maybe I try to delete the whole Wallet and reinstall it and just download the blockchain again.

Do I need to save anything else than the wallet.dat file ?

Yes, you just need the wallet.dat. You can try to do that, and if it doesn't help, then open a issue.

[1337leet]

The HDD is fine - I use it for several tasks - never had any Problems.

I reindexed 4 times in the past 6 Weeks and the problem came back 2-3 days later.

The HDD is encrypted - I reindex - after that the wallet syncs and works fine.
Then I close the wallet and unmount the external HDD.
I mount it again and open the wallet again - and I will get the database error again.

But that's exactely how I did it before - and before it worked fine for 2-3 years.

Honestly, then, I'm not sure what is wrong. You should open a issue on the github: https://github.com/bitcoin/bitcoin/issues. Or if you don't want, I can do it for you. You will need to include the debug.log and mention that you are using an encrypted external hard drive.

Also, software changes over time so newer versions may not behave the same way as a version from 2 or 3 years ago.

Maybe I try to delete the whole Wallet and reinstall it and just download the blockchain again.

Do I need to save anything else than the wallet.dat file ?

Yes, you just need the wallet.dat. You can try to do that, and if it doesn't help, then open a issue.

Yes - I already deleted the old one completely and installed it new to another folder and now download the blockchain. Let's see if this works.

Thanks for you help.

[1337leet]

I reloaded the whole Blockchain until it was synced, then I closed the wallet and unmounted the external HDD.
Now I mounted it again and started the wallet  it synced up to 3 hours and then got a fatal error.
It's not the same error. It doesn't say "database error" it just said "A major error occured".

I'm fed up with this shit...

2016-05-02 12:22:13 mapBlockIndex.size() = 409873
2016-05-02 12:22:13 nBestHeight = 409855
2016-05-02 12:22:13 setKeyPool.size() = 100
2016-05-02 12:22:13 mapWallet.size() = 25
2016-05-02 12:22:13 mapAddressBook.size() = 15
2016-05-02 12:22:13 init message: Lade Adressen...
2016-05-02 12:22:13 torcontrol thread start
2016-05-02 12:22:13 ERROR: Read: Failed to open file W:\Mining\_Bitcoin_\Bitcoin\banlist.dat
2016-05-02 12:22:13 Invalid or missing banlist.dat; recreating
2016-05-02 12:22:13 Loaded 38834 addresses from peers.dat  343ms
2016-05-02 12:22:13 dnsseed thread start
2016-05-02 12:22:13 net thread start
2016-05-02 12:22:13 upnp thread start
2016-05-02 12:22:13 addcon thread start
2016-05-02 12:22:13 opencon thread start
2016-05-02 12:22:13 msghand thread start
2016-05-02 12:22:13 init message: Laden abgeschlossen
2016-05-02 12:22:13 GUI: Platform customization: "windows"
2016-05-02 12:22:13 GUI: PaymentServer::LoadRootCAs: Loaded  40  root certificates
2016-05-02 12:22:15 UPnP: ExternalIPAddress = 134.3.108.26
2016-05-02 12:22:15 AddLocal(134.3.108.26:8333,3)
2016-05-02 12:22:15 AddPortMapping(8333, 8333, 192.168.178.33) failed with code 403 (UnknownError)
2016-05-02 12:22:19 ProcessMessages: advertizing address 134.3.108.26:8333
2016-05-02 12:22:19 receive version message: /Satoshi:0.12.0/: version 70012, blocks=409875, us=134.3.108.26:51201, peer=1
2016-05-02 12:22:19 AdvertizeLocal: advertizing address 134.3.108.26:8333
2016-05-02 12:22:25 Loading addresses from DNS seeds (could take a while)
2016-05-02 12:22:30 136 addresses found from DNS seeds
2016-05-02 12:22:30 dnsseed thread exit
2016-05-02 12:22:57 UpdateTip: new best=0000000000000000044031a64310b6ac5b399c70c910dc4770f00520efae6924  height=409856  log2_work=84.586957  tx=126221635  date=2016-05-02 09:38:31 progress=0.999929  cache=34.5MiB(17488tx)
2016-05-02 12:22:57 UpdateTip: 4 of last 100 blocks have unexpected version
2016-05-02 12:23:40 UpdateTip: new best=0000000000000000053bfc49340a416a7d67d2a1f5b4f985a900671fdf3ffc0b  height=409857  log2_work=84.586995  tx=126224754  date=2016-05-02 10:07:57 progress=0.999942  cache=36.4MiB(22567tx)
2016-05-02 12:23:40 UpdateTip: 4 of last 100 blocks have unexpected version

[Fever090]

I have the same error. My OS is ubuntu 14.04.1 Linux 4.2.0.37, HDD Western Digital Red 1TB. I have performed extensive surface testing through SMART, it has taken 3 hours /with linux distro Parted Magic "DISK HEALTH"/ - no problem was found. I do not use encryption. I think my hardware is O.K. I have performed reindex but in "16-th weeks" the reindex was extreme slow my debug log files was growing and growing and now it has more than 57 MB size.

2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing transaction
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing output
2016-06-01 17:13:57 ERROR: ApplyTxInUndo: undo data overwriting existing output

When I started bitcoin-qt again without reindex then reindex was not continue, I had to reindex again from first block. I plan  to unnistall bitcoin 12.1 and its dependencies. I know, that in last version the libraries was updated and optimalized for bitcoin - and bitcoin should run much more quicker - it was true - bitcoin-qt was quicker but after last sync I am not able to perform full reindex. More over I always do backup of entire blockchain to second partition on the same WD RED. I use ext4 filesystem. I always do backup and when "copy" is finished I always type in terminal command "sync". Then I ran litecoin 10.4 without any problem.
So I feel really frustrated by this "bitcoin-qt" problem.

So something is bad, but I think not with my hardware. Please let me give some advice. I wish to have full blockchain on my computer.



UPDATE:
So my system: 4GB RAM, WD 1TB, 2 partitions exclusively for blockchain. I do not use default bitcoin-qt directory .bitcoin. I have it on separate partition. The second "blockchain" partition is for backup purpose.
My OS is Xubuntu 14.04.1 64bit. I have tested RAM with memmtest /running 4 hours/ - no problem. I have tested WD 1TB with utility "DISK HEALTH", which I have found in Parted Magic distro /running 3 hours 33 minutes/, this test has examined disk surface - so called built in SMART test - no problem was found. My PSU is Seasonic so the voltage stability is great. My motherboard is is Gigabyte with ULTRA DURABLE 4. So my hardware is O.K.

Bitcoin-qt client 12.01. I have not problem with previous version. I have installed bitcoin-qt in linux with PPA.

I have reinstalled bitcoin-qt client 12.01 with all dependencies with apt-get. My last reindex after reinstallation was successfull. But problem was with RAM. Bitcoin-qt has consumed 3340 MB of RAM. My settings for "SIZE OF DATABASE CACHE" was 2000MB. My PC was slow such as old 486 with Win95. I have stopped bitcoin-qt in standard way. The next start ended with message: "Database is corrupted, do you wish rebuilt". I have pressed yes and now I am reindexing. But my new settings for "SIZE OF DATABASE" is now only 500MB. So now I am reindexing. I will write here the result. When someone has similar problem please let us to know about it, maybe together find some solution. But I can not imagine the situation, when the blockchain grows after years for example over 300 GB and then I  need to do reindex with HDD spinner and with my celeron cpu.


I always perform HDD optimalization. I run bitcoin-qt with "custom blockchain directory", which is on separate partition. The synchronization with network is done - I close bitcoin-qt. Then I reboot PC with Parted Magic. I format the second "blockchain partition" and copy all data from updated "blockchain partition" to second formated "blockchain partition". I copy data with standard file manager and then I type in terminal sync.

When this operation is done. I format the first updated "blockchain partition" and copy data back from second "blockchain partition" and type sync in terminal. The main reason is: I have fresh ex4 file system without fragments. This solutions worked perfectly up to 12.01 bitcoin-qt  /database is corrupted/. I have litecoin 10.4 which has its default directory on my "blockchain partition", too and it works perfectly without any problem.

blockchain partition = is the partition in which I have all files which should be in default .bitcoin directory located in /home directory. I have of course in this partition directories such as "BITCOIN BLOCKS", "LITECOIN BLOCKS" and so on.  



UPDATE: I have performed full reindex. My last sync with bitcoin network was 16 weeks ago. My settings for RAM ussage by database cache in bitcoin-qt is 1000MB. My total RAM 4096MB - 128MB for graphic memmory. I was 33 hours behind with bitcoin-qt client, and suddenly error occured and bitcoin-qt closed unexpectedly, due to small amount of free RAM the problem could not be analysed and sent to developers. So with bitcoin-qt 12.1 is something wrong.

Now, when I imagine, that buggy bitcoin-qt client reindexed my entire blockchain 4 times - wau maybe I will need to download entire blockchain once again. Next time I will wait with update or I will use litecoin instead.
Bitcoin costs now 520 USD its alltime high is cca 1100 USD. Litecoin now costs 4.12 USD its all-time high is about 43 USD. Maybe I should change my BTC to LTC with higher potential. So please repair bitcoin-qt.