I'll take your word on that. I have no idea what UTXO is. ;-)

Only Bitcoin Core used this drive at that time.

I think it is a 5400 rpms drive, but I agree, 12 minutes is way too long.

Turns out there was a hardware issue. Some RAM I recently got hold of and added to my pc turned out to have a defect that only Bitcoin Core could consistently hit. I did run memtest, men it found nothing.

With the faulty RAM removed, and with some smaller RAM added, the system appears stable again.

I had to fetch all data again, though, but it turns out to be faster than reindexing (which takes forever).

8GB cache however is not enough to. I exit Bitcoin Core once in a while (takes quite a few minutes) when the speed has dropped a lot and it picks up speed again on the next start. Not sure why this is so, but it works.

From debug.log:

2020-06-21T07:09:46Z FlushStateToDisk: write coins cache to disk (38850948 coins, 5485362kB) started
2020-06-21T07:21:39Z FlushStateToDisk: write coins cache to disk (38850948 coins, 5485362kB) completed (713.00s)

That is almost 12 minutes to exit the program.

Before the answers arrived, I did what I wrote, removed the chainstate folder and restarted bitcoin core. It was a fix that worked earlier for me.

It still works on this, but very very slowly, and I am wondering why.

I gave the bitcoin core 12GB of RAM  to work with and it seems to have taken most of that. As a consequence, it almost doesn't use the harddisk (which is good, as it is a rotating one).


On closer look, I think Bitcoin Core has crashed.

Yup, it crashed 20 hours ago (but didn't display any crash box, just a GUI that was frozen). Corrupt block. Probably the same corrupt block as before.

I'll take a closer look at your suggestions and see what I can do.

Well, I did just that and now it is processing 10 years of blocks chains… I'll take a nap. ;-)

Hmm reindex… Is that where I delete the chainstate folder and start Bitcoin Core up again?

My bitcoin core just started to report this error after I shut down my pc, apparently not waiting long enough for BC to shut down.

I waited over 5 minutes, but bitcoin remained unresponsive, but busy working on the harddisk. Couldn't delay the reboot any longer. I hate that bitcoin can't shut down faster than once every 5-10 minutes…

Anyway, what is the cheapest way for me to rescue the database? Is it even possible?

I get some weird entries in the debug log:

2020-06-14T20:40:11Z Bound to 0.0.0.0:8333
2020-06-14T20:40:11Z init message: Loading P2P addresses...
2020-06-14T20:40:11Z *** Corrupt block found indicating potential hardware failure; shutting down
2020-06-14T20:40:11Z Error: A fatal internal error occurred, see debug.log for details
2020-06-14T20:40:11Z Loaded 73275 addresses from peers.dat  262ms
2020-06-14T20:40:11Z init message: Starting network threads...
2020-06-14T20:40:11Z net thread start
2020-06-14T20:40:11Z init message: Done loading
2020-06-14T20:40:11Z addcon thread start
2020-06-14T20:40:11Z opencon thread start
2020-06-14T20:40:11Z dnsseed thread start
2020-06-14T20:40:11Z msghand thread start
2020-06-14T20:40:14Z UPnP: ExternalIPAddress = 85.184.138.108
2020-06-14T20:40:14Z AddLocal(85.184.138.108:8333,3)
2020-06-14T20:40:14Z UPnP Port Mapping successful.
2020-06-14T20:40:22Z Loading addresses from DNS seed seed.bitcoin.sipa.be
2020-06-14T20:40:23Z Loading addresses from DNS seed seed.btc.petertodd.org
2020-06-14T20:40:23Z Loading addresses from DNS seed seed.bitcoin.jonasschnelli.ch
2020-06-14T20:40:34Z Loading addresses from DNS seed seed.bitcoinstats.com
2020-06-14T20:40:34Z Loading addresses from DNS seed dnsseed.bitcoin.dashjr.org
2020-06-14T20:40:34Z Loading addresses from DNS seed dnsseed.emzy.de
2020-06-14T20:40:46Z Loading addresses from DNS seed seed.bitcoin.sprovoost.nl
2020-06-14T20:40:46Z Loading addresses from DNS seed dnsseed.bluematt.me
2020-06-14T20:40:46Z 165 addresses found from DNS seeds
2020-06-14T20:40:46Z dnsseed thread exit
2020-06-14T20:41:44Z ERROR: ConnectTip: ConnectBlock 000000000000000000075751d2ce1e7b550617d3f1b060352dbbaedb1b959cb3 failed, bad-txnmrklroot, hashMerkleRoot mismatch (code 16)
2020-06-14T20:41:44Z Failed to connect best block (bad-txnmrklroot, hashMerkleRoot mismatch (code 16))
2020-06-14T20:41:44Z GUI: Platform customization: "windows"
2020-06-14T20:41:44Z GUI: QObject::connect: Cannot queue arguments of type 'size_t'
(Make sure 'size_t' is registered using qRegisterMetaType().)
2020-06-14T20:41:44Z tor: Thread interrupt
2020-06-14T20:41:44Z addcon thread exit

Not sure what really has gone wrong here, and even less how to fix this (although I know I can delete it all and wait several weeks for the resync)...

I have been running bitcoin core for many years with default settings. As I mentioned, the database died due to a windows update causing a BSOD.

And everyone claiming fast speeds are using SSD's. I have a rotating disk (for bitcoin data). I have several other disks in the system, mostly rotating, but also 2 SSD's (that I will not overload with wear from bitcoin, as it seems to want to trash the disk quite a bit).

I merely feel there is basis for optimizations so bitcoin synchronization could run much better on rotating disks, and to use less memory than 5+GB while doing so.

Anyway, my DB is synchronized and I will now reduce DB setting to the default 300MB again.

Thanks for all the comments.

I can see I was right to assume the harddisk was the problem.

But I can't replace it with a SSD as I have no more harddisk slots available and I cannot spare the large space offered by the rotating disks.

Anyway, I could be looking into moving the chainstate to a SSD, but I'd like to know the amount of writes it receives before doing so. Is it going to wear down an SSD fast (seeing as SLC is suggested)?

As mentioned, I set the buffer to 4096, and bitcoin now uses 5.6GB of memory. I guess there must be some extra memory consumed that scales with the buffer setting (besides the buffer itself).

Progress is now at 80% (up from 58% yesterday) so I guess a week more and I am done (if I don't do the chainstate optimisation). And if the chainstate is only about 4GB, maybe offer a fast sync, having bitcoin temporarily cache the chainstate in memory and only do bulk writes occasionally? That way, rotating disks would be able to keep up during synchronization.

Well, "much faster" was an overstatement. Currently it claims it needs 3 days to complete the synchronization (with 0.54% increase per hour).

It still trashes the disk a lot, so I guess most of the time is spent waiting for IO (seeks).

It uses 1.5GB ram, no network IO, and disk averages about 50MBps (which is much more than before).

I guess this is as good as it gets.

I did find a webpage offering torrent downloads of the bitcoin database, but it was discontinued and only an old database was available. That could have solved the synchronization issue in a few hours...

I managed to stop bitcoin (it was pretty much unresponsive while trashing the disk, allowing for a mouse click every minute or so) and had to find the configuration settings, which are not in bitcoin.conf. That file is empty.

I found the settings in the registry, and I modified this setting to read 4096:

nDatabaseCache

Then I started bitcoin again. After the initial loading of the index, it started synchronizing again, but much faster. Currently it is using 900MB memory (less than before) yet it performs much better.

Could there be a memory leak filling up the cache with needless information seeing as it is running "fine" (still slow) now and uses less memory than before where it ran horrible? Maybe a simple restart would have been enough?

Anyway, I hope you guys figure out a way to make synchronization be completed way faster. It should take hours, not days.

During windows update, my pc crashed for some unknown reason and that apparently affected my bitcoin data as it decided synchronize everything. During this it completely trashes my disk with seeks (rotating disk), and currently claims it will need 2 weeks to 5 years to finish synchronizing!

This simply wont do. It is a complete deterrent for anyone wanting to host full nodes.

Current progress increase per hour is at -0.00% (yes, it is negative). Just before it was at 0.05%.

There must be a quicker way to synchronize this. I mean, I could download the full archive in 2 hours, maxing my internet and disk speed. If only I could find someone offering a recent and reliable copy of the bitcoin data for me to download.

bitcoin.exe consumes approx 5% cpu, about 1 GB mem, and has IO approx 5-10MBps.

It feels like bitcoin is attempting to work on several blocks at the same time, which will absolutely kill a rotating disk. Is that something it would do? Can it be made to just take one block at a time?

I don't know how to speed this up, and at this point, I am considering just to stop hosting this node.

Bitcoin Client Software and Version Number: 19.0.1
Operating System: Windows 10 Pro
System Hardware Specs: Core i7 920 @ 3.6GHz, 500 Mbps internet, 12GB RAM, 7200 rpms 8TB disk for bitcoin data.
Description of Problem: So very slow at synchronizing.
Any Related Addresses: -
Any Related Transaction IDs: -
Screenshot of the problem: -
Log Files from the Bitcoin Client: -

This single bit seems to be the culprit for all my problems.

BitCoin has run for over 9 hours now and seems stable. Synchronizing takes a long time and it is only 49% complete so far. But it is stable.

Thanks for all the help and suggestions.

After having tried (and failed) to use a different disk, my conclusion was that it had to be memory error. I ran memtest86 and it actually did find a single bit error. First time I have had a memory fail on me.

I have now mapped the page out in windows and am attempting to run bitcoin again. If I was successful in mapping out all errors, I expect this to have solved the problem.

Now I can only wonder how many files have errors in them due to this memory error... BitCoin fortunately uses checksums, but not much else does. :-/

If BitCoin fails, I'll have a look at the wallet software.

I'd rather not trust websites with my wallet. Is there any other bitcoin software that lets me use my wallet from my pc directly?

Well, the linux bitcoin also crashed:

2017-04-30 17:31:39 LevelDB read failure: Corruption: block checksum mismatch
2017-04-30 17:31:39 Corruption: block checksum mismatch
2017-04-30 19:13:55 Error reading from database: Database corrupted

I guess my pc isn't good enough for bitcoin anymore, but how do I get my bitcoins back? The wallet.dat file is only usable with a working bitcoin.

I have run Bitcoin Core for a long time without issues. I have a few hundred GB set aside for Bitcoin and use an i7 with 12GB ram to run it on. It has been very stable until resuming from the latest standby. I have rebooted since and checked disks. No error found nor any stability issues located.

The ubuntu (Bash for Windows 10) attempt is happily running, estimating 8 hours left. Though upnp doesn't work (lacks features in Bash for Windows), I tried port forward, but due to a router firmware bug, this isn't working (Netgear R7000). I'll let it continue until done and then clone the result and try continuing with the windows client. If I am lucky, this will work, but I guess this will cause the database to become corrupted again.

My guess so far is that there is something in the windows client that corrupts the database, probably a bad library.

I am now trying to see if it works with the linux binary, running via Bash for Windows 10 (full ubuntu 14 install, running bitcoin-qt using VcXsrv as X-server). I'll report back...

And it of course crashed again due to the first crash causing database corruption. So it seems I can get approx 10% synchronized before database corruptions start occurring. Seeing as I haven't found anyone else with this problem, I can only assume my system is bad. But how do I debug a system that only exhibits problem with bitcoin?

And it crashed again. This time the debug log was more informative:

2017-04-30 13:39:41 LevelDB read failure: Corruption: block checksum mismatch
2017-04-30 13:39:41 Corruption: block checksum mismatch
2017-04-30 13:43:38 AddPortMapping(8333, 8333, 192.168.1.4) failed with code 402 (Invalid Args)
2017-04-30 14:04:37 AddPortMapping(8333, 8333, 192.168.1.4) failed with code 402 (Invalid Args)
2017-04-30 14:06:31 Error reading from database: Database corrupted

Crash location is different also:

Faulting application name: bitcoin-qt.exe, version: 0.14.1.0, time stamp: 0x58f87f32
Faulting module name: bitcoin-qt.exe, version: 0.14.1.0, time stamp: 0x58f87f32
Exception code: 0x40000015
Fault offset: 0x00000000012e8203
Faulting process id: 0x77b8
Faulting application start time: 0x01d2c1b4b9d5a7d9
Faulting application path: C:\Program Files\Bitcoin\bitcoin-qt.exe
Faulting module path: C:\Program Files\Bitcoin\bitcoin-qt.exe
Report Id: 8aa16d81-4345-4032-b538-90ce4501f197
Faulting package full name:
Faulting package-relative application ID:

My pc is otherwise completely stable, except bitcoin wont work properly.