Restarting bitcoin-qt on HDD is extremely slow, lack some "preheat"?

[pseudo_geek]

OS: Win10 16299 x64
RAM: 16GB DDR3 1600 (dual channel)
CPU: Intel Core i7 4700MQ

My Bitcoin data dir is stored on my 5400rpm HDD. The OS is installed on 128GB SSD.
Though HDDs are expected to be slow, bitcoin-qt seems to be much more slower than my expectation.

Today, I was restarting bitcoin-qt full node on my PC, it showed that my node was 520 blocks behind the network. During the syncing process, taskmgr showed that HDD active time is 100%, but, the speed was EXTREMELY slow: validating one block seemed to take about one minute!

dbcache size was already set to 2048MB, which was not seemed to work.

Then, I tried some tricks to "boost" the syncing process:
1.Use procexp to suspend bitcoin-qt.exe;
2.Execute following command in cmd:

Code:

type D:\Bitcoin\data\chainstate\*.* > nul

3.Wait for the command to complete (took me about 5min).
After that, my taskmgr showed that the "cached" RAM size increased significantly.
4.Use procexp to resume bitcoin-qt.exe;

Dramatic things then happened: bitcoin-qt was significantly "boosted up". Time consumed by validating each block has shrunk to only several seconds!

I wonder if I made any mistakes? Doing such "manual optimization" is so strange...

[1714009074]

1714009074

[1714009074]

1714009074

[1714009074]

1714009074

[1714009074]

1714009074

[1714009074]

1714009074

[LoyceV]

First: I only have experience on Linux, with an i3 and 12 GB memory.

dbcache size was already set to 2048MB, which was not seemed to work.

I have never increased dbcache, it's set to the default 300 MB.

Code:

type D:\Bitcoin\data\chainstate\*.* > nul

3.Wait for the command to complete (took me about 5min).

I just tested "cat * > /dev/null", it took 0.685 seconds. That means the whole thing is in filecache already.
If reading 3 GB takes 5 minutes, you only get 10 MB/s. My guess is your hdd is fragmented too much.

Dramatic things then happened: bitcoin-qt was significantly "boosted up". Time consumed by validating each block has shrunk to only several seconds!

This is the speed I am used to.

I wonder if I made any mistakes? Doing such "manual optimization" is so strange...

I've seen many more threads about performance problems on Windows. I haven't used Windows in a long time, but my guess would be it's just bad at memory management.

I put my chainstate directory on my SSD a couple of months back, and created a simlink to that location. Note: this is not recommended for security reasons (but I don't worry about these).
This improved my Bitcoin Core startup time from 80 to 20 seconds. I would expect your slow fragmented chainstate to be much faster from SSD.

[pseudo_geek]

If reading 3 GB takes 5 minutes, you only get 10 MB/s. My guess is your hdd is fragmented too much.

Thanks. Just found that defragsvc was disabled unexpectedly.
(haha, someone encountered the same problem: [defragment-drives-dfrgui-exe-wont-start-after-1703-update from superuser - URL was automatically removed!? ] )
I've defraged my HDD, hope this will help.

[pseudo_geek]

Defragmentation indeed improved the performance, but "manual preheating" still got much more significant effect...

[cellard]

5400 rpm HDDs are ancient man, why are you still using that? The standard for HDDs is 7200, buying a 2 TB 7200 HDD is really cheap these days.

If you are paranoid about using SSD with anything Bitcoin (or if you can't afford a big sized SSD) you could also get those high quality raptor 10.000 HDDs for a decent price nowadays at +1 HDD capacity.

I believe by around 2020 one will be able to buy a 2 TB SSD m.2 for a reasonable price. I may cope with HDD until then.