Very slow synchronizing with Bitcoin Core

[piosteu]

Hi.

I'm trying to start Bitcoin Core (version 0.19.0.1) and let it synchronize
with the network, from scratch.

Operating System: Windows 10 64-bit
System Hardware Specs: Intel Core I5-2500K 3.3 GHz, primary memory 8 GB

The first 15 hours went relatively well. Perhaps more than 40% of the
blocks were downloaded. Peak download rate was about 45 Mbits/s.

After another 22 hours, the speed has sunk to 0.05% per hour, or
5 weeks estimated to complete, with 58% completed. This speed makes it
virtually useless.

Current CPU-usage is less than 1%, memory usage is 530 MB, disk usage is
0.4 MB/s and network usage is 0 MBits/s.

I tried to set the parameter dbcache=3000, but that didn't help. I also
removed the file peers.dat (those actions after normal shutdowns).

Here follow that last lines in debug.log:

2020-02-16T13:00:02Z Pre-allocating up to position 0xd00000 in rev01148.dat
2020-02-16T13:00:02Z UpdateTip: new best=00000000000000000046e3c526e9e6acc1e357dec1bffc092b0cbb0c0c18db35 height=505415 version=0x20000000 log2_work=87.930032 tx=294386471 date='2018-01-21T20:45:40Z' progress=0.582162 cache=202.8MiB(1545911txo)
2020-02-16T13:00:02Z socket sending timeout: 1298s
2020-02-16T13:00:02Z socket sending timeout: 1290s
2020-02-16T13:00:02Z socket sending timeout: 1290s
2020-02-16T13:00:02Z socket sending timeout: 1295s
2020-02-16T13:00:08Z New outbound peer connected: version: 70015, blocks=617634, peer=62 (full-relay)
2020-02-16T13:00:08Z Synchronizing blockheaders, height: 617634 (~100.00%)
2020-02-16T13:00:08Z New outbound peer connected: version: 70015, blocks=617634, peer=63 (full-relay)
2020-02-16T13:00:09Z New outbound peer connected: version: 70015, blocks=617634, peer=64 (full-relay)
2020-02-16T13:00:12Z Pre-allocating up to position 0x4000000 in blk01152.dat
2020-02-16T13:00:39Z UpdateTip: new best=0000000000000000005fb11ce1e189a6899923ef5dbfcbd42400d6e4bfd028bf height=505416 version=0x20000000 log2_work=87.930079 tx=294387632 date='2018-01-21T20:52:46Z' progress=0.582164 cache=203.3MiB(1549962txo)
2020-02-16T13:01:06Z UpdateTip: new best=000000000000000000093817c3777f1b2a695007563bf2224112bbcccc6bca01 height=505417 version=0x20000000 log2_work=87.930126 tx=294388417 date='2018-01-21T20:59:23Z' progress=0.582165 cache=203.6MiB(1552923txo)
2020-02-16T13:01:24Z UpdateTip: new best=000000000000000000051334104dcebb2c47e341a0f92061a92b9bb9f1313483 height=505418 version=0x20000000 log2_work=87.930173 tx=294390615 date='2018-01-21T21:45:41Z' progress=0.582170 cache=204.2MiB(1557523txo)
2020-02-16T13:02:15Z UpdateTip: new best=0000000000000000004e461f792190f663b649f6ce84d61094f86f961886330c height=505419 version=0x20000000 log2_work=87.93022 tx=294391606 date='2018-01-21T21:54:27Z' progress=0.582171 cache=205.0MiB(1563852txo)
2020-02-16T13:02:50Z UpdateTip: new best=0000000000000000005337c72e1bcd92bbf6e8e3f059098aaab1d1d65c03ee6e height=505420 version=0x20000000 log2_work=87.930266 tx=294393141 date='2018-01-21T21:55:38Z' progress=0.582174 cache=205.6MiB(1568735txo)
2020-02-16T13:03:13Z UpdateTip: new best=0000000000000000004896b4329ebf39cc10775e67a45b7ceaa9955a12442f76 height=505421 version=0x20000000 log2_work=87.930313 tx=294394825 date='2018-01-21T22:11:44Z' progress=0.582178 cache=205.9MiB(1571913txo)
2020-02-16T13:03:40Z UpdateTip: new best=00000000000000000038536500cfcd9bc04e9b5d450520486c302e0db262f214 height=505422 version=0x20000000 log2_work=87.93036 tx=294396622 date='2018-01-21T22:32:14Z' progress=0.582181 cache=206.5MiB(1576277txo)
2020-02-16T13:04:04Z Pre-allocating up to position 0xe00000 in rev01148.dat
2020-02-16T13:04:04Z UpdateTip: new best=00000000000000000016d527ae1047ba007d6ab87572dc345395e063871306c9 height=505423 version=0x20000000 log2_work=87.930407 tx=294397643 date='2018-01-21T22:32:52Z' progress=0.582183 cache=207.0MiB(1580736txo)
2020-02-16T13:04:34Z UpdateTip: new best=0000000000000000000314920673b219335e8133f9cfff93257f8532e2798052 height=505424 version=0x20000000 log2_work=87.930454 tx=294398329 date='2018-01-21T22:33:15Z' progress=0.582184 cache=207.2MiB(1582437txo)
2020-02-16T13:05:03Z UpdateTip: new best=000000000000000000088501c46a3078fd46847cfc4c2a7a47c5aa6c0a6d9adc height=505425 version=0x20000000 log2_work=87.9305 tx=294399362 date='2018-01-21T22:37:28Z' progress=0.582186 cache=207.6MiB(1585605txo)
2020-02-16T13:05:27Z UpdateTip: new best=00000000000000000078fa3303e12426de9df84e1ad088d1c74f80221bb4a404 height=505426 version=0x20000000 log2_work=87.930547 tx=294400698 date='2018-01-21T22:47:41Z' progress=0.582189 cache=208.1MiB(1589512txo)
2020-02-16T13:05:51Z UpdateTip: new best=000000000000000000390413e01dbf7461b5452357c221e7f828672ae6a5b374 height=505427 version=0x20000000 log2_work=87.930594 tx=294401691 date='2018-01-21T22:51:43Z' progress=0.582190 cache=208.7MiB(1594336txo)
2020-02-16T13:06:16Z UpdateTip: new best=0000000000000000006bdb4230790f2ab8640dc3f9de34e33a6e9641da1c7a7c height=505428 version=0x20000000 log2_work=87.930641 tx=294402393 date='2018-01-21T22:53:34Z' progress=0.582192 cache=209.2MiB(1598233txo)
2020-02-16T13:06:31Z Pre-allocating up to position 0xf00000 in rev01148.dat
2020-02-16T13:06:31Z UpdateTip: new best=0000000000000000007833ab6a5414dfcf4c5e03e2d1e0000f3f38ebd9443f71 height=505429 version=0x20000000 log2_work=87.930688 tx=294404248 date='2018-01-21T22:57:39Z' progress=0.582195 cache=209.5MiB(1601041txo)
2020-02-16T13:06:53Z UpdateTip: new best=0000000000000000004114f85c0781e9297edfb9e2302119326ce452eef51293 height=505430 version=0x20000000 log2_work=87.930734 tx=294405053 date='2018-01-21T23:00:53Z' progress=0.582197 cache=209.8MiB(1603160txo)

Thanks for any help.

[nc50lc]

The first 15 hours went relatively well. Perhaps more than 40% of the
blocks were downloaded. Peak download rate was about 45 Mbits/s.

-snip-

Current CPU-usage is less than 1%, memory usage is 530 MB, disk usage is
0.4 MB/s and network usage is 0 MBits/s.

The earlier blocks didn't contain as much transactions as the latest blocks had or totally empty, that's why it's fast when just starting to sync.
The "full blocks" should be slower but yours however (0.05% per hour?) is extremely slower.

You have already set dbcache to 3GB and yet your system only uses 530MB of RAM.
Where did you set the dbcache, in the Qt interface or config file?
Are you using a different data directory?

Regarding the log, I didn't see any issue.

[piosteu]

The first 15 hours went relatively well. Perhaps more than 40% of the
blocks were downloaded. Peak download rate was about 45 Mbits/s.

-snip-

Current CPU-usage is less than 1%, memory usage is 530 MB, disk usage is
0.4 MB/s and network usage is 0 MBits/s.

The earlier blocks didn't contain as much transactions as the latest blocks had or totally empty, that's why it's fast when just starting to sync.
The "full blocks" should be slower but yours however (0.05% per hour?) is extremely slower.

You have already set dbcache to 3GB and yet your system only uses 530MB of RAM.
Where did you set the dbcache, in the Qt interface or config file?
Are you using a different data directory?

Regarding the log, I didn't see any issue.

Thanks for reply.

The dbcache setting is in the file bitcoin.conf, located in the data directory
(or the same directory as debug.log and peers.dat are in).

The data directory is not the default. I'm using another drive than C: for it.

Best regards

[jackg]

Is it an external drive)(one plugged in) or an internal one that you can't remove?

Was it a NAS or computer drive and not a backup drive too if it didn't come with the machine...

[piosteu]

Is it an external drive)(one plugged in) or an internal one that you can't remove?

Was it a NAS or computer drive and not a backup drive too if it didn't come with the machine...

The drive is an internal Western Digital 2 TB.

[piosteu]

I tried to guess the meaning of the following last lines in debug.log.
I guess that height denotes a block index.
At the time of writing, there are 617,656 blocks in the chain.
That number minus the height is roughly equal to the number of remaining blocks, ackording to BitCoin Core.
The height then denotes the number of blocks from the chain head.
So the system seems to download one block for each period of some 10..30 seconds.
Either the peer transmits data slowly, or my computer requests data very seldom. I don't know how
the network protocol works.

2020-02-16T17:11:56Z Pre-allocating up to position 0xe00000 in rev01152.dat
2020-02-16T17:11:56Z UpdateTip: new best=0000000000000000004706bfffdbd63f93da934b6424e9c4daa0b42e33c2cc5b height=505916 version=0x20000000 log2_work=87.953301 tx=295120952 date='2018-01-24T18:22:17Z' progress=0.583548 cache=293.3MiB(2169217txo)
2020-02-16T17:12:06Z UpdateTip: new best=0000000000000000005e950da6a28760512b9bc4b066d2a5794ba13b5d7de192 height=505917 version=0x20000000 log2_work=87.953348 tx=295123209 date='2018-01-24T18:34:31Z' progress=0.583553 cache=293.6MiB(2171430txo)
2020-02-16T17:12:26Z UpdateTip: new best=0000000000000000002274608b72232a0fcf154de22d5b3c3b31c65d0c142d2a height=505918 version=0x20000000 log2_work=87.953394 tx=295125294 date='2018-01-24T18:40:30Z' progress=0.583557 cache=294.0MiB(2174774txo)
2020-02-16T17:12:50Z UpdateTip: new best=000000000000000000661585a46ba29ab801ea6dce47073c348b07880e4ef7a6 height=505919 version=0x20000000 log2_work=87.95344 tx=295127287 date='2018-01-24T18:45:18Z' progress=0.583561 cache=294.4MiB(2178106txo)
2020-02-16T17:13:05Z UpdateTip: new best=0000000000000000001e631ec761fe400f4c93f02338d9181f9a0d64db0e5239 height=505920 version=0x20000000 log2_work=87.953486 tx=295129739 date='2018-01-24T19:04:42Z' progress=0.583566 cache=294.9MiB(2182568txo)
2020-02-16T17:13:34Z UpdateTip: new best=0000000000000000003a9720fac96878d7c599f0bea6f1e4ce8912932d555068 height=505921 version=0x20000000 log2_work=87.953532 tx=295130919 date='2018-01-24T19:05:44Z' progress=0.583568 cache=296.1MiB(2191738txo)

[LoyceV]

disk usage is 0.4 MB/s

With many small read/write actions on a HDD that might be the bottleneck. Can you check if the disk is active (there might be a LED blinking very fast)?

[piosteu]

disk usage is 0.4 MB/s

With many small read/write actions on a HDD that might be the bottleneck. Can you check if the disk is active (there might be a LED blinking very fast)?

Thanks.
With the winsat utility, I got these numbers, with BitCoin running at the same time:
> Disk  Random 16.0 Read                       1.95 MB/s          4.2
> Disk  Sequential 64.0 Read                   111.96 MB/s          6.8
> Disk  Sequential 64.0 Write                  155.71 MB/s          7.1

There is no LED.

Currently, Windows task manager denotes 1,5% CPU for BitCoin Core,
327 MB memory, 2.8 MB/s disk activity and 0 MBit/s network activity.
The task manager does not mark the disk activity as heavy (the percentage
is 34%).
Yes, the disk is a HDD. The OS is not stored on that disk.

[LoyceV]

The OS is not stored on that disk.

Does your OS run from an SSD? I'm not sure if you can pull this off in Windows, but in Linux I've seen great speed improvements by moving Bitcoin Core's "chainstate" directory to a SSD, and creating a symbolic link on it's original location. Some say it's not recommended because of the scenario in which one of the disks becomes unavailable, but I'm willing to run that risk. Just backup your wallet.dat, that's the only part you can't download again if anything ever fails.

Obviously you shouldn't move directories while Bitcoin Core is running.

[piosteu]

The OS is not stored on that disk.

Does your OS run from an SSD? I'm not sure if you can pull this off in Windows, but in Linux I've seen great speed improvements by moving Bitcoin Core's "chainstate" directory to a SSD, and creating a symbolic link on it's original location. Some say it's not recommended because of the scenario in which one of the disks becomes unavailable, but I'm willing to run that risk. Just backup your wallet.dat, that's the only part you can't download again if anything ever fails.

Obviously you shouldn't move directories while Bitcoin Core is running.

Thanks. That is very tempting to try, so yes, I have OS on an SSD. But unfortunately the SSD
is too small - too few unused GB:s.

Maybe bitcoind performs better than bitcoin-qt.exe?

Best regards

[LoyceV]

But unfortunately the SSD is too small - too few unused GB:s.

For reference: my chainstate directory is 4.0 GB.

[piosteu]

But unfortunately the SSD is too small - too few unused GB:s.

For reference: my chainstate directory is 4.0 GB.

Thanks for the size data. That makes it indeed possible to transfer to the SSD.

[nc50lc]

-snip-

The dbcache setting is in the file bitcoin.conf, located in the data directory
(or the same directory as debug.log and peers.dat are in).

The data directory is not the default. I'm using another drive than C: for it.

Based from your other replies, you're using bitcoin-Qt?
Try to increase the database cache (Size of database cache) in the "settings->options->Main tab" to half of your RAM or 4096.
If core didn't used the one in the config file, it will use that setting instead.

Or try to add dbcache=4096 to the bitcoin.conf in the default data directory, create one if you didn't have a conf file.

[NeuroticFish]

Last time I needed a full sync I made a symlink onto the SSD only for the chainstate and it did wonders.
And after the sync I moved the chainstate folder back to HDD.

[piosteu]

But unfortunately the SSD is too small - too few unused GB:s.

For reference: my chainstate directory is 4.0 GB.

After transferring the chainstate directory to the SSD, the speed has dramatically risen.
Thanks a lot.

Progress per hour is now nearly 0.9%.

I must have made a mistake when reading the numbers from task manager.
Probably the HDD was utilized at near maximum, and not only 34%, which I wrote earlier.
The utilization of the SSD is now 100%.

The dbcache setting is 3000.

Best regards

[LoyceV]

Progress per hour is now nearly 0.9%.

For reference: I recently re-downloaded the entire blockchain after I bought a bigger HDD (no more need for pruning). I'm using an old i3 running Linux with 16 GB RAM. Chainstate is on my SSD, the blocks go to the HDD. I increased dbcache to 4096 (only for downloading). This took in total about 24 hours, while also using my PC for other tasks.

[LFC_Bitcoin]

I don’t see anything unusual in the OP’s debug log.

When I claimed my free BCH a couple of years ago I bought another laptop & downloaded Bitcoin Core to transfer my bitcoin’s to (it was the first time I’d downloaded the entire blockchain since 2014).
It took literally 3 days to download the blockchain. It all depends on your internet speed & mine is kind of shit due to where I live.

I usually run on 1.00% per hour but it can be as bad as 0.02% per hour.

It’s a good idea to not leave it too long to catch up which is what I do quite often. 4 weeks behind can take me 2 hours at times.

[io321]

Sorry for reviving a relatively old post, but I encountered the same problem (stuck IBD, almost no CPU/disk load).

What fixed it was:

Code:

bitcoin-cli setnetworkactive false
bitcoin-cli setnetworkactive true

I'm guessing my IP may have changed and old TCP connections were stuck in limbo, so to speak. The fix was pretty much instant.