Initial block download drastically slowed down

[28e7166c23e70c2014feb5d64]

I am attempting to run a bitcoin node for the first time. I downloaded bitcoin core from the official website. I am running `bitcoind` in a Raspberry Pi 4 which has a Samsung T7 with approximately 220 GB still free (checked 2 minutes ago). I started downloading the blockchain on August 17, exactly 2 weeks ago. The first 10 or so days it was downloading about half a year of blockchain per day, and roughly constant speed. However, since the last 3-4 days this drastically slowed down, and now it downloads about one or two days of blockchain per day. At this rate, I will need like one more year to complete the initial download, instead of a few more days as with the initial download rate. There seems to be no error, it just became very slow. Is this normal?

[LoyceV]

Which model Pi do you have exactly? Google tells me they range from 1 to 8 GB RAM, which is the first thing I suspect to be the cause.

[28e7166c23e70c2014feb5d64]

It has 8 GB of RAM.

```
$ free -h
               total        used        free      shared  buff/cache   available
Mem:           7.6Gi       452Mi        79Mi       456Ki       7.2Gi       7.2Gi
Swap:          511Mi       324Mi       187Mi
```

[BitMaxz]

Did you edit the bitcoin.conf file?
If not, try editing it and add dbcache to 8GB of RAM. Since you have 8GB, dbcache=4000 might help speed things up.


Or try this config below for Raspberry Pi.It was generated from the jlopp config generator setup for Raspberry Pi. Paste it to your Bitcoin.conf file and save, restart the node, and check for improvements.

Code:

# This config should be placed in following path:
# ~/.bitcoin/bitcoin.conf

# [core]
# Set database cache size in MB; machines sync faster with a larger cache. Recommend setting to 4000 if RAM is available.
dbcache=100
# Keep at most <n> unconnectable transactions in memory.
maxorphantx=10
# Keep the transaction memory pool below <n> megabytes.
maxmempool=50

# [network]
# Maintain at most N connections to peers.
maxconnections=40
# Tries to keep outbound traffic under the given target (in MiB per 24h), 0 = no limit.
maxuploadtarget=5000


# [Sections]
# Most options automatically apply to mainnet, testnet, and regtest networks.
# If you want to confine an option to just one network, you should add it in the relevant section.
# EXCEPTIONS: The options addnode, connect, port, bind, rpcport, rpcbind and wallet
# only apply to mainnet unless they appear in the appropriate section below.

# Options only for mainnet
[main]

# Options only for testnet
[test]

# Options only for regtest
[regtest]

[ABCbits]

Did you edit the bitcoin.conf file?
If not, try editing it and add dbcache to 8GB of RAM. Since you have 8GB, dbcache=4000 might help speed things up.

If OP doesn't use run other app that use lots of RAM, i would recommend higher value (dbcache=6500) and enable ZRAM.

Or try this config below for Raspberry Pi.It was generated from the jlopp config generator setup for Raspberry Pi. Paste it to your Bitcoin.conf file and save, restart the node, and check for improvements.

Code:

# This config should be placed in following path:
# ~/.bitcoin/bitcoin.conf

# [core]
# Set database cache size in MB; machines sync faster with a larger cache. Recommend setting to 4000 if RAM is available.
dbcache=100
# Keep at most <n> unconnectable transactions in memory.
maxorphantx=10
# Keep the transaction memory pool below <n> megabytes.
maxmempool=50

# [network]
# Maintain at most N connections to peers.
maxconnections=40
# Tries to keep outbound traffic under the given target (in MiB per 24h), 0 = no limit.
maxuploadtarget=5000


# [Sections]
# Most options automatically apply to mainnet, testnet, and regtest networks.
# If you want to confine an option to just one network, you should add it in the relevant section.
# EXCEPTIONS: The options addnode, connect, port, bind, rpcport, rpcbind and wallet
# only apply to mainnet unless they appear in the appropriate section below.

# Options only for mainnet
[main]

# Options only for testnet
[test]

# Options only for regtest
[regtest]

Setting by that generator website would slow down IBD process, since it reduce default dbcache value from 500 to 100.

[28e7166c23e70c2014feb5d64]

Code:

Thank you for your comments. I have `dbcache=4096`. I still have 
[code]
$ free -h
               total        used        free      shared  buff/cache   available
Mem:           7.6Gi       280Mi        85Mi       4.0Ki       7.4Gi       7.4Gi
Swap:          511Mi       337Mi       174Mi

and `top` is:

Code:

top - 21:26:42 up  1:02,  1 user,  load average: 1.11, 1.14, 1.38
Tasks: 172 total,   1 running, 171 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.3 us,  4.0 sy,  0.0 ni, 67.8 id, 27.8 wa,  0.0 hi,  0.1 si,  0.0 st 
MiB Mem :   7809.4 total,     78.1 free,    278.7 used,   7574.2 buff/cache     
MiB Swap:    512.0 total,    174.5 free,    337.5 used.   7530.8 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                                                 
    742 btcusr    20   0 9912.5m 881320 833248 S   6.9  11.0  11:42.52 bitcoind

Also, using `iotop` I get:

Code:

Total DISK READ:        36.06 M/s | Total DISK WRITE:         0.00 B/s
Current DISK READ:      36.06 M/s | Current DISK WRITE:       3.75 K/s
    TID  PRIO  USER     DISK READ DISK WRITE>    COMMAND                                                                                                                                       
    773 be/4 btcusr     36.06 M/s    0.00 B/s bitcoind [b-initload]

all the time. This could be the bottleneck.[/code]

[philipma1957]

why oh why did you use a raps pi and not an inexpensive laptop

and what size is that external ssd 1tb?

You are about the 20th person that has done this and asked the same questions.

to all of them I say do not use a rasp pi.

and use an empty 2tb ssd

not a 1tb ssd

lastly. 16gb ram is helpful.



as an aside what interest connection do you have/

some services will punish you for doing a 600gb plus download.

[28e7166c23e70c2014feb5d64]

I left `bitcoind` running for some hours in a very powerful machine and can confirm that now it is going very fast again, so the bottleneck is something in the Raspberry Pi.

In the powerful machine I am running it with the default settings, i.e. no bitcoin.conf, except for `-datadir` which I point to the USB storage SSD. In this powerful machine, `pmap` reports a total memory usage of `bitcoind` of about 14e9. It seems that 8 GB of RAM is at least in part responsible for the bottleneck, no matter what you put in `dbcache`.

[LoyceV]

In this powerful machine, `pmap` reports a total memory usage of `bitcoind` of about 14e9. It seems that 8 GB of RAM is at least in part responsible for the bottleneck, no matter what you put in `dbcache`.

Your chainstate directory will grow to about 12 GB, which means anything less than that largely increases disk activity.

[nc50lc]

It seems that 8 GB of RAM is at least in part responsible for the bottleneck, no matter what you put in `dbcache`.

Your first reply has a clue on this.

It may say 7.2Gi "available" memory, but that's after the 7.2Gi "buff/cache" are written to your disk.
Given that Core is already using the drive extensively, increasing the database cache doesn't help as much
since it will otherwise require your OS to swap more memory, further increasing the disk usage.

I have `dbcache=4096`. I still have -snip-

For the new setting like dbcache to apply, you'll have to restart Core (just in case you missed it instead of the above)

[28e7166c23e70c2014feb5d64]

I am curious, if I may ask, why is it so memory hungry? From my limited understanding, while downloading the blockchain it is also validating it. This has to be done one block at a time, because of the nature of the blockchain itself (I guess I can validate a random block, but at some point I will need a hash of the previous, which I must check is valid before I can say this one is valid). If each block weights 1 MB, then how are we needing > 8 GB?

[LoyceV]

I am curious, if I may ask, why is it so memory hungry? From my limited understanding, while downloading the blockchain it is also validating it. This has to be done one block at a time, because of the nature of the blockchain itself (I guess I can validate a random block, but at some point I will need a hash of the previous, which I must check is valid before I can say this one is valid). If each block weights 1 MB, then how are we needing > 8 GB?

The blocks directory contains all confirmed transactions. The chainstate directory contains all knowledge on all existing UTXOs ("funds"). With each block Bitcoin Core downloads, it verifies all transactions in that block, and needs to check if no UTXO that doesn't exist gets spent (as that would mean Bitcoin is created out of thin air), and it updates chainstate by removing the spent UTXOs and adding the newly created ones. So for each block, it has to check (up to) thousands of UTXOs, and chainstate tends to grow over time. If it fits in your RAM, it's quite fast. If it doesn't come from RAM, it has to read and write thousands of UTXOs from disk for each block it verifies. Even on SSDs that just takes time, and USB only makes things worse.

I'm not a blockchain expert, so this is my layman interpretation