|
Title: Initial block download drastically slowed down Post by: 28e7166c23e70c2014feb5d64 on August 31, 2025, 08:01:02 PM I am attempting to run a bitcoin node for the first time. I downloaded bitcoin core from the official website. I am running `bitcoind` in a Raspberry Pi 4 which has a Samsung T7 with approximately 220 GB still free (checked 2 minutes ago). I started downloading the blockchain on August 17, exactly 2 weeks ago. The first 10 or so days it was downloading about half a year of blockchain per day, and roughly constant speed. However, since the last 3-4 days this drastically slowed down, and now it downloads about one or two days of blockchain per day. At this rate, I will need like one more year to complete the initial download, instead of a few more days as with the initial download rate. There seems to be no error, it just became very slow. Is this normal?
Title: Re: Initial block download drastically slowed down Post by: LoyceV on August 31, 2025, 08:57:23 PM Which model Pi do you have exactly? Google tells me they range from 1 to 8 GB RAM, which is the first thing I suspect to be the cause.
Title: Re: Initial block download drastically slowed down Post by: 28e7166c23e70c2014feb5d64 on August 31, 2025, 09:39:23 PM It has 8 GB of RAM.
``` $ free -h total used free shared buff/cache available Mem: 7.6Gi 452Mi 79Mi 456Ki 7.2Gi 7.2Gi Swap: 511Mi 324Mi 187Mi ``` Title: Re: Initial block download drastically slowed down Post by: BitMaxz on September 01, 2025, 12:46:06 AM Did you edit the bitcoin.conf file?
If not, try editing it and add dbcache to 8GB of RAM. Since you have 8GB, dbcache=4000 might help speed things up. Or try this config below for Raspberry Pi.It was generated from the jlopp config generator setup for Raspberry Pi. Paste it to your Bitcoin.conf file and save, restart the node, and check for improvements. Code: # This config should be placed in following path: Title: Re: Initial block download drastically slowed down Post by: ABCbits on September 01, 2025, 09:00:34 AM Did you edit the bitcoin.conf file? If not, try editing it and add dbcache to 8GB of RAM. Since you have 8GB, dbcache=4000 might help speed things up. If OP doesn't use run other app that use lots of RAM, i would recommend higher value (dbcache=6500) and enable ZRAM. Or try this config below for Raspberry Pi.It was generated from the jlopp config generator setup for Raspberry Pi. Paste it to your Bitcoin.conf file and save, restart the node, and check for improvements. Code: # This config should be placed in following path: Setting by that generator website would slow down IBD process, since it reduce default dbcache value from 500 to 100. Title: Re: Initial block download drastically slowed down Post by: 28e7166c23e70c2014feb5d64 on September 01, 2025, 08:31:33 PM Code: Thank you for your comments. I have `dbcache=4096`. I still have Code: top - 21:26:42 up 1:02, 1 user, load average: 1.11, 1.14, 1.38 Also, using `iotop` I get: Code: Total DISK READ: 36.06 M/s | Total DISK WRITE: 0.00 B/s Title: Re: Initial block download drastically slowed down Post by: philipma1957 on September 01, 2025, 08:55:03 PM why oh why did you use a raps pi and not an inexpensive laptop
and what size is that external ssd 1tb? You are about the 20th person that has done this and asked the same questions. to all of them I say do not use a rasp pi. and use an empty 2tb ssd not a 1tb ssd lastly. 16gb ram is helpful. as an aside what interest connection do you have/ some services will punish you for doing a 600gb plus download. Title: Re: Initial block download drastically slowed down Post by: 28e7166c23e70c2014feb5d64 on September 02, 2025, 05:34:33 AM I left `bitcoind` running for some hours in a very powerful machine and can confirm that now it is going very fast again, so the bottleneck is something in the Raspberry Pi.
In the powerful machine I am running it with the default settings, i.e. no bitcoin.conf, except for `-datadir` which I point to the USB storage SSD. In this powerful machine, `pmap` reports a total memory usage of `bitcoind` of about 14e9. It seems that 8 GB of RAM is at least in part responsible for the bottleneck, no matter what you put in `dbcache`. Title: Re: Initial block download drastically slowed down Post by: LoyceV on September 02, 2025, 06:43:26 AM In this powerful machine, `pmap` reports a total memory usage of `bitcoind` of about 14e9. It seems that 8 GB of RAM is at least in part responsible for the bottleneck, no matter what you put in `dbcache`. Your chainstate directory will grow to about 12 GB, which means anything less than that largely increases disk activity.Title: Re: Initial block download drastically slowed down Post by: nc50lc on September 02, 2025, 07:48:28 AM It seems that 8 GB of RAM is at least in part responsible for the bottleneck, no matter what you put in `dbcache`. Your first reply has a clue on this.It may say 7.2Gi "available" memory, but that's after the 7.2Gi "buff/cache" are written to your disk. Given that Core is already using the drive extensively, increasing the database cache doesn't help as much since it will otherwise require your OS to swap more memory, further increasing the disk usage. Quote from: 28e7166c23e70c2014feb5d64 I have `dbcache=4096`. I still have -snip- For the new setting like dbcache to apply, you'll have to restart Core (just in case you missed it instead of the above)Title: Re: Initial block download drastically slowed down Post by: 28e7166c23e70c2014feb5d64 on September 02, 2025, 04:03:29 PM I am curious, if I may ask, why is it so memory hungry? From my limited understanding, while downloading the blockchain it is also validating it. This has to be done one block at a time, because of the nature of the blockchain itself (I guess I can validate a random block, but at some point I will need a hash of the previous, which I must check is valid before I can say this one is valid). If each block weights 1 MB, then how are we needing > 8 GB?
Title: Re: Initial block download drastically slowed down Post by: LoyceV on September 02, 2025, 04:29:52 PM I am curious, if I may ask, why is it so memory hungry? From my limited understanding, while downloading the blockchain it is also validating it. This has to be done one block at a time, because of the nature of the blockchain itself (I guess I can validate a random block, but at some point I will need a hash of the previous, which I must check is valid before I can say this one is valid). If each block weights 1 MB, then how are we needing > 8 GB? The blocks directory contains all confirmed transactions. The chainstate directory contains all knowledge on all existing UTXOs ("funds"). With each block Bitcoin Core downloads, it verifies all transactions in that block, and needs to check if no UTXO that doesn't exist gets spent (as that would mean Bitcoin is created out of thin air), and it updates chainstate by removing the spent UTXOs and adding the newly created ones. So for each block, it has to check (up to) thousands of UTXOs, and chainstate tends to grow over time. If it fits in your RAM, it's quite fast. If it doesn't come from RAM, it has to read and write thousands of UTXOs from disk for each block it verifies. Even on SSDs that just takes time, and USB only makes things worse.I'm not a blockchain expert, so this is my layman interpretation :) Title: Re: Initial block download drastically slowed down Post by: ABCbits on September 03, 2025, 08:54:02 AM So for each block, it has to check (up to) thousands of UTXOs, and chainstate tends to grow over time. If it fits in your RAM, it's quite fast. If it doesn't come from RAM, it has to read and write thousands of UTXOs from disk for each block it verifies. Even on SSDs that just takes time, and USB only makes things worse. For additional information, 1. https://statoshi.info/d/000000009/unspent-transaction-output-set?orgId=1&from=now-5y&to=now&timezone=browser&refresh=10m (https://statoshi.info/d/000000009/unspent-transaction-output-set?orgId=1&from=now-5y&to=now&timezone=browser&refresh=10m) shows total and size UTXO growth in last 5 years. Peak UTXO size was about 12.8GB. 2. AFAIK the read/write for those UTXO considered as random read/write operation. So you'll notice the slowdown if you use HDD, very cheap SSD or other slow storage. |