22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 07, 2024, 12:40:39 PM |
|
I have a fairly robust dedicated pc running ubuntu: Processor Intel Core i5 Memory 8gb Disk 500 gb ssd
The progress started quickly with about 70% in a couple of days. Then it slowed down dramatically to where now like today for example it has downloaded not even 1% in the last 12 hours, maybe getting like 2% per day at this point.
Been checking to debug.log no errors, running continuously with percentage complete increasing as expected (just slow). Config: prune=3000 dbcache=16384
Looking for any suggestions to try out. Seems like at this rate it would take a few weeks to load, that seems too slow, or it this how long it seems to take nowadays?
|
|
|
|
DaveF
Legendary
Offline
Activity: 3640
Merit: 6645
Crypto Swap Exchange
|
|
June 07, 2024, 02:04:30 PM |
|
1) I5 is a bit vague. The original Intel Core i5 was released almost 15 years ago. What gen is yours? 2) The 500 GB drive is not large enough you need at least 1 TB now. https://www.blockchain.com/explorer/charts/blocks-size 3) How fast is your connection? Downloading 540GB on a 1M connection is still going to take a while no matter what else you have. -Dave
|
|
|
|
BitMaxz
Legendary
Offline
Activity: 3416
Merit: 3161
Is the $100k BTC possible?
|
Memory 8gb
Config: dbcache=16384
It seems that you set the dbcache more than your current RAM memory size since you have 8GB of memory you shouldn't set it to more than 4096MB. I suggest try to change it to 4096 but before you do that make sure to stop the syncing process first if you are using a GUI just exit the GUI, and it will shut down automatically if not you can use "bitcoin-cli stop" to avoid corruption reboot your PC and rerun everything then monitor it again if the syncing process still slow or not.
|
█████████████████████████ ████████▀▀████▀▀█▀▀██████ █████▀████▄▄▄▄██████▀████ ███▀███▄████████▄████▀███ ██▀███████████████████▀██ █████████████████████████ █████████████████████████ █████████████████████████ ██▄███████████████▀▀▄▄███ ███▄███▀████████▀███▄████ █████▄████▀▀▀▀████▄██████ ████████▄▄████▄▄█████████ █████████████████████████ | BitList | | █▀▀▀▀ █ █ █ █ █ █ █ █ █ █ █ █▄▄▄▄ | ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ . REAL-TIME DATA TRACKING CURATED BY THE COMMUNITY . ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄ | ▀▀▀▀█ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▄█ | | List #kycfree Websites |
|
|
|
Cricktor
Legendary
Offline
Activity: 924
Merit: 1445
Crypto Swap Exchange
|
OP has prune=3000 in his bitcoin.conf, limiting his blockchain data to approx. 3GB. The directory of chainstate will still occupy ~12GB. Depending on how full the 500GB drive is with other stuff, it should still suffice. You're correct when you speak of an unpruned, full archival, node which needs 611GB for the blockchain data alone. Here's my current storage usage: $ du -hd1 .bitcoin/ 611G .bitcoin/blocks 12G .bitcoin/chainstate 59G .bitcoin/indexes 216M .bitcoin/wallets 682G .bitcoin/
As BitMaxz already pointed out, dbcache is way too large for only 8GB RAM. Use approx. half the RAM for dbcache. For an Initial Blockchain Download (IBD) you can also set the option blocksonly=1, so that your node is not bothered with transaction verification and relay and mempool stuff.
|
|
|
|
22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 08, 2024, 03:31:09 AM |
|
1) I5 is a bit vague. The original Intel Core i5 was released almost 15 years ago. What gen is yours? Intel® Core™ i5-3470T CPU @ 2.90GHz × 4 OK, good info, thanks 3) How fast is your connection? Downloading 540GB on a 1M connection is still going to take a while no matter what else you have. Just did a test, down = 197.94 Mbps, wired fiber
|
|
|
|
22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 08, 2024, 03:32:45 AM |
|
It seems that you set the dbcache more than your current RAM memory size since you have 8GB of memory you shouldn't set it to more than 4096MB.
I suggest try to change it to 4096 but before you do that make sure to stop the syncing process first if you are using a GUI just exit the GUI, and it will shut down automatically if not you can use "bitcoin-cli stop" to avoid corruption reboot your PC and rerun everything then monitor it again if the syncing process still slow or not. Thanks, great suggestion. I will do this. I am running the bitcoind as a systemd service and am able to gracefully shut it down and restart it.
|
|
|
|
22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 08, 2024, 03:33:46 AM |
|
OP has prune=3000 in his bitcoin.conf, limiting his blockchain data to approx. 3GB. The directory of chainstate will still occupy ~12GB. Depending on how full the 500GB drive is with other stuff, it should still suffice. OK, thanks, that tracks what I thought.
|
|
|
|
22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 08, 2024, 03:36:10 AM |
|
So just as an update, I am running the bitcoind as a systemd job. I also have a crontab job to track hourly the progress. The internet speed is at some times better than others. I get between .03% and .15% per hour of progress.
I will update the cache parameter setting and retry.
[I updated dbcache=4096 and will continue to monitor progress hourly, and update here after a while, thanks all, number one priority on my list to get this working:)]
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3472
Merit: 17588
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
June 08, 2024, 07:22:27 AM |
|
I updated dbcache=4096 and will continue to monitor progress hourly How is your memory load? My guess is you've reached the part where your SSD-speed (both reads and writes) is the limiting factor due to the low RAM. There's a huge difference between SSDs: some have very bad performance when it comes to combined reads and writes. If it's a budget SSD, that may be the cause. On a similar setup, it took me less than 2 days to complete the IBD. See this post on my progress. Considering the low price of RAM, if at all possible, add some That should make things run much smoother.
|
| | Peach BTC bitcoin | │ | Buy and Sell Bitcoin P2P | │ | . .
▄▄███████▄▄ ▄██████████████▄ ▄███████████████████▄ ▄█████████████████████▄ ▄███████████████████████▄ █████████████████████████ █████████████████████████ █████████████████████████ ▀███████████████████████▀ ▀█████████████████████▀ ▀███████████████████▀ ▀███████████████▀ ▀▀███████▀▀
▀▀▀▀███████▀▀▀▀ | | EUROPE | AFRICA LATIN AMERICA | | | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
███████▄█ ███████▀ ██▄▄▄▄▄░▄▄▄▄▄ █████████████▀ ▐███████████▌ ▐███████████▌ █████████████▄ ██████████████ ███▀███▀▀███▀ | . Download on the App Store | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
▄██▄ ██████▄ █████████▄ ████████████▄ ███████████████ ████████████▀ █████████▀ ██████▀ ▀██▀ | . GET IT ON Google Play | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ |
|
|
|
22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 08, 2024, 07:34:39 AM |
|
How is your memory load? My guess is you've reached the part where your SSD-speed (both reads and writes) is the limiting factor due to the low RAM. There's a huge difference between SSDs: some have very bad performance when it comes to combined reads and writes. If it's a budget SSD, that may be the cause. total used free shared buff/cache available Mem: 7909288 1471828 127476 235508 6309984 5935240 Swap: 2097148 1010240 1086908 I have not used #free before but the numbers don't seem to look to bad and are similar to your outputs. Still not explaining my slow times as far as I can tell. Also after updating the dbcache setting still ridiculously slow like .09% per hour. Like 10 times too slow.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3472
Merit: 17588
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
June 08, 2024, 07:47:21 AM |
|
Also after updating the dbcache setting still ridiculously slow like .09% per hour. Like 10 times too slow. Have a look at hard disk io/load. I forgot what I used exactly, but Google gives many options. Quick (but inconclusive) test: how long does this take?
|
| | Peach BTC bitcoin | │ | Buy and Sell Bitcoin P2P | │ | . .
▄▄███████▄▄ ▄██████████████▄ ▄███████████████████▄ ▄█████████████████████▄ ▄███████████████████████▄ █████████████████████████ █████████████████████████ █████████████████████████ ▀███████████████████████▀ ▀█████████████████████▀ ▀███████████████████▀ ▀███████████████▀ ▀▀███████▀▀
▀▀▀▀███████▀▀▀▀ | | EUROPE | AFRICA LATIN AMERICA | | | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
███████▄█ ███████▀ ██▄▄▄▄▄░▄▄▄▄▄ █████████████▀ ▐███████████▌ ▐███████████▌ █████████████▄ ██████████████ ███▀███▀▀███▀ | . Download on the App Store | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
▄██▄ ██████▄ █████████▄ ████████████▄ ███████████████ ████████████▀ █████████▀ ██████▀ ▀██▀ | . GET IT ON Google Play | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ |
|
|
|
nc50lc
Legendary
Offline
Activity: 2576
Merit: 6290
Self-proclaimed Genius
|
|
June 08, 2024, 07:49:26 AM |
|
Also after updating the dbcache setting still ridiculously slow like .09% per hour. Like 10 times too slow.
That new setting is fine. Setting the database cache to 16GB while having 8GB of RAM is bad for your SSD since your system will be relying too much on page file. Bitcoin Core use almost all of the allotted dbcache ( about 15GB with your previous setting) to save the UTXO set to RAM before writing it on disk. Since more than half is being moved from/to disk as page file, plus the data being dumped to the data directory, your SSD must had been on a lot of strain before that could make IBD slower. ( apparent on HDD, not much on SSD, not on separate physical drives for OS and bitcoin datadir) How about your peers? try: bitcoin-cli getpeerinfoIs it populated with nodes with bad connections? Peers keep disconnecting? ( 'conntime' just recent)
|
|
|
|
22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 08, 2024, 07:57:52 AM |
|
How about your peers? try: bitcoin-cli getpeerinfo shows a blank list. I see continual but slow progress, does this indicate a problem that the list is blank? core@core:~/.bitcoin$ bitcoin-cli getpeerinfo [ ] time sync core@core:~/.bitcoin$ time sync
real 0m0.416s user 0m0.002s sys 0m0.000s not sure what this is showing me. After thinking about it for a while my thinking at this point is I am getting somehow ISP throttled which fits the facts that initially I was seeing fairly normal progress. It is being done in such a way as to not affect other activities I am doing on a separate computer.
|
|
|
|
nc50lc
Legendary
Offline
Activity: 2576
Merit: 6290
Self-proclaimed Genius
|
|
June 08, 2024, 08:06:40 AM |
|
How about your peers? try: bitcoin-cli getpeerinfo shows a blank list. I see continual but slow progress, does this indicate a problem that the list is blank. That outputs your list of peers, zero means that you're currently not connected to any Bitcoin node. That must be the issue, low to zero peer means slow IBD since there's few/no node to download the blockchain from. You must check your system for anything that blocks any connection to Bitcoin Core. Start with your firewall.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3472
Merit: 17588
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
June 08, 2024, 08:11:12 AM |
|
time sync core@core:~/.bitcoin$ time sync
real 0m0.416s user 0m0.002s sys 0m0.000s not sure what this is showing me. I have a very slow HDD in my system. When doing the above, it can take minutes to sync. It it would be many seconds for your SSD, I would have called it bad. At 0.4s, I can't really tell. For an idle system it's a few ms. After thinking about it for a while my thinking at this point is I am getting somehow ISP throttled which fits the facts that initially I was seeing fairly normal progress. That's easy to test: rename your Bitcoin Core directory and restart from scratch. If it's fast again, it's not your ISP.
|
| | Peach BTC bitcoin | │ | Buy and Sell Bitcoin P2P | │ | . .
▄▄███████▄▄ ▄██████████████▄ ▄███████████████████▄ ▄█████████████████████▄ ▄███████████████████████▄ █████████████████████████ █████████████████████████ █████████████████████████ ▀███████████████████████▀ ▀█████████████████████▀ ▀███████████████████▀ ▀███████████████▀ ▀▀███████▀▀
▀▀▀▀███████▀▀▀▀ | | EUROPE | AFRICA LATIN AMERICA | | | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
███████▄█ ███████▀ ██▄▄▄▄▄░▄▄▄▄▄ █████████████▀ ▐███████████▌ ▐███████████▌ █████████████▄ ██████████████ ███▀███▀▀███▀ | . Download on the App Store | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
▄██▄ ██████▄ █████████▄ ████████████▄ ███████████████ ████████████▀ █████████▀ ██████▀ ▀██▀ | . GET IT ON Google Play | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ |
|
|
|
22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 08, 2024, 12:24:48 PM |
|
That's easy to test: rename your Bitcoin Core directory and restart from scratch. If it's fast again, it's not your ISP. I saved off the blocks, chainstate and debug.log and restarted. Will know soon, good idea To start with I see a bunch of peers in the getpeerinfo now....
|
|
|
|
22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 08, 2024, 01:24:50 PM |
|
I am confused. So now at the start of running fresh it is quick again.
First hour = 4.18% which is good.
Not sure what is happening.
getpeerinfo still showing several peers.
Any clues or hints on why it would run fast at the beginning and then slow down? I am not using a firewall.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3472
Merit: 17588
Thick-Skinned Gang Leader and Golden Feather 2021
|
|
June 08, 2024, 03:09:16 PM |
|
Any clues or hints on why it would run fast at the beginning and then slow down? Your chainstate directory gets bigger, and eventually doesn't fit your RAM anymore. That large increases the disk reads and writes. But still, about 10 times slower than mine shouldn't happen. What SSD do you have? I started at the same rate: Converted to progress per hour:
|
| | Peach BTC bitcoin | │ | Buy and Sell Bitcoin P2P | │ | . .
▄▄███████▄▄ ▄██████████████▄ ▄███████████████████▄ ▄█████████████████████▄ ▄███████████████████████▄ █████████████████████████ █████████████████████████ █████████████████████████ ▀███████████████████████▀ ▀█████████████████████▀ ▀███████████████████▀ ▀███████████████▀ ▀▀███████▀▀
▀▀▀▀███████▀▀▀▀ | | EUROPE | AFRICA LATIN AMERICA | | | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
███████▄█ ███████▀ ██▄▄▄▄▄░▄▄▄▄▄ █████████████▀ ▐███████████▌ ▐███████████▌ █████████████▄ ██████████████ ███▀███▀▀███▀ | . Download on the App Store | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
▄██▄ ██████▄ █████████▄ ████████████▄ ███████████████ ████████████▀ █████████▀ ██████▀ ▀██▀ | . GET IT ON Google Play | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ |
|
|
|
22bits (OP)
Member
Offline
Activity: 85
Merit: 25
|
|
June 08, 2024, 03:26:51 PM |
|
What SSD do you have? 500GB TOSHIBA MQ01ABF0 Interesting about the slow down under normal circumstances, makes sense. I will see on this go round how progress is in a while. I might get a 1000 mbps fiber dedicated separate line to troubleshoot further.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3472
Merit: 17588
Thick-Skinned Gang Leader and Golden Feather 2021
|
500GB TOSHIBA MQ01ABF0 That's not an SSD, it's a HDD. That explains everything! It needs to write several TB of data, read a lot of data, and have random access all the time. My SSD was already the limiting factor, it makes total sense your HDD is a factor 10 slower. I might get a 1000 mbps fiber dedicated separate line to troubleshoot further. Your internet connection isn't the bottleneck. There are only 2 things that can improve performance: 1. Get a SSD, preferrably a good brand (Samsung Evo for instance), not a budget disk. Check some reviews on Tom's Hardware to make sure you get a good one. But even a budget SSD will be much faster than your current HDD. 2. Add more RAM. If you do both, you're entire computing experience will get much smoother. Without those upgrades, you should probably just use Electrum instead of Bitcoin Core.
|
| | Peach BTC bitcoin | │ | Buy and Sell Bitcoin P2P | │ | . .
▄▄███████▄▄ ▄██████████████▄ ▄███████████████████▄ ▄█████████████████████▄ ▄███████████████████████▄ █████████████████████████ █████████████████████████ █████████████████████████ ▀███████████████████████▀ ▀█████████████████████▀ ▀███████████████████▀ ▀███████████████▀ ▀▀███████▀▀
▀▀▀▀███████▀▀▀▀ | | EUROPE | AFRICA LATIN AMERICA | | | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
███████▄█ ███████▀ ██▄▄▄▄▄░▄▄▄▄▄ █████████████▀ ▐███████████▌ ▐███████████▌ █████████████▄ ██████████████ ███▀███▀▀███▀ | . Download on the App Store | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ | ▄▀▀▀ █ █ █ █ █ █ █ █ █ █ █ ▀▄▄▄ |
▄██▄ ██████▄ █████████▄ ████████████▄ ███████████████ ████████████▀ █████████▀ ██████▀ ▀██▀ | . GET IT ON Google Play | ▀▀▀▄ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▀ |
|
|
|
|