Bitcoin Forum
May 07, 2024, 11:29:40 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: What is the bottleneck in synchronizing?  (Read 270 times)
DrakonX (OP)
Jr. Member
*
Offline Offline

Activity: 99
Merit: 4


View Profile
June 20, 2018, 10:20:46 AM
 #1

What is the bottleneck in synchronizing the blockchain? Computational Power (CPU), Internet bandwidth or hard disk speed?
"This isn't the kind of software where we can leave so many unresolved bugs that we need a tracker for them." -- Satoshi
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1715124580
Hero Member
*
Offline Offline

Posts: 1715124580

View Profile Personal Message (Offline)

Ignore
1715124580
Reply with quote  #2

1715124580
Report to moderator
1715124580
Hero Member
*
Offline Offline

Posts: 1715124580

View Profile Personal Message (Offline)

Ignore
1715124580
Reply with quote  #2

1715124580
Report to moderator
1715124580
Hero Member
*
Offline Offline

Posts: 1715124580

View Profile Personal Message (Offline)

Ignore
1715124580
Reply with quote  #2

1715124580
Report to moderator
Sophokles
Hero Member
*****
Offline Offline

Activity: 1218
Merit: 516


Eloncoin.org - Mars, here we come!


View Profile WWW
June 20, 2018, 10:30:01 AM
 #2

Depends on your hardware and internet connection. On a PC with a DSL (or slower) connection, it is internet bandwidth. For a Raspi Pi, it is CPU power. HW write speed will never be an issue.









▄▄████████▄▄
▄▄████████████████▄▄
▄██
████████████████████▄
▄███
██████████████████████▄
▄████
███████████████████████▄
███████████████████████▄
█████████████████▄███████
████████████████▄███████▀
██████████▄▄███▄██████▀
████████▄████▄█████▀▀
██████▄██████████▀
███▄▄█████
███████▄
██▄██████████████
░▄██████████████▀
▄█████████████▀
████████████
███████████▀
███████▀▀
.
▄▄███████▄▄
▄███████████████▄
▄███████████████████▄
▄██████████
███████████
▄███████████████████████▄
█████████████████████████
█████████████████████████
█████████████████████████
▀█
██████████████████████▀
▀██
███████████████████▀
▀███████████████████▀
▀█████████
██████▀
▀▀███████▀▀
.
 ElonCoin.org 
.
████████▄▄███████▄▄
███████▄████████████▌
██████▐██▀███████▀▀██
███████████████████▐█▌
████▄▄▄▄▄▄▄▄▄▄██▄▄▄▄▄
███▐███▀▄█▄█▀▀█▄█▄▀
███████████████████
█████████████▄████
█████████▀░▄▄▄▄▄
███████▄█▄░▀█▄▄░▀
███▄██▄▀███▄█████▄▀
▄██████▄▀███████▀
████████▄▀████▀
█████▄▄
.
"I could either watch it
happen or be a part of it"
▬▬▬▬▬
bob123
Legendary
*
Offline Offline

Activity: 1624
Merit: 2481



View Profile WWW
June 20, 2018, 11:15:48 AM
 #3

Internet bandwith is not a problem nowadays (assuming you live in a country with somewhat developed technology).
The biggets bottleneck are the CPU processing and the disk usage.

If you are running a somewhat modern computer (e.g. quad core), the device which slows down the synchronisation process is the hard drive.
SSD's decrease the time it takes to sync the blockchain noticeable, compared to a HDD.

Carlton Banks
Legendary
*
Offline Offline

Activity: 3430
Merit: 3074



View Profile
June 20, 2018, 11:57:21 AM
Merited by fmirg21 (1)
 #4

Bitcoin 0.17.0 will speed up a little if you have fast (>10 Mbit/s) internet, fast/modern CPU & SSD disk

  • Changes to the way leveldb handles blockchain I/O will speed things up 5-10%
  • Validation will improve <5% for a range of CPUs with SSE4 and AVX2 instructions (maybe including ARM)

Most of the big gains have already been leveraged, but the developers often bounce new ideas around on the bitcoin dev mailing list.

Vires in numeris
DrakonX (OP)
Jr. Member
*
Offline Offline

Activity: 99
Merit: 4


View Profile
June 21, 2018, 10:16:24 AM
Merited by ABCbits (1)
 #5

Everyone writes something different here. This may also be due to the question. Let me rephrase the question. Related to the blockchain, which CPU speeds, internet speeds, and hard drive speeds match? How can I investigate the relationships?
Carlton Banks
Legendary
*
Offline Offline

Activity: 3430
Merit: 3074



View Profile
June 21, 2018, 11:07:26 AM
Merited by Welsh (2), ABCbits (1)
 #6

Everyone writes something different here. This may also be due to the question. Let me rephrase the question. Related to the blockchain, which CPU speeds, internet speeds, and hard drive speeds match? How can I investigate the relationships?

  • 1st bottleneck is internet connection speed
  • 2nd bottleneck is CPU speed
  • 3rd bottleneck is hard disk speed


Improve your setup in that order, or all 3 simultaneously. Improvements to Bitcoin Core can also increase blockchain sync performance with any hardware. RAM is not so much of a bottleneck; even basic computers have 1GB+ RAM these days which is good enough (if you look at Settings -> Options menu in Bitcoin Core, you can improve the syncing performance a little if you increase database cache size to 1GB from the default value)

That's the best summary I can provide you.

Vires in numeris
pebwindkraft
Sr. Member
****
Offline Offline

Activity: 257
Merit: 343


View Profile
June 21, 2018, 11:17:21 AM
Merited by ABCbits (2)
 #7

Everyone writes something different here. This may also be due to the question. Let me rephrase the question. Related to the blockchain, which CPU speeds, internet speeds, and hard drive speeds match? How can I investigate the relationships?

Summary: there is no generic rule - one would need to understand the way, how the initial load of the blockchain works :-)
There are too many variables. One would have to setup a weighted matrix with all dependencies... hence you find only general info, like this:

With my bitcoin core node (a fully validating node), it fetches all transactions from it peers, and verifies the transactions for the correctness (see here: https://en.bitcoin.it/wiki/Protocol_rules#.22tx.22_messages).
This involves many, many lookups of previous transactions in the local blockchain, hence the dependency to the hard disk. I have had situations, where an SSD provided a times five speedup.
The calculations with a Quad-Core CPU is usually not a limiting factor, I had it running on a 4 years old Macintosh, and it never got limited. CPU calculations was always faster than data delivered from other peers.
On the bandwidth: I had a 10mbit/s connection, and did never run into congestion. Au contraire: I didn't have "enough" peers to send data continuously.
Then I did the opposite: I took a RasPI2+, attached it to a 2mbit line, and let it run 3 weeks, before he was nearly sync'd with the blockchain. And then it came into the time of full blocks (near the end 2017), and suddenly the RasPi would never completly sync. The CPU was probably too slow to verify the tx data, and the attached external disk (no SSD!) via USB could not cope with the amount of requests. I set up a monitor, and measured CPU, processes (load averages), disk I/O, network I/O... I could see that the disk I/O was not going down to zero a single time, and the load average never went under 1.0, it merely stayed between 3 and 4. So there were many processes in the working queue, that could not be worked on by the CPU, cause other tasks occupied it.

Summary: you have to have a setup, and measure all details.
I believe that core team does a similiar thing, and I recall they had a process setup, which (re-) loads the blockchain on a daily basis (to ensure correctness of the system). But I haven't seen any statistics.
DrakonX (OP)
Jr. Member
*
Offline Offline

Activity: 99
Merit: 4


View Profile
June 28, 2018, 04:10:01 PM
 #8

  • 1st bottleneck is internet connection speed
  • 2nd bottleneck is CPU speed
  • 3rd bottleneck is hard disk speed
This was really helpful. Thank you. Smiley
pebwindkraft, your post also gave me a clue how the process works. Thank you too.
bachel
Sr. Member
****
Offline Offline

Activity: 261
Merit: 250


View Profile
June 30, 2018, 11:38:05 PM
 #9

It's actually ram and HDD

Made 2 test
8 core 3.2ghz 8gb ddr3 ram normal HDD sync time 4days

Seconde running as proxmox container 4 cores 2.2ghz 72gb DDR 3 ram and ssd sync time 14 hours

Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!