Bitcoin Forum
February 23, 2020, 08:01:04 PM *
News: Latest Bitcoin Core release: 0.19.0.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: What is the bottleneck in synchronizing?  (Read 176 times)
DrakonX
Newbie
*
Offline Offline

Activity: 92
Merit: 0


View Profile
June 20, 2018, 10:20:46 AM
 #1

What is the bottleneck in synchronizing the blockchain? Computational Power (CPU), Internet bandwidth or hard disk speed?
100% First Deposit Bonus Instant Withdrawals Best Odds 10+ Sports Since 2014 No KYC Asked Play Now
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1582488064
Hero Member
*
Offline Offline

Posts: 1582488064

View Profile Personal Message (Offline)

Ignore
1582488064
Reply with quote  #2

1582488064
Report to moderator
1582488064
Hero Member
*
Offline Offline

Posts: 1582488064

View Profile Personal Message (Offline)

Ignore
1582488064
Reply with quote  #2

1582488064
Report to moderator
Sophokles
Hero Member
*****
Offline Offline

Activity: 543
Merit: 500


die-hard GPU miner


View Profile
June 20, 2018, 10:30:01 AM
 #2

Depends on your hardware and internet connection. On a PC with a DSL (or slower) connection, it is internet bandwidth. For a Raspi Pi, it is CPU power. HW write speed will never be an issue.
ETFbitcoin
Legendary
*
Offline Offline

Activity: 1918
Merit: 2208

Use SegWit and enjoy lower fees.


View Profile WWW
June 20, 2018, 11:11:06 AM
 #3

Depends on your hardware and internet connection. On a PC with a DSL (or slower) connection, it is internet bandwidth. For a Raspi Pi, it is CPU power. HW write speed will never be an issue.

HDD write speed will be issue if we increase block weight limit a lot, just like what happen with Ethereum. FYI, this is one reasons 2nd layer is preferred rather increase block size/weight limit as scaling solution.
Besides, internet latency/ping would make block propagation on network slower. But usually this only problematic on blockchain/cryptocurrency with fast block time.

Connected to nodes with slow connection also possible, but usually full nodes have good connection.

bob123
Legendary
*
Offline Offline

Activity: 1148
Merit: 1655



View Profile WWW
June 20, 2018, 11:15:48 AM
 #4

Internet bandwith is not a problem nowadays (assuming you live in a country with somewhat developed technology).
The biggets bottleneck are the CPU processing and the disk usage.

If you are running a somewhat modern computer (e.g. quad core), the device which slows down the synchronisation process is the hard drive.
SSD's decrease the time it takes to sync the blockchain noticeable, compared to a HDD.

Carlton Banks
Legendary
*
Offline Offline

Activity: 2646
Merit: 2206



View Profile
June 20, 2018, 11:57:21 AM
Merited by fmirg21 (1)
 #5

Bitcoin 0.17.0 will speed up a little if you have fast (>10 Mbit/s) internet, fast/modern CPU & SSD disk

  • Changes to the way leveldb handles blockchain I/O will speed things up 5-10%
  • Validation will improve <5% for a range of CPUs with SSE4 and AVX2 instructions (maybe including ARM)

Most of the big gains have already been leveraged, but the developers often bounce new ideas around on the bitcoin dev mailing list.

Vires in numeris
DrakonX
Newbie
*
Offline Offline

Activity: 92
Merit: 0


View Profile
June 21, 2018, 10:16:24 AM
 #6

Everyone writes something different here. This may also be due to the question. Let me rephrase the question. Related to the blockchain, which CPU speeds, internet speeds, and hard drive speeds match? How can I investigate the relationships?
Carlton Banks
Legendary
*
Offline Offline

Activity: 2646
Merit: 2206



View Profile
June 21, 2018, 11:07:26 AM
Merited by Welsh (2)
 #7

Everyone writes something different here. This may also be due to the question. Let me rephrase the question. Related to the blockchain, which CPU speeds, internet speeds, and hard drive speeds match? How can I investigate the relationships?

  • 1st bottleneck is internet connection speed
  • 2nd bottleneck is CPU speed
  • 3rd bottleneck is hard disk speed


Improve your setup in that order, or all 3 simultaneously. Improvements to Bitcoin Core can also increase blockchain sync performance with any hardware. RAM is not so much of a bottleneck; even basic computers have 1GB+ RAM these days which is good enough (if you look at Settings -> Options menu in Bitcoin Core, you can improve the syncing performance a little if you increase database cache size to 1GB from the default value)

That's the best summary I can provide you.

Vires in numeris
pebwindkraft
Sr. Member
****
Offline Offline

Activity: 259
Merit: 256


View Profile
June 21, 2018, 11:17:21 AM
 #8

Everyone writes something different here. This may also be due to the question. Let me rephrase the question. Related to the blockchain, which CPU speeds, internet speeds, and hard drive speeds match? How can I investigate the relationships?

Summary: there is no generic rule - one would need to understand the way, how the initial load of the blockchain works :-)
There are too many variables. One would have to setup a weighted matrix with all dependencies... hence you find only general info, like this:

With my bitcoin core node (a fully validating node), it fetches all transactions from it peers, and verifies the transactions for the correctness (see here: https://en.bitcoin.it/wiki/Protocol_rules#.22tx.22_messages).
This involves many, many lookups of previous transactions in the local blockchain, hence the dependency to the hard disk. I have had situations, where an SSD provided a times five speedup.
The calculations with a Quad-Core CPU is usually not a limiting factor, I had it running on a 4 years old Macintosh, and it never got limited. CPU calculations was always faster than data delivered from other peers.
On the bandwidth: I had a 10mbit/s connection, and did never run into congestion. Au contraire: I didn't have "enough" peers to send data continuously.
Then I did the opposite: I took a RasPI2+, attached it to a 2mbit line, and let it run 3 weeks, before he was nearly sync'd with the blockchain. And then it came into the time of full blocks (near the end 2017), and suddenly the RasPi would never completly sync. The CPU was probably too slow to verify the tx data, and the attached external disk (no SSD!) via USB could not cope with the amount of requests. I set up a monitor, and measured CPU, processes (load averages), disk I/O, network I/O... I could see that the disk I/O was not going down to zero a single time, and the load average never went under 1.0, it merely stayed between 3 and 4. So there were many processes in the working queue, that could not be worked on by the CPU, cause other tasks occupied it.

Summary: you have to have a setup, and measure all details.
I believe that core team does a similiar thing, and I recall they had a process setup, which (re-) loads the blockchain on a daily basis (to ensure correctness of the system). But I haven't seen any statistics.
DrakonX
Newbie
*
Offline Offline

Activity: 92
Merit: 0


View Profile
June 28, 2018, 04:10:01 PM
 #9

  • 1st bottleneck is internet connection speed
  • 2nd bottleneck is CPU speed
  • 3rd bottleneck is hard disk speed
This was really helpful. Thank you. Smiley
pebwindkraft, your post also gave me a clue how the process works. Thank you too.
bachel
Sr. Member
****
Offline Offline

Activity: 261
Merit: 250


View Profile
June 30, 2018, 11:38:05 PM
 #10

It's actually ram and HDD

Made 2 test
8 core 3.2ghz 8gb ddr3 ram normal HDD sync time 4days

Seconde running as proxmox container 4 cores 2.2ghz 72gb DDR 3 ram and ssd sync time 14 hours

Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!