Bitcoin Forum
November 19, 2024, 09:35:39 AM *
News: Check out the artwork 1Dq created to commemorate this forum's 15th anniversary
 
   Home   Help Search Login Register More  
Pages: [1] 2 »  All
  Print  
Author Topic: Great news! blockchain initial downloading are much faster now.  (Read 2353 times)
finway (OP)
Hero Member
*****
Offline Offline

Activity: 714
Merit: 500


View Profile
March 22, 2012, 05:50:50 AM
Last edit: March 22, 2012, 06:08:21 AM by finway
 #1

Thanks to sipa,
with less than 10 lines code changing,
the initial blockchain downloading are much faster than ever,
with good network condition,
you can download in less than 2 hours.

 Grin Grin Grin Grin Shocked Shocked Shocked Shocked Cheesy Cheesy Cheesy Cheesy

sipa's patch :
https://github.com/bitcoin/bitcoin/pull/964

Test builds from luke-jr :
https://bitcointalk.org/index.php?topic=70580.msg812714#msg812714


Ente
Legendary
*
Offline Offline

Activity: 2126
Merit: 1001



View Profile
March 22, 2012, 02:26:28 PM
 #2

Oh yeah, great! :-)

It really was a pain to wait one, two days for the whole chain..

Out of historical curiosity: What made it took so long? Downloading the blocks from other peers, or verifying the blocks when they got there?
Also, at the beginning of the downloading they came in by the thousands, at the end you can watch the blocks come in one after another per second?

Ente
rjk
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250


1ngldh


View Profile
March 22, 2012, 02:41:21 PM
 #3

Oh yeah, great! :-)

It really was a pain to wait one, two days for the whole chain..

Out of historical curiosity: What made it took so long? Downloading the blocks from other peers, or verifying the blocks when they got there?
Also, at the beginning of the downloading they came in by the thousands, at the end you can watch the blocks come in one after another per second?

Ente
Verification. Later blocks were larger and had more transactions and spam, causing them to be slower. Running the initial download in a ramdisk has always been speedy, because the disk throughput was the bottleneck. Sounds like that has either been made more efficient or removed.

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
Pieter Wuille
Legendary
*
Offline Offline

Activity: 1072
Merit: 1181


View Profile WWW
March 22, 2012, 03:02:37 PM
 #4

And today it was merged into mainline, so it will be part of 0.6.0 final.

I do Bitcoin stuff.
Ente
Legendary
*
Offline Offline

Activity: 2126
Merit: 1001



View Profile
March 22, 2012, 03:05:27 PM
 #5

Verification. Later blocks were larger and had more transactions and spam, causing them to be slower. Running the initial download in a ramdisk has always been speedy, because the disk throughput was the bottleneck. Sounds like that has either been made more efficient or removed.

Verification -> CPU is the bottleneck
Download -> HDD is the bottleneck

So, which one of the two then? :-P

Defragmenting the local (large) blockchain doesnt help, in my tests.
I cant believe its a HDD throughput problem.. Sure, many thousand blocks, but its just a few gigs data?

So I guess "verification/CPU" is the problem. My CPU was at 100% while loading the chain, too..
But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

Ente
rjk
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250


1ngldh


View Profile
March 22, 2012, 03:18:43 PM
 #6

Verification. Later blocks were larger and had more transactions and spam, causing them to be slower. Running the initial download in a ramdisk has always been speedy, because the disk throughput was the bottleneck. Sounds like that has either been made more efficient or removed.

Verification -> CPU is the bottleneck
Download -> HDD is the bottleneck

So, which one of the two then? :-P

Defragmenting the local (large) blockchain doesnt help, in my tests.
I cant believe its a HDD throughput problem.. Sure, many thousand blocks, but its just a few gigs data?

So I guess "verification/CPU" is the problem. My CPU was at 100% while loading the chain, too..
But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

Ente
No, it was disk bound because of huge numbers of tiny read and write operations. For those without a ramdisk, even an SSD improved the speed dramatically. Disk throughput could not be a factor in download, since a normal internet connection (less than 50mbps) would not be able to saturate a hard drive's throughput speed.

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
Pieter Wuille
Legendary
*
Offline Offline

Activity: 1072
Merit: 1181


View Profile WWW
March 22, 2012, 03:20:08 PM
 #7

No:

verification: CPU is the bottleneck
indexing: HDD latency is the bottleneck
download: network/p2p is the bottleneck

Indexing almost always outweighed the other two, unless you were running on a ramdrive.

I do Bitcoin stuff.
rjk
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250


1ngldh


View Profile
March 22, 2012, 03:21:19 PM
 #8

No:

verification: CPU is the bottleneck
indexing: HDD latency is the bottleneck
download: network/p2p is the bottleneck

Indexing almost always outweighed the other two, unless you were running on a ramdrive.

Yes, sorry I confused verification with indexing.

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1079


Gerald Davis


View Profile
March 22, 2012, 03:23:49 PM
 #9

But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

It was fast.  The block chain took over 1000 days to create.  If you downloaded and verified it in a day it was ~1000x faster to verify than create.  With things like ramdisk (or improved client which keep portion of chain in ram) and the improvements the developers made to 0.6 it likely is 10,000x faster under optimal conditions.
FreeMoney
Legendary
*
Offline Offline

Activity: 1246
Merit: 1016


Strength in numbers


View Profile WWW
March 22, 2012, 04:22:10 PM
 #10

But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

It was fast.  The block chain took over 1000 days to create.  If you downloaded and verified it in a day it was ~1000x faster to verify than create.  With things like ramdisk (or improved client which keep portion of chain in ram) and the improvements the developers made to 0.6 it likely is 10,000x faster under optimal conditions.

No doubt. And it took 1000 days for a very substantial amount of resources. And it could be done in 2 days (now hours?!) by one weak machine.

Play Bitcoin Poker at sealswithclubs.eu. We're active and open to everyone.
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1079


Gerald Davis


View Profile
March 22, 2012, 04:25:05 PM
 #11

Of course (can't believe I missed that).  So yeah it is more like a million fold (maybe billion fold) easier to verify than produce.
torusJKL
Hero Member
*****
Offline Offline

Activity: 619
Merit: 500


View Profile
March 22, 2012, 04:32:58 PM
 #12

That's great news.
Thanks sipa.

If you find my post useful send some Bitcoin: 167XM1Za8aG9CdbYuHFMpL2kvPsw6uC8da
Bitrated || bitcoin-otc || Moon Bitcoin Faucet
Pieter Wuille
Legendary
*
Offline Offline

Activity: 1072
Merit: 1181


View Profile WWW
March 22, 2012, 04:51:03 PM
 #13

Of course (can't believe I missed that).  So yeah it is more like a million fold (maybe billion fold) easier to verify than produce.

That factor is determined exactly by difficulty*4295032833.

That is how much harder it is to find a block nonce at a given difficulty vs. verifying it.


I do Bitcoin stuff.
Pieter Wuille
Legendary
*
Offline Offline

Activity: 1072
Merit: 1181


View Profile WWW
March 22, 2012, 04:54:35 PM
 #14

That's great news.
Thanks sipa.

I just did some testing for good parameters. Tweaking them was Gavin's idea.

I do Bitcoin stuff.
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1079


Gerald Davis


View Profile
March 22, 2012, 04:56:16 PM
 #15

That factor is determined exactly by difficulty*4295032833.

That is how much harder it is to find a block nonce at a given difficulty vs. verifying it.

Thanks for quantifying it.
cnbtcnews
Newbie
*
Offline Offline

Activity: 54
Merit: 0



View Profile
March 23, 2012, 01:19:31 AM
 #16

顶!
pastory99
Newbie
*
Offline Offline

Activity: 49
Merit: 0


View Profile
March 23, 2012, 05:29:37 AM
 #17

nice, thanks Cheesy
MaxSan
Sr. Member
****
Offline Offline

Activity: 369
Merit: 250


View Profile
March 23, 2012, 07:39:30 AM
 #18

excellent. looking forward to the next release guys Smiley
bitclown
Full Member
***
Offline Offline

Activity: 185
Merit: 100


View Profile
March 23, 2012, 07:48:41 AM
 #19

This is the kind of headlines that will drive people to help test the pre-releases. Everybody loves massive performance improvements. Fingersnappin' good work!
Ente
Legendary
*
Offline Offline

Activity: 2126
Merit: 1001



View Profile
March 23, 2012, 08:39:38 AM
 #20

Thank you for the clarifications and thoughts, guys! Nice!

Ente
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!