Bitcoin Forum
December 11, 2016, 12:43:22 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: Great news! blockchain initial downloading are much faster now.  (Read 2097 times)
finway
Hero Member
*****
Offline Offline

Activity: 714


View Profile
March 22, 2012, 05:50:50 AM
 #1

Thanks to sipa,
with less than 10 lines code changing,
the initial blockchain downloading are much faster than ever,
with good network condition,
you can download in less than 2 hours.

 Grin Grin Grin Grin Shocked Shocked Shocked Shocked Cheesy Cheesy Cheesy Cheesy

sipa's patch :
https://github.com/bitcoin/bitcoin/pull/964

Test builds from luke-jr :
https://bitcointalk.org/index.php?topic=70580.msg812714#msg812714


1481417002
Hero Member
*
Offline Offline

Posts: 1481417002

View Profile Personal Message (Offline)

Ignore
1481417002
Reply with quote  #2

1481417002
Report to moderator
1481417002
Hero Member
*
Offline Offline

Posts: 1481417002

View Profile Personal Message (Offline)

Ignore
1481417002
Reply with quote  #2

1481417002
Report to moderator
Goat says: "Bitcoin is NOT Illegal in Thailand. There is no law against Bitcoin in Thailand!"
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
Ente
Legendary
*
Offline Offline

Activity: 1834



View Profile
March 22, 2012, 02:26:28 PM
 #2

Oh yeah, great! :-)

It really was a pain to wait one, two days for the whole chain..

Out of historical curiosity: What made it took so long? Downloading the blocks from other peers, or verifying the blocks when they got there?
Also, at the beginning of the downloading they came in by the thousands, at the end you can watch the blocks come in one after another per second?

Ente
rjk
Sr. Member
****
Offline Offline

Activity: 420


1ngldh


View Profile
March 22, 2012, 02:41:21 PM
 #3

Oh yeah, great! :-)

It really was a pain to wait one, two days for the whole chain..

Out of historical curiosity: What made it took so long? Downloading the blocks from other peers, or verifying the blocks when they got there?
Also, at the beginning of the downloading they came in by the thousands, at the end you can watch the blocks come in one after another per second?

Ente
Verification. Later blocks were larger and had more transactions and spam, causing them to be slower. Running the initial download in a ramdisk has always been speedy, because the disk throughput was the bottleneck. Sounds like that has either been made more efficient or removed.

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
Pieter Wuille
Legendary
*
Offline Offline

Activity: 1036


View Profile WWW
March 22, 2012, 03:02:37 PM
 #4

And today it was merged into mainline, so it will be part of 0.6.0 final.

aka sipa, core dev team

Tips and donations: 1KwDYMJMS4xq3ZEWYfdBRwYG2fHwhZsipa
Ente
Legendary
*
Offline Offline

Activity: 1834



View Profile
March 22, 2012, 03:05:27 PM
 #5

Verification. Later blocks were larger and had more transactions and spam, causing them to be slower. Running the initial download in a ramdisk has always been speedy, because the disk throughput was the bottleneck. Sounds like that has either been made more efficient or removed.

Verification -> CPU is the bottleneck
Download -> HDD is the bottleneck

So, which one of the two then? :-P

Defragmenting the local (large) blockchain doesnt help, in my tests.
I cant believe its a HDD throughput problem.. Sure, many thousand blocks, but its just a few gigs data?

So I guess "verification/CPU" is the problem. My CPU was at 100% while loading the chain, too..
But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

Ente
rjk
Sr. Member
****
Offline Offline

Activity: 420


1ngldh


View Profile
March 22, 2012, 03:18:43 PM
 #6

Verification. Later blocks were larger and had more transactions and spam, causing them to be slower. Running the initial download in a ramdisk has always been speedy, because the disk throughput was the bottleneck. Sounds like that has either been made more efficient or removed.

Verification -> CPU is the bottleneck
Download -> HDD is the bottleneck

So, which one of the two then? :-P

Defragmenting the local (large) blockchain doesnt help, in my tests.
I cant believe its a HDD throughput problem.. Sure, many thousand blocks, but its just a few gigs data?

So I guess "verification/CPU" is the problem. My CPU was at 100% while loading the chain, too..
But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

Ente
No, it was disk bound because of huge numbers of tiny read and write operations. For those without a ramdisk, even an SSD improved the speed dramatically. Disk throughput could not be a factor in download, since a normal internet connection (less than 50mbps) would not be able to saturate a hard drive's throughput speed.

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
Pieter Wuille
Legendary
*
Offline Offline

Activity: 1036


View Profile WWW
March 22, 2012, 03:20:08 PM
 #7

No:

verification: CPU is the bottleneck
indexing: HDD latency is the bottleneck
download: network/p2p is the bottleneck

Indexing almost always outweighed the other two, unless you were running on a ramdrive.

aka sipa, core dev team

Tips and donations: 1KwDYMJMS4xq3ZEWYfdBRwYG2fHwhZsipa
rjk
Sr. Member
****
Offline Offline

Activity: 420


1ngldh


View Profile
March 22, 2012, 03:21:19 PM
 #8

No:

verification: CPU is the bottleneck
indexing: HDD latency is the bottleneck
download: network/p2p is the bottleneck

Indexing almost always outweighed the other two, unless you were running on a ramdrive.

Yes, sorry I confused verification with indexing.

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
March 22, 2012, 03:23:49 PM
 #9

But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

It was fast.  The block chain took over 1000 days to create.  If you downloaded and verified it in a day it was ~1000x faster to verify than create.  With things like ramdisk (or improved client which keep portion of chain in ram) and the improvements the developers made to 0.6 it likely is 10,000x faster under optimal conditions.
FreeMoney
Legendary
*
Offline Offline

Activity: 1246


Strength in numbers


View Profile WWW
March 22, 2012, 04:22:10 PM
 #10

But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

It was fast.  The block chain took over 1000 days to create.  If you downloaded and verified it in a day it was ~1000x faster to verify than create.  With things like ramdisk (or improved client which keep portion of chain in ram) and the improvements the developers made to 0.6 it likely is 10,000x faster under optimal conditions.

No doubt. And it took 1000 days for a very substantial amount of resources. And it could be done in 2 days (now hours?!) by one weak machine.

Play Bitcoin Poker at sealswithclubs.eu. We're active and open to everyone.
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
March 22, 2012, 04:25:05 PM
 #11

Of course (can't believe I missed that).  So yeah it is more like a million fold (maybe billion fold) easier to verify than produce.
torusJKL
Hero Member
*****
Offline Offline

Activity: 595


View Profile
March 22, 2012, 04:32:58 PM
 #12

That's great news.
Thanks sipa.

If you find my post useful send some BTC: 167XM1Za8aG9CdbYuHFMpL2kvPsw6uC8da
Bitrated || bitcoin-otc || Moon Bitcoin Faucet
Pieter Wuille
Legendary
*
Offline Offline

Activity: 1036


View Profile WWW
March 22, 2012, 04:51:03 PM
 #13

Of course (can't believe I missed that).  So yeah it is more like a million fold (maybe billion fold) easier to verify than produce.

That factor is determined exactly by difficulty*4295032833.

That is how much harder it is to find a block nonce at a given difficulty vs. verifying it.


aka sipa, core dev team

Tips and donations: 1KwDYMJMS4xq3ZEWYfdBRwYG2fHwhZsipa
Pieter Wuille
Legendary
*
Offline Offline

Activity: 1036


View Profile WWW
March 22, 2012, 04:54:35 PM
 #14

That's great news.
Thanks sipa.

I just did some testing for good parameters. Tweaking them was Gavin's idea.

aka sipa, core dev team

Tips and donations: 1KwDYMJMS4xq3ZEWYfdBRwYG2fHwhZsipa
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
March 22, 2012, 04:56:16 PM
 #15

That factor is determined exactly by difficulty*4295032833.

That is how much harder it is to find a block nonce at a given difficulty vs. verifying it.

Thanks for quantifying it.
cnbtcnews
Jr. Member
*
Offline Offline

Activity: 54



View Profile
March 23, 2012, 01:19:31 AM
 #16

顶!
pastory99
Jr. Member
*
Offline Offline

Activity: 36


View Profile
March 23, 2012, 05:29:37 AM
 #17

nice, thanks Cheesy

MaxSan
Sr. Member
****
Offline Offline

Activity: 368


View Profile
March 23, 2012, 07:39:30 AM
 #18

excellent. looking forward to the next release guys Smiley
bitclown
Full Member
***
Offline Offline

Activity: 186


View Profile
March 23, 2012, 07:48:41 AM
 #19

This is the kind of headlines that will drive people to help test the pre-releases. Everybody loves massive performance improvements. Fingersnappin' good work!
Ente
Legendary
*
Offline Offline

Activity: 1834



View Profile
March 23, 2012, 08:39:38 AM
 #20

Thank you for the clarifications and thoughts, guys! Nice!

Ente
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!