Great news! blockchain initial downloading are much faster now.

[finway]

Thanks to sipa,
with less than 10 lines code changing,
the initial blockchain downloading are much faster than ever,
with good network condition,
you can download in less than 2 hours.

 

sipa's patch :
https://github.com/bitcoin/bitcoin/pull/964

Test builds from luke-jr :
https://bitcointalk.org/index.php?topic=70580.msg812714#msg812714

[Ente]

Oh yeah, great! :-)

It really was a pain to wait one, two days for the whole chain..

Out of historical curiosity: What made it took so long? Downloading the blocks from other peers, or verifying the blocks when they got there?
Also, at the beginning of the downloading they came in by the thousands, at the end you can watch the blocks come in one after another per second?

Ente

[rjk]

Oh yeah, great! :-)

It really was a pain to wait one, two days for the whole chain..

Out of historical curiosity: What made it took so long? Downloading the blocks from other peers, or verifying the blocks when they got there?
Also, at the beginning of the downloading they came in by the thousands, at the end you can watch the blocks come in one after another per second?

Ente

Verification. Later blocks were larger and had more transactions and spam, causing them to be slower. Running the initial download in a ramdisk has always been speedy, because the disk throughput was the bottleneck. Sounds like that has either been made more efficient or removed.

[Pieter Wuille]

And today it was merged into mainline, so it will be part of 0.6.0 final.

[Ente]

Verification. Later blocks were larger and had more transactions and spam, causing them to be slower. Running the initial download in a ramdisk has always been speedy, because the disk throughput was the bottleneck. Sounds like that has either been made more efficient or removed.

Verification -> CPU is the bottleneck
Download -> HDD is the bottleneck

So, which one of the two then? :-P

Defragmenting the local (large) blockchain doesnt help, in my tests.
I cant believe its a HDD throughput problem.. Sure, many thousand blocks, but its just a few gigs data?

So I guess "verification/CPU" is the problem. My CPU was at 100% while loading the chain, too..
But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

Ente

[rjk]

Verification. Later blocks were larger and had more transactions and spam, causing them to be slower. Running the initial download in a ramdisk has always been speedy, because the disk throughput was the bottleneck. Sounds like that has either been made more efficient or removed.

Verification -> CPU is the bottleneck
Download -> HDD is the bottleneck

So, which one of the two then? :-P

Defragmenting the local (large) blockchain doesnt help, in my tests.
I cant believe its a HDD throughput problem.. Sure, many thousand blocks, but its just a few gigs data?

So I guess "verification/CPU" is the problem. My CPU was at 100% while loading the chain, too..
But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

Ente

No, it was disk bound because of huge numbers of tiny read and write operations. For those without a ramdisk, even an SSD improved the speed dramatically. Disk throughput could not be a factor in download, since a normal internet connection (less than 50mbps) would not be able to saturate a hard drive's throughput speed.

[Pieter Wuille]

No:

verification: CPU is the bottleneck
indexing: HDD latency is the bottleneck
download: network/p2p is the bottleneck

Indexing almost always outweighed the other two, unless you were running on a ramdrive.

[rjk]

No:

verification: CPU is the bottleneck
indexing: HDD latency is the bottleneck
download: network/p2p is the bottleneck

Indexing almost always outweighed the other two, unless you were running on a ramdrive.

Yes, sorry I confused verification with indexing.

[DeathAndTaxes]

But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

It was fast.  The block chain took over 1000 days to create.  If you downloaded and verified it in a day it was ~1000x faster to verify than create.  With things like ramdisk (or improved client which keep portion of chain in ram) and the improvements the developers made to 0.6 it likely is 10,000x faster under optimal conditions.

[FreeMoney]

But then: The point in cryptographic tools (like hashing the block to verify) is that the verify is quick, the reverse (brute-force) painfully slow?

It was fast.  The block chain took over 1000 days to create.  If you downloaded and verified it in a day it was ~1000x faster to verify than create.  With things like ramdisk (or improved client which keep portion of chain in ram) and the improvements the developers made to 0.6 it likely is 10,000x faster under optimal conditions.

No doubt. And it took 1000 days for a very substantial amount of resources. And it could be done in 2 days (now hours?!) by one weak machine.

[DeathAndTaxes]

Of course (can't believe I missed that).  So yeah it is more like a million fold (maybe billion fold) easier to verify than produce.

[torusJKL]

That's great news.
Thanks sipa.

[Pieter Wuille]

Of course (can't believe I missed that).  So yeah it is more like a million fold (maybe billion fold) easier to verify than produce.

That factor is determined exactly by difficulty*4295032833.

That is how much harder it is to find a block nonce at a given difficulty vs. verifying it.

[Pieter Wuille]

That's great news.
Thanks sipa.

I just did some testing for good parameters. Tweaking them was Gavin's idea.

[DeathAndTaxes]

That factor is determined exactly by difficulty*4295032833.

That is how much harder it is to find a block nonce at a given difficulty vs. verifying it.

Thanks for quantifying it.

[cnbtcnews]

顶！

[pastory99]

nice, thanks

[MaxSan]

excellent. looking forward to the next release guys

[bitclown]

This is the kind of headlines that will drive people to help test the pre-releases. Everybody loves massive performance improvements. Fingersnappin' good work!

[Ente]

Thank you for the clarifications and thoughts, guys! Nice!

Ente