Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: jgarzik on October 18, 2010, 05:59:20 AM



Title: [RFC] Best method for initial block download?
Post by: jgarzik on October 18, 2010, 05:59:20 AM

We really need to improve bitcoin's startup time for the first-time user, notably the block chain download.  Has anyone given this any thought already?

I would suggest
  • guarantee that blkNNNN.dat is the same on all platforms (Linux, Windows, 32-bit, 64-bit)
  • publish zip-compressed block chain up to block XXXXX on bitcoin.org, where XXXXX == the largest height of hashes built into bitcoin
    binary
  • publish zip-compressed block chain via trackerless torrent
  • call out to remote BT program to download block chain, falling back to HTTP if not working

Another option is to add a more efficient network retrieval method, such as a "download-pkg" command, where a "pkg" is defined as a pre-packaged, pre-compressed range of blocks.  a list of block chain ranges packaged could be retrieved via "list-pkgs".  Or generalize this concept further by permitting the bitcoin network protocol to share and download unspecified binary objects based on SHA-1 hash.


Title: Re: [RFC] Best method for initial block download?
Post by: genjix on October 18, 2010, 06:36:27 AM
not that bad. i downloaded it and had them all in a few hours


Title: Re: [RFC] Best method for initial block download?
Post by: jgarzik on October 18, 2010, 06:38:41 AM
Most software doesn't require a few hours, post-install, to be useable...

Compressed, it's not very much data either.  bzip2 says 25MB:
Quote
[jgarzik@bd data]$ bzip2 -9v blk0001.dat
  blk0001.dat:  1.445:1,  5.537 bits/byte, 30.78% saved, 36566910 in, 25310234 out.


Title: Re: [RFC] Best method for initial block download?
Post by: LZ on October 18, 2010, 07:17:14 AM
With XZ it will be about 22. :)


Title: Re: [RFC] Best method for initial block download?
Post by: caveden on October 18, 2010, 09:53:15 AM
Most software doesn't require a few hours, post-install, to be useable...

Can't you send coins while the chain is downloading?
Won't you see received transactions with 0 confirmations until the corresponding block is downloaded?

I thought the only problem was the generation, which is not that serious as a problem since the chance of generating something in a couple hours is really tiny.

But anyway, your ideas are good. The faster the better. :)



Title: Re: [RFC] Best method for initial block download?
Post by: Insti on October 18, 2010, 11:25:46 AM

1) Download the most recent block (X) first.

As new blocks (Y)..(Z)..(AA)... come in, add them to the chain as normal.
This means new users can start generating straight away, and will see their transactions as soon as they come in.

2) Grow the block chain backwards by fetching (W)..(V)..(U).. etc

Let the user know when they have the full 'trusted' blockchain from the origin.

If they somehow get an evil node X, they'll work it out eventually when the previous blockchain does not validate to the origin.







Title: Re: [RFC] Best method for initial block download?
Post by: caveden on October 18, 2010, 02:50:12 PM
haha, forgot that "detail"... der!


Title: Re: [RFC] Best method for initial block download?
Post by: nightrow on October 18, 2010, 06:56:44 PM
What about giving current block chain in the binary release, or put a link to a zipped version on the download page ?
It isn't automated download, but is nonetheless much faster.


Title: Re: [RFC] Best method for initial block download?
Post by: LZ on October 18, 2010, 09:27:27 PM
I think that giving current block chain in the binary release is bad idea.
It should be block chain up to 74000 block because it is hardcoded. :)


Title: Re: [RFC] Best method for initial block download?
Post by: theymos on October 18, 2010, 11:26:44 PM
Most of the "download" time is actually caused by the verification of all the blocks (doing the crypto verification on 130,000+ transactions). The actual download doesn't take very much time at all, as you can see if you download the database files. Things could be sped up considerably if Bitcoin just skipped verification on blocks before the latest checkpoint. There's a philosophical difference, though, which is probably why this is not already done: the checkpoints tell you when the blocks are wrong, but skipping verification would tell you when they are right, which makes things more centralized.