[RFC] Best method for initial block download?

[jgarzik]

We really need to improve bitcoin's startup time for the first-time user, notably the block chain download.  Has anyone given this any thought already?

I would suggest

• guarantee that blkNNNN.dat is the same on all platforms (Linux, Windows, 32-bit, 64-bit)
• publish zip-compressed block chain up to block XXXXX on bitcoin.org, where XXXXX == the largest height of hashes built into bitcoin
  binary
• publish zip-compressed block chain via trackerless torrent
• call out to remote BT program to download block chain, falling back to HTTP if not working

Another option is to add a more efficient network retrieval method, such as a "download-pkg" command, where a "pkg" is defined as a pre-packaged, pre-compressed range of blocks.  a list of block chain ranges packaged could be retrieved via "list-pkgs".  Or generalize this concept further by permitting the bitcoin network protocol to share and download unspecified binary objects based on SHA-1 hash.

[1715062266]

1715062266

[1715062266]

1715062266

[genjix]

not that bad. i downloaded it and had them all in a few hours

[jgarzik]

Most software doesn't require a few hours, post-install, to be useable...

Compressed, it's not very much data either.  bzip2 says 25MB:

[jgarzik@bd data]$ bzip2 -9v blk0001.dat
  blk0001.dat:  1.445:1,  5.537 bits/byte, 30.78% saved, 36566910 in, 25310234 out.

[LZ]

With XZ it will be about 22.

[caveden]

Most software doesn't require a few hours, post-install, to be useable...

Can't you send coins while the chain is downloading?
Won't you see received transactions with 0 confirmations until the corresponding block is downloaded?

I thought the only problem was the generation, which is not that serious as a problem since the chance of generating something in a couple hours is really tiny.

But anyway, your ideas are good. The faster the better.

[Insti]

1) Download the most recent block (X) first.

As new blocks (Y)..(Z)..(AA)... come in, add them to the chain as normal.
This means new users can start generating straight away, and will see their transactions as soon as they come in.

2) Grow the block chain backwards by fetching (W)..(V)..(U).. etc

Let the user know when they have the full 'trusted' blockchain from the origin.

If they somehow get an evil node X, they'll work it out eventually when the previous blockchain does not validate to the origin.

[caveden]

haha, forgot that "detail"... der!

[nightrow]

What about giving current block chain in the binary release, or put a link to a zipped version on the download page ?
It isn't automated download, but is nonetheless much faster.

[LZ]

I think that giving current block chain in the binary release is bad idea.
It should be block chain up to 74000 block because it is hardcoded.

[theymos]

Most of the "download" time is actually caused by the verification of all the blocks (doing the crypto verification on 130,000+ transactions). The actual download doesn't take very much time at all, as you can see if you download the database files. Things could be sped up considerably if Bitcoin just skipped verification on blocks before the latest checkpoint. There's a philosophical difference, though, which is probably why this is not already done: the checkpoints tell you when the blocks are wrong, but skipping verification would tell you when they are right, which makes things more centralized.