Once the height of the next checkpoint is publicly known, everyone may run that script to independently generate bootstrap.dat with the exact same file size and SHA256 checksum.

It will result in seeders already having 90% of any new torrent, thus having to fill in the remaining 10%.


Yes, this is open data and an open file format.  Seeders may independently generate a byte-for-byte identical bootstrap.dat by running https://github.com/jgarzik/pynode/blob/master/mkbootstrap.py

Each time the blockchain torrent is updated, seeders may run that script to guarantee they have 100% of bootstrap.dat immediately.  A nice, decentralized solution

This is special to our use case:  bootstrap.dat is essentially an append-only file.  Blocks are simply concatenated onto the end.

Today's torrent at height 193000 is 2,491,771,562 bytes in size.

The next torrent, a few months from now, will have the same first 2,491,771,562 bytes.

Thus, to bittorrent, the next torrent will simply appear to be a truncated / not fully downloaded bootstrap.dat.  Bittorrent is built to fill in the missing pieces of a file, so that is what it does here

At present, that sort of setup would be challenging for users to use effectively.  It is far easier to have one file to import (automatically, in 0.7.1+) than a collection of files.

Longer term, the idea is to fix any issues with bitcoin P2P peer selection, so that downloading recent blocks from your peers is faster and not burdensome.


That's the plan, with a slight change:  each new torrent will contain all blocks from zero to X.

For bitcoin users, the largest user population served by this is those that have zero blocks, and are jumpstarting a fresh node installation.  Other bitcoin users will likely have blocks fresher than 3-6 months old; catching up via P2P network is fine for them.  There is only a tiny remaining segment of bitcoin users who would then be served by new-torrent-for-checkpoint, those that only turn on their bitcoin clients once every ~6 months.

Note:  For existing torrent seeders, they may simply swap out the .torrent file, perhaps kick their torrent client to manually re-verify a file, and bootstrap.dat in their Uploads directory will simply be extended.  Seeders will automatically already have 90% of each new torrent's bootstrap.dat.

That is the open and operative question.  GLBSE had bots, bidding and asking.  It seems reasonable to assume that any P2P solution would quickly see the same, right out of the gate.

Correct.  It is not much use, without participating seeders.  We definitely have several right now... but it does look like users occasionally get partitioned off into "Azureus island" or "rtorrent island" or "everybody else's island."

Yes, that's the hope.  Already had reports of 7 MB/s downloads...

Sadly it varies wildly depending on your hard drive configuration.  Import time here is under an hour.

Not quite...  this data is the precisely the same data you see on the P2P network.

The client validates bootstrap.dat data to the same level it validates data downloaded from the P2P network.

If all checkpointing code is disabled, this data remains fully valid and useful.


A lot of the time is due to Berkeley DB slowness (fixed by ultraprune).  Another yet-unfixed cause of slowness is poor bitcoin P2P network peer selection, for block download.


We want to do exactly the same amount of validation as the client does with network blocks...  break no additional link in the trust chain

UPDATE Feb 2013: This thread is obsolete.  See the non-beta Bitcoin blockchain data torrent thread for further torrents and updates.


This is a beta test of a blockchain torrent project.  Interested participants are invited to try, and comment.

Version 0.7.1, which just entered testing, includes a new feature:  If the file "bootstrap.dat" is found in the bitcoin data directory, it will validate and import all blockchain data found in that file.  The following torrent presents a bootstrap.dat file for that feature.

Here is the PGP-signed torrent information.  Details follow below the signature.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Torrent info hash: 0bb0521942f586ed96203c6f4d136324756f8a9a
Torrent magnet link:
magnet:?xt=urn:btih:0bb0521942f586ed96203c6f4d136324756f8a9a&dn=bootstrap.dat

Filename: bootstrap.dat
Byte size: 2491771562
SHA1: e70ca90775dfdb13fd0014425805a0bdf4a31677
SHA256: a3f258e7af030165360596e4cb0b9beb24b4ce97352c22e65349b89ad5fc5d3e

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIVAwUBUHeHSdodwg8tvwyoAQJ2DA/8CcINxuD7DzLv96fE+R6GVLvRASLcy0ig
8d2YyDbJ9r9dPVODDPGC7I/ooDVJAcQsqdrLYHST2DTCt6c4zZ/7iXzFEahRopsH
PmxYOgkHie7438nqpmH9uC+d5t0pPHUFS6dBSBgdSVPaLfS86fToXrV/bx30rHBi
60FJz5A99aXrrrUny0cGIjafqVv8XjqNoA1lzjsRjeiu3EgTm8Ibcr8ZI6DLp/80
Siv3potlOArTI6sxjc/vKUa6nZILnW8mKdwc/d8LUdRaBPoo71c6Q4YOSQh/OVht
B1rZ8NEX/2SlS3PbhhMELcY/2wvgPGovkIqOgiw6dDIkhsD8iJAD1DqCkZbsq9S1
kKobGmceuQBcyRUCavkafdHJpZzyCAKLnQLV3FvZ3O4QZQVmYGPYn1H8a5UFRDQq
LGKcQmwm4Cet7162woEiNAaR9p7HvTJ4LV2uEvY56m2GfZbToCk2aiycO+v6Fm8K
ZY8cX+cbEOW3AM5rYVa6Rks029LIrFFzIutlo5MJ7uc7oMqciWrcsPtEh59QY+yJ
SENk6cLyWCAHl4jsvUTBvdNGy3fHKSEyQOjG5cmAMXiJTX/iiB/DJf7koTZAj4ub
wez3/HwuenMYjHgjsVXJAXprcEpTjpEcicg4W0MIDw07dPjol0dnRRNpwfJ1HMRc
dlTqLwlXqgs=
=uwRB
-----END PGP SIGNATURE-----


What is bootstrap.dat?

It is a flat, binary file containing bitcoin blockchain data, from the genesis block through height 193,000.  Height 193,000 is the height of the current checkpoint baked into the reference client.

Version 0.7.1 (when released) will automatically validate and import a file in the data directory named "bootstrap.dat".  Version 0.7 or later will also import this file by passing the command line argument "-loadblock=/path/to/bootstrap.dat" to bitcoin-qt or bitcoind.


Who wants bootstrap.dat?

Anyone bringing up a new node using the reference client.  This is one method of accelerating the initial blockchain download process, while helping the bitcoin P2P network by offloading data download traffic from public P2P nodes.

This download is not for those who are already running the bitcoin client.


NOTE: This torrent requires DHT torrent capability

This torrent is a so-called "trackerless torrent", to avoid making any of the open torrent trackers targets of any bitcoin antipathy.  Peers for this torrent are discovered via DHT, and early results seem to indicate that some bittorrent clients take a while to find their initial peers.  We have also discovered clients (rtorrent) that disable DHT by default; you will need to turn it on.

Please report results, good or bad, on using this trackerless torrent.


How often will this torrent be updated?

Assuming this project is deemed useful and worth continuing... the torrent will be updated once every few months, when the checkpoints are updated in the reference client source code.


Why not update the torrent more often?

A torrent works best when it is a large, static dataset that changes infrequently.  That maximizes the ability to seed the data, enabling even part-timer seeders to contribute meaningfully.  Less frequent changes also minimizes the risk that a malicious torrent will appear, with a long, malicious side chain.  The current policy only updates the torrent after blocks are buried many thousands deep in the chain.


Why should I trust you?

You don't have to:  This data is raw block chain data.  The client will verify this data during import.

Independent third parties may generate their own bootstrap.dat, up to height 193000, and verify that the sha256sum matches that posted above.  The file format is simple and publicly known:

     <4-byte pchMessageStart><32-bit length><CBlock, serialized in network wire format>


Torrent file download?

If the magnet link does not work, download http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent


Comments welcome

Post any comments or experiences in this thread.  I'll update the OP as needed.

Maybe trackerless will be a #fail, but let's see how it goes.

...and here it a call for "it works" testing, for the current release candidate of 0.7.1:
https://bitcointalk.org/index.php?topic=117874.0

Any testing beyond "it works" is appreciated as well of course.  

Volunteers are encouraged to self-organize in this thread to come up (and execute) more formal test plans.

Search the forums for distributed bonds and colored coins.

It seems likely that people will hook up bots to distributed bonds as soon as they pop into existence.  Some sort of mitigation strategy will be needed, at a minimum.  Otherwise the P2P network will be flooded with sales offers.

It would be really nice to have that option.  Bitcoin-funded AWS accounts seem like niche that needs filling.

True, however:  a chain specifically for smart property could potentially keep bloat out of the main chain.

Yes, torrents would be useful -- and some infrastructure is coming down the pipe for this.

Upcoming version 0.7.1, released hopefully this week or early next week, will automatically import 'bootstrap.dat', if that file exists in the bitcoin data directory.

This feature is intended to enable a bootstrap torrent to be downloaded.  Simply drop the torrent's data file (bootstrap.dat) into your directory, and bitcoin will do the rest.

The next step is for some volunteers to generate the torrent, and other volunteers to "beta test" the torrent by downloading it, and making sure it imports correctly into bitcoin.

The nice thing about this process is that these volunteers may be untrusted:  bitcoin will fully verify the input data, just like it does over the network.  A malicious torrent would largely waste everybody's time, but be quickly noticed as malicious.

pybond follows the distributed bond design laid out in that post -- including the P2P and DHT bits.

I am thinking there will be a third component as well:  P2P network will permit traders to advertise "coordination points", a single address where traders may connect for high-speed trading between themselves.  These coordination points may be created or destroyed at any time, and are self-organizing among participants.

No -- losing color by accident (or intentional design) just means the property is lost.  Just like you can "burn money" today by throwing away a private key, or sending bitcoins to a non-existent key, or any number of other ways.

Yes.  More generally, many RPC and P2P requests require a functioning (or simulated) network with remote nodes to query.

A lot of coverage may be gained with simply two nodes, a "local" and "remote" node talking to each other.

However, two nodes will not exercise code important to peer-to-peer software, like good peer selection.

As an aside, typically this is the point where hardware engineers suck in a shocked breath, then gasp "what??? you do not already test every single function?" 

That is the an ideal goal, but it is incredibly difficult given (a) all the different configuration permutations and (b) the need to rely on a mostly volunteer Q/A army.