[ANN] Bitcoin blockchain data torrent

[jgarzik]

This is a torrent of the publicly available bitcoin blockchain data.

IMPORTANT NOTE:  As of Bitcoin Core version 0.10.0 or later, this torrent is slower than a direct download using the bitcoin P2P protocol & client.  This torrent is now only maintained for research and development purposes as a secondary option.  Please use Bitcoin Core directly to download the blockchain.

Bitcoin-Qt/bitcoind version 0.7.1 and later supports a special import feature:  If the file "bootstrap.dat" is found in the bitcoin data directory, it will validate and import all blockchain data found in that file.  The following torrent presents a bootstrap.dat file for that feature.

Here is the PGP-signed torrent information.  The public key for jgarzik@bitpay.com may be found at public keyservers or http://bitcoin.org/jgarzik-bitpay.asc   Details follow below the signature.

Updated:  August 23, 2014

http://gtf.org/garzik/bitcoin/bootstrap.txt

Code:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512


Torrent info hash: 36719ba2cecf9f3bd7c5abfb7a88e939611b536c
Torrent magnet link:
magnet:?xt=urn:btih:36719ba2cecf9f3bd7c5abfb7a88e939611b536c&dn=bootstrap.dat&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Fcoppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337&tr=http%3A%2F%2Fbttracker.crunchbanglinux.org%3A6969%2Fannounce

Block height: 317000
Filename: bootstrap.dat
Byte size: 22566124235
SHA1: 2e6e5dd20cd9c21960ba94f7265f42c4b6443758
SHA256: d05062052be4dd6fbaeac5f17598e52f7ad545f9f938acefaf7067ecce1d7b5a

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQEcBAEBCgAGBQJT+PcrAAoJEL8a4JGi25zKmj4H/i8xKIXW/xnw5i5Pc+ciUuc2
Zo+KxQd1YGRmJSvGsyTgTqK4IJUBD8s8tIGTtznCJ2/Q3wIdZmoqxkST7r4rqJly
cOyJ/n19K5beYvVekIgHVXFmhdJhZf8KNnyuV+DpHmmLKYyZtvkoY9b3hEBHVtSM
AkjuWyA9IEPeghxBX32JAToXOZGs9vZghNyIMg/fWbGLuv1oX7QCjAAua0NAMbLX
diSvVcLXskTRy38W4whz1DX6ruL8mPRYH5Ko6/cmtVqEkr53aEwYf1XJGHHAz3+J
nEHecxBUbiRhDwMdCVtqzKudXWVDtkgVQmRxrkKTw63NTVMge4eEziE2+7JQgq8=
=p9TY
-----END PGP SIGNATURE-----

What is bootstrap.dat?

It is a flat, binary file containing bitcoin blockchain data, from the genesis block through height 317,000.

Versions 0.7.1+ automatically validates and imports a file in the data directory named "bootstrap.dat".

Special note: Version prior to 0.8.0 have a bug which will only import 2G of data from a file.  This is fixed in 0.8.0.


Who wants bootstrap.dat?

Anyone bringing up a new node using the reference client.  This is one method of accelerating the initial blockchain download process, while helping the bitcoin P2P network by offloading data download traffic from public P2P nodes.

This download is not for those who are already running the bitcoin client.


How often will this torrent be updated?

Assuming this project is deemed useful and worth continuing... the torrent will be updated once every few months, when the checkpoints are updated in the reference client source code.


Why not update the torrent more often?

A torrent works best when it is a large, static dataset that changes infrequently.  That maximizes the ability to seed the data, enabling even part-timer seeders to contribute meaningfully.  Less frequent changes also minimizes the risk that a malicious torrent will appear, with a long, malicious side chain.  The current policy only updates the torrent after blocks are buried many thousands deep in the chain.


Why should I trust you?

You don't have to:  This data is raw block chain data.  The client will verify this data during import.

Independent third parties may generate their own bootstrap.dat, up to height 317,000, and verify that the sha256sum matches that posted above.  The file format is simple and publicly known:

     <4-byte pchMessageStart><32-bit length><CBlock, serialized in network wire format>


Torrent file download?

If the magnet link does not work, download the torrent file from https://bitcoin.org/bin/block-chain/   http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent

How can I help?  Do you need more seeds?

Yes, we need as many long term seeds as possible.  This ensures we can meet torrent download demand immediately at high speeds, and remain idle the remainder of the time.


Generate the file myself?

You may generate the file yourself, rather than downloading it, if you have Bitcore Core and python.  The tool used to generate bootstrap.dat is https://github.com/bitcoin/bitcoin/tree/master/contrib/linearize


Comments welcome

Post any comments or experiences in this thread.  I'll update the OP as needed.

[Scrat Acorns]

Direct download mirror: https://bitfetch.com/static/bootstrap.7z
(compressed using 7z/LZMA2 - 2.7 vs. 4.7 GiB)

If you can seed, use the torrent instead.

[coblee]

If I already have some block data and want to replace them with this, do I need to delete my blk*.dat files? How about the blocks or chainstate directory?

[BitcoinBug]

I was seeding the previous version (2.3 GB, share ratio 15) and when I added this one, it picked up and verified the existing version and continued downloading from 51%. Very cool!

[mc_lovin]

This is a great idea, although I am a bit confused as to the future as we won't all need to keep a full node anymore, but I'm sure that will come into understanding in the coming weeks. 

Seeding this as much as possible!

[Sukrim]

How to find the data directory (where this file belongs to):
https://en.bitcoin.it/wiki/Data_directory (<-- this also needs a bit of updating for the different files/folders with 0.8)

Some more hash values of this bootstrap.dat file (up to 216116):
Adler32: FA0EEF4C
CRC32: DDF59E6B
MD5: 1B437D44213B7D98C974546B55834D10
SHA-1: 985380032B618281EC5B9B49AFDCC99A4BCD69EB
SHA-256: BF658C7055B733BFC15EA167F298C5599B89D220B14DBE7C8EF20B18E468C451

Unfortunately it's not possible (yet?) in Bittorrent to share HTTP seeds in a swarm (DDoS potential!), but still - if you have a web server to seed this file but cannot run Bittorrent, you might still post a link here, so you can be added to the Magnet URI/torrent file.
HTTP seeds are in most clients actually quite rarely used, so unless people start to direct download from these you might not even have heavy traffic going on.

Also if you cannot use Bittorrent, there are services around that download torrents for BTC for you and offer a HTTPS download link. I won't link these here as I don't want to make it look like I endorse any of these.

[jgarzik]

If I already have some block data and want to replace them with this, do I need to delete my blk*.dat files? How about the blocks or chainstate directory?

This is not a replacement for block data you already have.  This is a separate file, that is only used once, during data import.

After data import, the file is renamed to bootstrap.dat.old, and never touched again.  Most will want to manually delete it, after use.

[deepceleron]

If I already have some block data and want to replace them with this, do I need to delete my blk*.dat files? How about the blocks or chainstate directory?

This is not a replacement for block data you already have.  This is a separate file, that is only used once, during data import.

After data import, the file is renamed to bootstrap.dat.old, and never touched again.  Most will want to manually delete it, after use.

More specifically answering the question, if your Bitcoin client is already close to block 216116 (January 11) or even halfway completed (block 190000), the bootstrap.dat likely won't be of much use to you. It may take more time to download and then import the torrent than to complete the blockchain synchronization normally. This torrent is also only for those using the v0.8.0 release or after, as 0.7.1-0.7.2 are not able to import more than 2.0GiB of the 4.7GiB file.

You can use the bootstrap.dat torrent with no ill effects regardless of your current block. When you restart and Bitcoin sees the bootstrap.dat that you have dropped in it's data directory, any block hashes you already have will be skipped quickly, and the import will pick up at the blocks you don't have.

[dooglus]

I have an up-to-date copy of the blockchain.  Is there an easy way to generate bootstrap.dat from it so I can seed it without having to download the whole thing again?

I'm thinking it might be relatively easy to modify the armory source code to dump out a bootstrap.dat file, but won't bother if something already exists.

[jgarzik]

I have an up-to-date copy of the blockchain.  Is there an easy way to generate bootstrap.dat from it so I can seed it without having to download the whole thing again?

This script is used to generate bootstrap.dat:
https://github.com/jgarzik/pynode/blob/master/mkbootstrap.py

It requires an up-to-date pynode chain database.

In theory, someone could write a script that processes bitcoind's $DATADIR/blocks directory into bootstrap.dat, but no one has written that yet.

[dooglus]

In theory, someone could write a script that processes bitcoind's $DATADIR/blocks directory into bootstrap.dat, but no one has written that yet.

They have now...

https://gist.github.com/dooglus/5002764

It uses the Armory code to do all the work, and creates a bootstrap.dat with the correct sha256 hash:

$ sha256sum bootstrap.dat
bf658c7055b733bfc15ea167f298c5599b89d220b14dbe7c8ef20b18e468c451  bootstrap.dat

[coblee]

I've tried a few times and couldn't get this to work. First time, I just put the bootstrap.dat file there and tried to run it against my current blocks. It didn't like that and got stuck.

The next time, I deleted all my block data and tried it again. This time it got stuck at block 211551 (iirc) and wouldn't load any block after that. Looking at the debug.log, I see that my peers are trying to give me blocks higher than that but my client keeps rejecting them for some reason.

The third time, I tried the same thing again. This time, when it got to block 155088, it gave me a database corrupt message and the client quit. Restarting the client does the same thing:
init message: Importing blocks from block database...
LevelDB read failure: Corruption: block checksum mismatch

I'm running this on a Mac if that makes a difference. Seems like 0.8 is buggy.

[deepceleron]

I'm running this on a Mac if that makes a difference. Seems like 0.8 is buggy.

Seems like your computer is sick.

Run Linpack for a few hours, your computer will likely give inconsistent results, crash, hit 99 degrees C, throttle cores.

http://software.intel.com/en-us/articles/intel-math-kernel-library-linpack-download

A forum for it

Or run this Windows software: IntelBurnTest

[jomplox]

Thanks for your work OP. I'm seeding the new torrent and will be doing so for at least a year at the very least.

[neofutur]

Direct download mirror: https://bitfetch.com/static/bootstrap.7z
(compressed using 7z/LZMA2 - 2.7 vs. 4.7 GiB)

I also added a download on :

http://fh.gd/blockchain/

 you ll find the torrent, the bootstrap.dat and the md5sum, and the same bootstrap.dat gzipped

 I ll try to update it whenever a new torrent and checkpoint comes out
( also seeding the torrent )

 OP: perhaps you should setup a mailing list for mirrors, so you can ping us all when a new version of the torrent have to be downloaded ?

If you can seed, use the torrent instead.

 same ! prefer the torrent !

mirror sponsored by http://bitcoin.gw.gd/

[mrlithium]

Can this be added to the sourceforge blockchain project sourceforge.net/projects/bitcoin/files/Bitcoin/blockchain/ or update that somehow?
It took me forever to find this thread.

[jgarzik]

Torrent file now also available via bitcoin's SourceForge page:

     https://sourceforge.net/projects/bitcoin/files/Bitcoin/blockchain/

[streblo]

I'm sure this has been discussed before, but I couldn't find it: Why isn't a (moderately up-to-date) bootstrap.dat (optionally) included with the offical bitcoind/bitcoin-qt binaries? The trust model isn't invalidated because (a) you're downloading binaries and (b) the bootstrap's data are verified. Sourceforge already hosts the blockchain. It seems it would remove a lot of redundant data transfer from the bitcoin network, no?

[cypherdoc]

i'll just say that i've been maintaining a few torrent sites since Jeff started this thread.  activity was initially zilch.

in the last couple of weeks the upload bandwidth has skyrocketed.

[streblo]

i'll just say that i've been maintaining a few torrent sites since Jeff started this thread.  activity was initially zilch.

in the last couple of weeks the upload bandwidth has skyrocketed.

Link?