Bitcoin Forum
April 19, 2014, 09:00:36 PM *
News: Due to the OpenSSL heartbleed bug, changing your forum password is recommended.
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 3 4 5 6 7 8 9 10 11  All
  Print  
Author Topic: [ANN] Bitcoin blockchain data torrent  (Read 58965 times)
jgarzik
Staff
Hero Member
*****
Offline Offline

Activity: 1260


View Profile

Ignore
February 20, 2013, 03:55:14 PM
 #1

This is a torrent of the publicly available bitcoin blockchain data.

Bitcoin-Qt/bitcoind version 0.7.1 and later supports a special import feature:  If the file "bootstrap.dat" is found in the bitcoin data directory, it will validate and import all blockchain data found in that file.  The following torrent presents a bootstrap.dat file for that feature.

Here is the PGP-signed torrent information.  The public key for jgarzik@bitpay.com may be found at public keyservers or http://bitcoin.org/jgarzik-bitpay.asc   Details follow below the signature.

Updated:  April 14, 2014

https://bitcoin.org/bin/blockchain/bootstrap.txt
Code:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512


Torrent info hash: cb7caf0b4c0ee266cce5fbd8f8ba3903f5efa82e
Torrent magnet link:
magnet:?xt=urn:btih:cb7caf0b4c0ee266cce5fbd8f8ba3903f5efa82e&dn=bootstrap.dat&tr=udp://tracker.openbittorrent.com:80&tr=udp://tracker.publicbt.com:80&tr=udp://tracker.ccc.de:80&tr=udp://tracker.istole.it:80

Block height: 295000
Filename: bootstrap.dat
Byte size: 17522056398
SHA1: 78a4b41ba7ed823c78cd9ca81a6de393c0ccc226
SHA256: f451257f4e22e83dd49756b368b5b97dc1f92d31fa6dc114a0a9a69442490f4c

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQEcBAEBCgAGBQJTTBXaAAoJEL8a4JGi25zKLdAH/325k2O2bgaw24xeAT/huOqV
3+b9p4CZJJm6jDUoqGv2AbZ/dKNeKdU1b5dhKwbmlH2j65udeG9nScaiDgsxYEWM
pYf/7zfxAxzBuY15ILi9iHRXfeVrNgFMtrQZAvLEaejDV04zvMRTslSqZ0eZg8Mn
M6tfbJqsrpQY5kamuZMg3zvXHIimzXyvnBV2A0M+fPo7b7s2hi20cyQ7nbkKTsbw
0gujASpHGusCZbmjDcS/O9GIqCoyQwrq+3uTdY2ZmLhSwxTW4Ey8rHm1vyW1VVQQ
tjAAPS+t6XxQwxW4oSLwbppmMv9FB4SS7b9sgNJ5ENH6Bdz1zpgtH0mVSHNdafs=
=6CHB
-----END PGP SIGNATURE-----


What is bootstrap.dat?

It is a flat, binary file containing bitcoin blockchain data, from the genesis block through height 295,000.

Versions 0.7.1+ automatically validates and imports a file in the data directory named "bootstrap.dat".

Special note: Version prior to 0.8.0 have a bug which will only import 2G of data from a file.  This is fixed in 0.8.0.


Who wants bootstrap.dat?

Anyone bringing up a new node using the reference client.  This is one method of accelerating the initial blockchain download process, while helping the bitcoin P2P network by offloading data download traffic from public P2P nodes.

This download is not for those who are already running the bitcoin client.


How often will this torrent be updated?

Assuming this project is deemed useful and worth continuing... the torrent will be updated once every few months, when the checkpoints are updated in the reference client source code.


Why not update the torrent more often?

A torrent works best when it is a large, static dataset that changes infrequently.  That maximizes the ability to seed the data, enabling even part-timer seeders to contribute meaningfully.  Less frequent changes also minimizes the risk that a malicious torrent will appear, with a long, malicious side chain.  The current policy only updates the torrent after blocks are buried many thousands deep in the chain.


Why should I trust you?

You don't have to:  This data is raw block chain data.  The client will verify this data during import.

Independent third parties may generate their own bootstrap.dat, up to height 295,000, and verify that the sha256sum matches that posted above.  The file format is simple and publicly known:

     <4-byte pchMessageStart><32-bit length><CBlock, serialized in network wire format>


Torrent file download?

If the magnet link does not work, download the torrent file from https://bitcoin.org/bin/blockchain/

How can I help?  Do you need more seeds?

Yes, we need as many long term seeds as possible.  This ensures we can meet torrent download demand immediately at high speeds, and remain idle the remainder of the time.


Generate the file myself?

You may generate the file yourself, rather than downloading it, if you have Bitcore Core and python.  The tool used to generate bootstrap.dat is https://github.com/bitcoin/bitcoin/tree/master/contrib/linearize


Comments welcome

Post any comments or experiences in this thread.  I'll update the OP as needed.

Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj

Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1397941236
Hero Member
*
Offline Offline

Posts: 1397941236

View Profile Personal Message (Offline)

Ignore
1397941236
Reply with quote  #2

1397941236
Report to moderator
1397941236
Hero Member
*
Offline Offline

Posts: 1397941236

View Profile Personal Message (Offline)

Ignore
1397941236
Reply with quote  #2

1397941236
Report to moderator
1397941236
Hero Member
*
Offline Offline

Posts: 1397941236

View Profile Personal Message (Offline)

Ignore
1397941236
Reply with quote  #2

1397941236
Report to moderator
Scrat Acorns
Sr. Member
****
Offline Offline

Activity: 293



View Profile

Ignore
February 20, 2013, 04:00:17 PM
 #2

Direct download mirror: https://bitfetch.com/static/bootstrap.7z
(compressed using 7z/LZMA2 - 2.7 vs. 4.7 GiB)

If you can seed, use the torrent instead.
coblee
Donator
Hero Member
*
Offline Offline

Activity: 826


firstbits.com/1ce5j


View Profile WWW

Ignore
February 20, 2013, 05:57:56 PM
 #3

If I already have some block data and want to replace them with this, do I need to delete my blk*.dat files? How about the blocks or chainstate directory?

Litecoin: http://litecoin.org - LhK2kQwiaAvhjWY799cZvMyYwnQAcxkarr
Casascius physical bitcoins database: http://casascius.appspot.com/
BitcoinBug
Full Member
***
Offline Offline

Activity: 195


View Profile

Ignore
February 20, 2013, 06:23:18 PM
 #4

I was seeding the previous version (2.3 GB, share ratio 15) and when I added this one, it picked up and verified the existing version and continued downloading from 51%. Very cool!
mc_lovin
Hero Member
*****
Offline Offline

Activity: 938


www.bitcointrading.com


View Profile WWW

Ignore
February 20, 2013, 06:36:24 PM
 #5

This is a great idea, although I am a bit confused as to the future as we won't all need to keep a full node anymore, but I'm sure that will come into understanding in the coming weeks. 

Seeding this as much as possible!

Sukrim
Hero Member
*****
Offline Offline

Activity: 994


View Profile

Ignore
February 20, 2013, 07:37:08 PM
 #6

How to find the data directory (where this file belongs to):
https://en.bitcoin.it/wiki/Data_directory (<-- this also needs a bit of updating for the different files/folders with 0.8)

Some more hash values of this bootstrap.dat file (up to 216116):
Adler32: FA0EEF4C
CRC32: DDF59E6B
MD5: 1B437D44213B7D98C974546B55834D10
SHA-1: 985380032B618281EC5B9B49AFDCC99A4BCD69EB
SHA-256: BF658C7055B733BFC15EA167F298C5599B89D220B14DBE7C8EF20B18E468C451

Unfortunately it's not possible (yet?) in Bittorrent to share HTTP seeds in a swarm (DDoS potential!), but still - if you have a web server to seed this file but cannot run Bittorrent, you might still post a link here, so you can be added to the Magnet URI/torrent file.
HTTP seeds are in most clients actually quite rarely used, so unless people start to direct download from these you might not even have heavy traffic going on.

Also if you cannot use Bittorrent, there are services around that download torrents for BTC for you and offer a HTTPS download link. I won't link these here as I don't want to make it look like I endorse any of these.

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf | https://just-dice.com/ <-- Bitcoin gambling done right!
jgarzik
Staff
Hero Member
*****
Offline Offline

Activity: 1260


View Profile

Ignore
February 20, 2013, 07:53:52 PM
 #7

If I already have some block data and want to replace them with this, do I need to delete my blk*.dat files? How about the blocks or chainstate directory?

This is not a replacement for block data you already have.  This is a separate file, that is only used once, during data import.

After data import, the file is renamed to bootstrap.dat.old, and never touched again.  Most will want to manually delete it, after use.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
deepceleron
Hero Member
*****
Offline Offline

Activity: 1022



View Profile WWW

Ignore
February 20, 2013, 10:09:02 PM
 #8

If I already have some block data and want to replace them with this, do I need to delete my blk*.dat files? How about the blocks or chainstate directory?

This is not a replacement for block data you already have.  This is a separate file, that is only used once, during data import.

After data import, the file is renamed to bootstrap.dat.old, and never touched again.  Most will want to manually delete it, after use.
More specifically answering the question, if your Bitcoin client is already close to block 216116 (January 11) or even halfway completed (block 190000), the bootstrap.dat likely won't be of much use to you. It may take more time to download and then import the torrent than to complete the blockchain synchronization normally. This torrent is also only for those using the v0.8.0 release or after, as 0.7.1-0.7.2 are not able to import more than 2.0GiB of the 4.7GiB file.

You can use the bootstrap.dat torrent with no ill effects regardless of your current block. When you restart and Bitcoin sees the bootstrap.dat that you have dropped in it's data directory, any block hashes you already have will be skipped quickly, and the import will pick up at the blocks you don't have.

http://we.lovebitco.in/how-bitcoin-works/
1DCeLERonUTsTERdpUNqxKTVMmnwU6reu5
"A Part of Us Remains Wherever We Have Been" - fortune cookie
dooglus
Hero Member
*****
Offline Offline

Activity: 1036


firstbits: 1doog7


View Profile WWW

Ignore
February 21, 2013, 05:03:36 AM
 #9

I have an up-to-date copy of the blockchain.  Is there an easy way to generate bootstrap.dat from it so I can seed it without having to download the whole thing again?

I'm thinking it might be relatively easy to modify the armory source code to dump out a bootstrap.dat file, but won't bother if something already exists.

jgarzik
Staff
Hero Member
*****
Offline Offline

Activity: 1260


View Profile

Ignore
February 21, 2013, 05:12:05 AM
 #10

I have an up-to-date copy of the blockchain.  Is there an easy way to generate bootstrap.dat from it so I can seed it without having to download the whole thing again?

This script is used to generate bootstrap.dat:
https://github.com/jgarzik/pynode/blob/master/mkbootstrap.py

It requires an up-to-date pynode chain database.

In theory, someone could write a script that processes bitcoind's $DATADIR/blocks directory into bootstrap.dat, but no one has written that yet.


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
dooglus
Hero Member
*****
Offline Offline

Activity: 1036


firstbits: 1doog7


View Profile WWW

Ignore
February 21, 2013, 06:45:33 AM
 #11

In theory, someone could write a script that processes bitcoind's $DATADIR/blocks directory into bootstrap.dat, but no one has written that yet.

They have now...

https://gist.github.com/dooglus/5002764

It uses the Armory code to do all the work, and creates a bootstrap.dat with the correct sha256 hash:

$ sha256sum bootstrap.dat
bf658c7055b733bfc15ea167f298c5599b89d220b14dbe7c8ef20b18e468c451  bootstrap.dat

coblee
Donator
Hero Member
*
Offline Offline

Activity: 826


firstbits.com/1ce5j


View Profile WWW

Ignore
February 21, 2013, 08:21:14 AM
 #12

I've tried a few times and couldn't get this to work. First time, I just put the bootstrap.dat file there and tried to run it against my current blocks. It didn't like that and got stuck.

The next time, I deleted all my block data and tried it again. This time it got stuck at block 211551 (iirc) and wouldn't load any block after that. Looking at the debug.log, I see that my peers are trying to give me blocks higher than that but my client keeps rejecting them for some reason.

The third time, I tried the same thing again. This time, when it got to block 155088, it gave me a database corrupt message and the client quit. Restarting the client does the same thing:
init message: Importing blocks from block database...
LevelDB read failure: Corruption: block checksum mismatch

I'm running this on a Mac if that makes a difference. Seems like 0.8 is buggy.

Litecoin: http://litecoin.org - LhK2kQwiaAvhjWY799cZvMyYwnQAcxkarr
Casascius physical bitcoins database: http://casascius.appspot.com/
deepceleron
Hero Member
*****
Offline Offline

Activity: 1022



View Profile WWW

Ignore
February 21, 2013, 08:45:01 AM
 #13

I'm running this on a Mac if that makes a difference. Seems like 0.8 is buggy.

Seems like your computer is sick.

Run Linpack for a few hours, your computer will likely give inconsistent results, crash, hit 99 degrees C, throttle cores.

http://software.intel.com/en-us/articles/intel-math-kernel-library-linpack-download

A forum for it

Or run this Windows software: IntelBurnTest

http://we.lovebitco.in/how-bitcoin-works/
1DCeLERonUTsTERdpUNqxKTVMmnwU6reu5
"A Part of Us Remains Wherever We Have Been" - fortune cookie
jomplox
Member
**
Offline Offline

Activity: 80


Super Saiyan


View Profile

Ignore
February 21, 2013, 07:56:56 PM
 #14

Thanks for your work OP. I'm seeding the new torrent and will be doing so for at least a year at the very least.

1LwmWDsBsi1oHKKrgXxaHGbuvxuU5Qvt1q
neofutur
Full Member
***
Offline Offline

Activity: 145



View Profile WWW

Ignore
February 22, 2013, 09:31:27 AM
 #15

Direct download mirror: https://bitfetch.com/static/bootstrap.7z
(compressed using 7z/LZMA2 - 2.7 vs. 4.7 GiB)
I also added a download on :

http://fh.gd/blockchain/

 you ll find the torrent, the bootstrap.dat and the md5sum, and the same bootstrap.dat gzipped

 I ll try to update it whenever a new torrent and checkpoint comes out
( also seeding the torrent )

 OP: perhaps you should setup a mailing list for mirrors, so you can ping us all when a new version of the torrent have to be downloaded ?

Quote
If you can seed, use the torrent instead.

 same ! prefer the torrent !

mirror sponsored by
http://bitcoin.gw.gd/

mrlithium
Jr. Member
*
Offline Offline

Activity: 46


View Profile

Ignore
March 01, 2013, 05:15:51 PM
 #16

Can this be added to the sourceforge blockchain project sourceforge.net/projects/bitcoin/files/Bitcoin/blockchain/ or update that somehow?
It took me forever to find this thread.
jgarzik
Staff
Hero Member
*****
Offline Offline

Activity: 1260


View Profile

Ignore
March 08, 2013, 07:19:06 PM
 #17

Torrent file now also available via bitcoin's SourceForge page:

     https://sourceforge.net/projects/bitcoin/files/Bitcoin/blockchain/


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
streblo
Full Member
***
Offline Offline

Activity: 165


View Profile

Ignore
March 27, 2013, 09:13:35 PM
 #18

I'm sure this has been discussed before, but I couldn't find it: Why isn't a (moderately up-to-date) bootstrap.dat (optionally) included with the offical bitcoind/bitcoin-qt binaries? The trust model isn't invalidated because (a) you're downloading binaries and (b) the bootstrap's data are verified. Sourceforge already hosts the blockchain. It seems it would remove a lot of redundant data transfer from the bitcoin network, no?
cypherdoc
Hero Member
*****
Offline Offline

Activity: 1120



View Profile

Ignore
March 27, 2013, 09:16:17 PM
 #19

i'll just say that i've been maintaining a few torrent sites since Jeff started this thread.  activity was initially zilch.

in the last couple of weeks the upload bandwidth has skyrocketed.
streblo
Full Member
***
Offline Offline

Activity: 165


View Profile

Ignore
March 27, 2013, 10:58:28 PM
 #20

i'll just say that i've been maintaining a few torrent sites since Jeff started this thread.  activity was initially zilch.

in the last couple of weeks the upload bandwidth has skyrocketed.
Link?
Pages: [1] 2 3 4 5 6 7 8 9 10 11  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!