[ANN] Bitcoin blockchain data torrent

[ddrown]

Since I was curious about the new torrent, here's some info:

new bootstrap.dat contains up to block height 313000 (July 8 )

WTF are you talking about? Last torrent was from april 2014.

See post on page 12:

Beta-testing and pre-seeding a new torrent at http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent

Please only join if you already have an old bootstrap.dat file, or you built the new one yourself using linearize.py and bitcoind.

...

[zvs]

The linearize.py thing always craps out before it finishes for me.  I have the blockchain up to 316481 just as a 7z file, and it's 16,023,242,033 bytes vs this bootstrap from April (?)  that's 17,111,384,xxx...

[rav3n_pl]

See in linearize log, in my case it crashes on rpc timeout sometimes.
I has change timeout in code and run linearize as nice

[jgarzik]

Torrent officially updated!  See OP.  New height:  317,000 @ 22.5 GB

[jgarzik]

Note:  For a minority of users, the bootstrap.dat "one huge 20GB file" import method does not work beyond 4GB of data.  This is due to some limitations with large files on some 32-bit operating systems.  It is planned to move away from a single bootstrap.dat file to a directory full of blkNNNNN.dat files that may be imported directly into bitcoind via "-reindex".  The bootstrap.dat file also causes a 2x disk space increase over the "-reindex" method, which occasionally triggers complaints from users on low disk space cloud servers.

For these reasons, bootstrap.dat import method is being deprecated (at least with regards to this torrent).  Today's torrent update will be the last bootstrap.dat torrent.

Future torrents will include the block chain split into multiple files, never exceeding 1GB in size.

[Sukrim]

As far as I understood it, bootstrap.dat gets loaded automatically while with "-reindex" people need to know how to add something to a command line. I am not sure if this is not causing more issues for more people than the ones trying to run Bitcoin-Core on some ancient hardware.

Also Bitcoin-Core could prompt the user after importing from bootstrap.dat to delete the file, to combat the "double space taken" issue. I'd at least like to see both approaches continued until Bitcoin-Core finally is capable of downloading blocks from more than one source, then this whole thing can be discontinued anyways.

[Adriano]

As far as I understood it, bootstrap.dat gets loaded automatically while with "-reindex" people need to know how to add something to a command line. I am not sure if this is not causing more issues for more people than the ones trying to run Bitcoin-Core on some ancient hardware.

Also Bitcoin-Core could prompt the user after importing from bootstrap.dat to delete the file, to combat the "double space taken" issue. I'd at least like to see both approaches continued until Bitcoin-Core finally is capable of downloading blocks from more than one source, then this whole thing can be discontinued anyways.

I agree keeping the process automagically performed is better for new users.

How about include an empty file (flag file) in the torrent which only purpose is to indicate to the bitcoin daemon/wallet run the reindex process (the daemon/wallet would automatically remove this file after the reindex process is done) ?

Adriano

[sd]

I agree keeping the process automagically performed is better for new users.

Absolutely. Using a torrent at all is a hack and a (small) barrier to entry that should be removed if possible.

But if using a torrent is the current plan I can throw bandwidth at it.

There seem to be more downloaders than seeders on the new torrent right now. After changing torrents I'm downloading at 0kB/s and uploading at 1 MB/s.

[Adriano]

I agree keeping the process automagically performed is better for new users.

Absolutely. Using a torrent at all is a hack and a (small) barrier to entry that should be removed if possible.

But if using a torrent is the current plan I can throw bandwidth at it.

There seem to be more downloaders than seeders on the new torrent right now. After changing torrents I'm downloading at 0kB/s and uploading at 1 MB/s.

If you already have the blockchain you should generate the file using the auxiliary script linearize.py instead of downloading it through the torrent (it is faster than downloading it and doesn't hurt the ones already seeding it, leaving bandwith for the ones who really need to download it).

Despite of that, it is good to have more seeders indeed.

Thanks,


Adriano

[ddrown]

Torrent officially updated!  See OP.  New height:  317,000 @ 22.5 GB

No surprises comparing this file (sha256sum d05062052be4dd6fbaeac5f17598e52f7ad545f9f938acefaf7067ecce1d7b5a) with the blockchain.  I hope posting these verifications is useful to others.

I'm doing it this way because it's faster to stream data out of bootstrap.dat rather than pulling the data out of bitcoin-cli.  So I can generate the sha256sum from the client once for a given range, and use it to verify that the data hasn't changed.  Future updates just need a checksum out of bitcoin-cli covering the new parts.  If anyone's interested in the blockheight program, I can publish it.

Blocks 0 through 295000, bootstrap matches blockchain:

Code:

$ ./blockheight -e 295000 <bootstrap-317000.dat | sha256sum
d83ea8602fb5d2ad863afc7fae1b25794bc7da37d2da642e6aca15f2ad19d9e9  -
$ for((i = 0; i < 295001; i++)); do  ./bitcoin-cli getblock `./bitcoin-cli getblockhash $i` false ; done | sha256sum
d83ea8602fb5d2ad863afc7fae1b25794bc7da37d2da642e6aca15f2ad19d9e9  -

Blocks 295001 through 313000, bootstrap matches blockchain:

Code:

$ ./blockheight -s 295001 -e 313000 < bootstrap-317000.dat | sha256sum
6cccd54b215799d91b6781b190903fb5673cf6413d2a93c5ebe26223e3b2de21  -
$ for((i = 295001; i < 313001; i++)); do  ./bitcoin-cli getblock `./bitcoin-cli getblockhash $i` false ; done | sha256sum
6cccd54b215799d91b6781b190903fb5673cf6413d2a93c5ebe26223e3b2de21  -

Blocks 313001 through 317000, bootstrap matches blockchain:

Code:

$ ./blockheight -s 313001 < bootstrap-317000.dat | sha256sum
a4ab78b82bb7b75b640a6f128fd66cb911362aa8058ebf574b4e02c111f63ba5  -
$ for((i = 313001; i < 317001; i++)); do  ./bitcoin-cli getblock `./bitcoin-cli getblockhash $i` false ; done | sha256sum
a4ab78b82bb7b75b640a6f128fd66cb911362aa8058ebf574b4e02c111f63ba5  -

[jgarzik]

Thanks for double-checking my work!

[VEscudero]

Future torrents will include the block chain split into multiple files, never exceeding 1GB in size.

I agree 100% with this choice, it does not make any sense to have a large file when smaller ones are easier to manage.

This way, people that already has the first blocks can skip those ones and let their bittorrent client only download the last ones.

Armory already downloads the blockchain through bittorrent by default, I wish the bitcore core client start doing it as well, until that moment you can count on me seeding this torrent manually for everyone at any time.

[hops]

I'm having some trouble getting the 22.5 GB file. I have the 21GB which appears to be from last April. I tried using the torrent link from the OP as well as manually adding the infohash. They both lead me to the 21 GB file. Any thought as to what I'm doing wrong?

[Wealthy]

I am confused, what is this data used for? Is this even useful? How can we benefit from it?

[dserrano5]

I am confused, what is this data used for? Is this even useful? How can we benefit from it?

If you're setting up a new full node, it's faster to get the blockchain via torrent (and then let bitcoin core import and validate it) than to let bitcoin core download it by itself. This is because bitcoin core currently downloads from one peer at once, which isn't optimal.

[hops]

i still think the new torrent link is incorrect.

[btcformat]

How much time I need to sync the blockchain from zero using bitcoin-qt and how much with torrent?
I have a 7mb adsl.

[jgarzik]

i still think the new torrent link is incorrect.

Can you be more specific?  Which link is incorrect?

[Muhammed Zakir]

How much time I need to sync the blockchain from zero using bitcoin-qt and how much with torrent?
I have a 7mb adsl.

Downloading bootstrap.dat will take some hours and syncing will take a few hours more. It might go upto 1-2 days.

~~MZ~~

[future_quark369]

I'm downloading. Thank you all. It looks like a lot of time saved using them for more than a pc.