[ANN] Bitcoin blockchain data torrent

[shorena]

-story-
Now my points:

 - there are computers with slower CPU. Keep this in mind. My case was extreme, but it's just proof, that CPU is also very important during downloading the blockchain, and people having problems with slow downloads perhaps have probelem with CPU, not network.

Yes, the overal speed depends on more than just bandwith.

- precomputation is just reasonable. Why everyone should compute the same blocks every time they start a new node?

You either used the "bootstrap.dat" in which case you did not safe any computation time, but rather wasted it by seperating download and verification. From the times you states and "precomputation" however I assume you downloaded a preverified blockchain. This is naturally faster, because you dont do all the work. Thus the comparisson is invalid and you need to trust the other party to not have messed with the blockchain.

You should verifiy the blocks to avoid this exact problem, trusting someone else. The same way you verify the binaries you download or the signed message you receive.

- if you prepare "certified" blockchain (can be GPG signed, etc.), you will show the people that you CARE about them, and about their time, and freedom of choice of downloading method. This is important for people that are hesitating about running a bitcoin daemon.

Thanks for reading.

Which will require you to trust that person and their ability to secure their PGP key. People are not hesitating to run a full node because of the initial time it takes, but because of the constant time it needs to download and verify data. If it would just be 1-3 days (relativly modern desktop/laptop computer) it would not be an issue.

[washuu]

You either used the "bootstrap.dat" in which case you did not safe any computation time, but rather wasted it by seperating download and verification. From the times you states and "precomputation" however I assume you downloaded a preverified blockchain. This is naturally faster, because you dont do all the work. Thus the comparisson is invalid and you need to trust the other party to not have messed with the blockchain.

You should verifiy the blocks to avoid this exact problem, trusting someone else. The same way you verify the binaries you download or the signed message you receive.

That's exactly my point - why would I have needed trust someone else, if I had had the opportunity to download pre-verified blockchain as a package, certified by bitcoin.org ? That's just giving the people another way of getting bitcoin node up and running very fast. Precomputing means not only wasted CPU cycles, but also HDD operations, as there was reported that every block requires much more HDD transfer, than just size of the blocks.

There still will be option to run bitcoind from scratch, for any people that just want to "fire and forget", and that is good.

[jackg]

January 11th 2016 blockchain full download
~~~~~~~~~~~~~~~~~~~~~~~~~
Here I report my most recent performance as it will help others to decide what hardware is sufficient. 
It was to a VBOX virtual machine in linux on a quad-core i5 throttled to <75% x 4cpu cores.
Download of the whole blockchain from block 1 to present took about 2 days.  /.bitcoin/ is now about 65GB
2009 to 2013 parts of the blockchain maxed out the 4x cpu usage.
After that it slowed down, with 4x cpu usage decreased from about 95% x 4 x 2.3 GHz to (4 x 2.3 GHz about 40% of the time).
I suspect that the most recent 40GB of the blockchain were synching at a rate limited by my HDD rw speed and seek time, so a faster HDD such as a SSD would be the best available speedup, and might get a full blockchain download down to less that 40 hours.  What have others been seeing?

I don't think that the original bootstrap.dat is much help now as it only does the "easy" third at the start of the blockchain.

I was considering running the bitcoin core of a hard drive on a raspbery pi and transferrring the data over afterward to my laptop.

Would this save power? It only has to power a hard drive, raspberry pi and WiFi adapter.

[whizz94]

Definitely no for BTC.

For the whole month of August I had my quad-core B2 raspberry pi trying to download and sort the whole blockchain to its compiled-from-src bitcoin-qt.  It crashed and required bitcoin-qt -reindex several times and in one month of usage never completed the necessary first -reindex task.  It seemed to get slower and slower as the number of weeks remaining decreased to 50 wk, (synched up to mid-2014) and eventually I started using a spare i5 instead, which overtook the rPi in a couple of days.

In 2016, I have this rPi being an unobtanium-qt node and a couple of other altcoins too.  They are all a bit underused but seem to have faster confirmation times than BTC and be better suited to (if my neighbours were to pay pocket money for mowing the lawn).  I've just tested an unobtanium electrum-uno lightweight wallet which showed "pending" within the same minute as sending a test transaction from the rPi node via the international UNO blockchain to .. the other PC across my desk. 

If you are serious about power saving then try something like this by all means, but read around to avoid the various scamcoins and proceeds-of-crime coins which are around.

[dadach]

so downloading this and importing would take longer than 2 weeks my bitcoin core is indicating?

[jackg]

so downloading this and importing would take longer than 2 weeks my bitcoin core is indicating?

Yes, and you might want to look at when one of these was last released. It might not have been for some time considering this thread isn't very active and it's not particularly that useful as the new wallets should load the chain much faster.

[nagobinga]

Both openbittorrent and publicbt work for me, your "seedboxes" will not be able to BitTorrent traffic and everything you might configure it by the way they do not receive public torrent files / links?

[tave]

any bootstrap.dat update 2017?

[OmegaStarScream]

any bootstrap.dat update 2017?

You don't need it anymore and It's no longer as It used to be, It's now slower. If you want to download the blockchain then do it directly from Bitcoin Core, It should take less time If you are running the latest versions.

[rav3n_pl]

IMPORTANT NOTE:  As of Bitcoin Core version 0.10.0 or later, this torrent is slower than a direct download using the bitcoin P2P protocol & client.  This torrent is now only maintained for research and development purposes as a secondary option.  Please use Bitcoin Core directly to download the blockchain.

This topic can be closed? Some people still asking about thing that is not need for like year.

[2112]

This topic can be closed? Some people still asking about thing that is not need for like year.

Please don't close it. Lots of people with unreliable hardware or network problems are still better off downloading separately from initializing the blockchain storage.

There are many people with non-flat-rate network hookups or with over-the-radio hookups with shared IPv4 addresses that continue to download the torrents from this thread.
 

[rav3n_pl]

You do realize, that data in torrent are WAY outdated and new node need get lots of data anyway?
It is NOT good way to bootstrap node.

[2112]

You do realize, that data in torrent are WAY outdated and new node need get lots of data anyway?

So what? It is and it forever will be a proper subset of the valid data. This torrent is sufficient to reliably trigger bugs in certain popular/cheap flash storage devices. This alone would make it an useful test case for at least decade, maybe more.

Also, there are (non-English language) papers that use this torrent for performance comparison of various file systems, alternate storage engines and file/partition layouts.

It is NOT good way to bootstrap node.

This is just your personal limitation that you can't or won't understand why separating over-the-WAN download from over-the-LAN initialization is and will be useful troubleshooting and testing tool.

[Sukrim]

Also, there are (non-English language) papers that use this torrent for performance comparison of various file systems, alternate storage engines and file/partition layouts.

Got some links? Also the ad-hominem is not really useful, using the data in this torrent is objectively a really bad way to bootstrap your node in case you just want to get it up and running as fast as possible.

[miguelmorales85]

there are some people who has contrary belief to what is better:
download the entire boostrap.dat
OR
start a node and download the data from other nodes.

I wonder if someone has the technical knowledge to clarify this

[jackg]

there are some people who has contrary belief to what is better:
download the entire boostrap.dat
OR
start a node and download the data from other nodes.

I wonder if someone has the technical knowledge to clarify this

Until about version 0.10.0 it was much faster to download the bootstrap.dat file.
Since that release of version 0.10.0 it was faster just to leave a node to download it unless you have a dial-up modem or limited internet access (but it'd have to be less than about 100kbps to slow if down by an unreasonable amount that makes downloading impossible). If you have an internet speed that sliw however, if wouldn't be productive to run a node anyway and you may as well use a light wallet instead.

[Jully Bells]

Torrents make heavy use of checksumming/hashing, so while in the block chain you need to have all previous headers to verify the current one, in a torrent file you just need any chunk of data. Other comments were saying that a feature was being implemented so that all of the headers would be downloaded first and the actual blocks could then be downloaded in any order. I'm guessing the client pretty much has to download the headers serially still in order for the distributed nature to continue working. Serial downloading of the blockchain probably makes it easier for the client to check that nobody is bullshitting them with an incorrect version of the blockchain.

[sunk818]

Active torrents. Sharing Nov 7, 2017 - https://blockchaindownload.nl/en/

[SangkuniBird]

It is very convenient, when I install bitcoin-qt on another device, it takes about an hour instead of almost a week

[jackg]

Torrents make heavy use of checksumming/hashing, so while in the block chain you need to have all previous headers to verify the current one, in a torrent file you just need any chunk of data. Other comments were saying that a feature was being implemented so that all of the headers would be downloaded first and the actual blocks could then be downloaded in any order. I'm guessing the client pretty much has to download the headers serially still in order for the distributed nature to continue working. Serial downloading of the blockchain probably makes it easier for the client to check that nobody is bullshitting them with an incorrect version of the blockchain.

Unless you're talking about parallel processes, I don't see how what you're suggesting would increase the download time.
The headers are downloaded first though so that could be what it does... (But as it tracks the height and suggests the number of blocks downloaded this probably doesnt happen). Also, a feature like that would take about longer and more CPU power IMO as it has to check whether it's done a file and call each block relating to the header randomly.