[ANN] Bitcoin blockchain data torrent

[Sukrim]

No change to https://github.com/bitcoin/bitcoin/blob/master/src/checkpoints.cpp, no change to this file.
I'm not going to seed some random stranger's torrent, sorry.

That being said, yes, it really could use an update + new checkpoint...

[rav3n_pl]

Good point!
It is my torrent. I just made it and seed it on my machine.
If your torrent is showing up IPs you can see it is rav3n.dtdns.net
But if you not need/want it - do not download!

[sd]

I'm not going to seed some random stranger's torrent, sorry.

If you connect a full node to the bitcoin network you also download data from random strangers. As I understand it the data in this file is subjected to exactly the same checks as data downloaded by the bitcoin client itself. Should the data be misformatted it will be rejected. If some of the end blocks got reorganized the client will sort it out once it connects to the network proper.

Of course you are right to be paranoid but I don't see the need for this torrent to align with checkpoints in the code itself.

[Sukrim]

After checkpoints there is little need to get data from a torrent file instead of the network itself, since it is likely a CPU or disk IO bound operation, not network speed.

Also with this other torrent file, there is no way for me (other than downloading the whole file and decompressing it) to recreate + verify it locally.

[EpptEinK]

Hey, thanks for the torrent. I downloaded the file and placed it in my btc folder. I open the client and it says its importing the block from the disk, but nothing hapens, it still says 0 out of 309616 blocks processed. It´s been like that for 1 hour. Is this normal? Thank you

EDIT: OK, its now syncronizing and the bootstrap.dat changed the name to bootstrap.dat.old, still at 0 blocks...

NEW EDIT: It doesnt sync, what should I do? Thanks.

[sd]

After checkpoints there is little need to get data from a torrent file instead of the network itself, since it is likely a CPU or disk IO bound operation, not network speed.

Also with this other torrent file, there is no way for me (other than downloading the whole file and decompressing it) to recreate + verify it locally.

Network speed is an issue, that's why we are using a torrent. There are currently 14620 blocks between the end of this torrent and the end of the blockchain. Bitcoind should download about as fast as torrent does, but sadly it doesn't.

Bitcoind verifies this data when it imports it, just like when it verifies data it pulls off random network nodes. The only possible attack is a 50%+1, and if an attacker could pull that off they would do it on the main network, or more likely they would just mine half the coins instead.

EDIT: For clarity

[Sukrim]

The verification speed when verifying blocks past the latest checkpoint is far slower than getting the blocks from the network usually.

[sd]

The verification speed when verifying blocks past the latest checkpoint is far slower than getting the blocks from the network usually.

Experimentation being the basis of science I tested your theory. I used a AWS m3.xlarge with a 60GB SSD block device for the bitcoin client. I created a bootstrap.dat for block 305000 and compared the time it takes to update from block 295000 to 305000 via my bootstrap.dat and by pulling blocks from the live network. It turns out you are right.

Time to update from 295000 to 305000 with a custom bootstrap.dat: 3845 seconds
Time to update from 295000 to 305000 by connecting to the live network: 3848 seconds

As a 3 second difference on a 1 hour load is meaningless I can only conclude there is no reason to update this torrent. I'll keep throwing bandwidth at it.

[DeeBo]

The verification speed when verifying blocks past the latest checkpoint is far slower than getting the blocks from the network usually.

Experimentation being the basis of science I tested your theory. I used a AWS m3.xlarge with a 60GB SSD block device for the bitcoin client. I created a bootstrap.dat for block 305000 and compared the time it takes to update from block 295000 to 305000 via my bootstrap.dat and by pulling blocks from the live network. It turns out you are right.

Time to update from 295000 to 305000 with a custom bootstrap.dat: 3845 seconds
Time to update from 295000 to 305000 by connecting to the live network: 3848 seconds

As a 3 second difference on a 1 hour load is meaningless I can only conclude there is no reason to update this torrent. I'll keep throwing bandwidth at it.

One flaw in your test:  I'm pretty sure when you give the Bitcoin client a bootstrap.dat file it doesn't just "update" but instead rescans the entire blockchain database on your hard drive.  That's why the bootstrap.dat is meant to be used on a fresh install because if you already have a good chunk of the blockchain downloaded you're usually better off just letting the client update from the network.

Also not everyone has the bandwidth of an extra large AWS instance.  With my slow 3mbps/384kbps DSL line, I may be the exception in that I could possibly rebuild my local database faster than I could download the latest blocks.

[sd]

One flaw in your test...

I didn't go into exact details about how I tested because discussions like this tend to never end. If there was a significant advantage or disadvantage to using the torrent I think I would have seen it.

Please do treat my results with scepticism and test yourself.

[giveBTCpls]

Would be too much to ask if they added like, not daily but maybe weekly updated blockchain file? Everytime someone new comes to Bitcoin, the first thing they say is how much of a pain in the ass it is to sync, unless they are using 3rd party wallets that is, but im too used to the simple niceness of the classic wallet to change 

[rav3n_pl]

Looks like update needed
I made a linear block chain up to 309000, 7zip it and put into torrent.
It is like 12,3GB.
My home upload is ~500kb/s so it will take some time to move it to better seeds... hopefully one day tops
magnet link:

Code:

magnet:?xt=urn:btih:8347fcc3df300e5a31e2304442c128e308256dfc&dn=bitcoin309000&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a80&tr=udp%3a%2f%2ftracker.publicbt.com%3a80&tr=udp%3a%2f%2ftracker.istole.it%3a80%2fannounce&tr=udp%3a%2f%2fopen.demonii.com%3a1337

Have fun

Looks like at least 10 ppl downloaded it but noone seed? :/

[sd]

Would be too much to ask if they added like, not daily but maybe weekly updated blockchain file? Everytime someone new comes to Bitcoin, the first thing they say is how much of a pain in the ass it is to sync, unless they are using 3rd party wallets that is, but im too used to the simple niceness of the classic wallet to change  

There isn't a 'they', there is only an 'us'. If you think it should be done then do it. I was going to but my tests showed that using a bootstrap file doesn't load data into bitcoind any faster than pulling the data from the live network for blocks beyond 295000. The recent blocks seem more complicated than earlier ones. Maybe a bigger bootstrap file would still be useful to some people.

I really think that most users should be using a light client like electrum anyway. Interested companies, organizations, and technical users should be running the full nodes, not every guy who wants to make small purchases.

[georgem]

I am checking this out now.

A month ago it took me about a day to download the whole blockchain with the wallet.
I will see how fast the torrent and validating is.

At the moment it's very promising, like 25 MB/s download speed.

[georgem]

Alright. It took 26 minutes to download.
Now starting the wallet..

It says it's importing data.... fine...

10 minutes in... it has imported about 170'000 blocks....

1 hour later ... 230k blocks.

1.5 hours later .... importing block 250k.... importing sure takes a lot of time...   but we are getting there...

2 hours, 270k (that's end of november 2013)... at this rate it should take another hour to finish...

I was wrong... 3 hours, it's at 283k... I give up... I have to get some sleep... lol...

PS progress bar is at about 40%... but don't think its an accurate estimation of time needed.

But if it is, then it might take another 3-4 hours to stop.... will see tomorrow...

[georgem]

7 weeks to go.
it took 7 hours to get here.
It'll probably finish now in a 1 hour or so...

Now it switched to synchronizing with network for the remaining blocks that weren't in the torrent.

[Muhammed Zakir]

Alright. It took 26 minutes to download.
Now starting the wallet..

It says it's importing data.... fine...

10 minutes in... it has imported about 170'000 blocks....

1 hour later ... 230k blocks.

1.5 hours later .... importing block 250k.... importing sure takes a lot of time...   but we are getting there...

2 hours, 270k (that's end of november 2013)... at this rate it should take another hour to finish...

I was wrong... 3 hours, it's at 283k... I give up... I have to get some sleep... lol...

PS progress bar is at about 40%... but don't think its an accurate estimation of time needed.

But if it is, then it might take another 3-4 hours to stop.... will see tomorrow...

You can use this command for increasing syncing speed from your computer not from internet.

Code:

-dbcache=<n>           Set database cache size in megabytes (4 to 4096, default: 100)

You can use this command to decrease the connection to number of peers, if it is affecting your internet browsing.

Code:

-maxconnections=<n>    Maintain at most <n> connections to peers (default: 125)

IMO it won't affect in a great way but might affect in a very low speed networks.

Kindly,
        MZ

[jgarzik]

Beta-testing and pre-seeding a new torrent at http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent

Please only join if you already have an old bootstrap.dat file, or you built the new one yourself using linearize.py and bitcoind.

Code:

magnet link:
magnet:?xt=urn:btih:682759b001fac5d3b0d8517d2a179af2e65ebed6&dn=bootstrap.dat&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.ccc.de%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A80

[2112]

Does tracker.ccc.de work for anyone anywhere in the world besides .at, .ch & .de?

Maybe it should be changed to "udp://open.demonii.com:1337" like all the newer torrents? This new tracker is accessible worldwide.

Edit: After reviewing the recent statistics of reach-ability of demonii.com I no longer recommend it.

What about other trackers? Open, less politicized,  trackers, that aren't associated with thepiratebay.se? Or maybe differently politicized trackers associated with other organizations? While those 4 trackers are reliable and well defended against DDoS, they are also the ones that are most commonly blocked/censored/filtered by various software nannies.

[jgarzik]

What about other trackers? Open, less politicized,  trackers, that aren't associated with thepiratebay.se? Or maybe differently politicized trackers associated with other organizations? While those 4 trackers are reliable and well defended against DDoS, they are also the ones that are most commonly blocked/censored/filtered by various software nannies.

I don't pretend to be a tracker expert, and actively solicit suggestions.

I would certainly bias towards less politicized + more reliable trackers.

Please post your tracker suggestions.