Bitcoin Forum
November 22, 2026, 09:50:28 AM *
News: Latest Bitcoin Core release: 32.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 [12] 13 14 15 16 17 18 19 20 21 22 »  All
  Print  
Author Topic: [ANN] Bitcoin blockchain data torrent  (Read 211339 times)
Sukrim
Legendary
*
Offline

Activity: 2618
Merit: 1014


View Profile
July 06, 2014, 05:41:02 PM
 #221

After checkpoints there is little need to get data from a torrent file instead of the network itself, since it is likely a CPU or disk IO bound operation, not network speed.

Also with this other torrent file, there is no way for me (other than downloading the whole file and decompressing it) to recreate + verify it locally.

https://www.coinlend.org <-- automated lending at various exchanges.
https://www.bitfinex.com <-- Trade BTC for other currencies and vice versa.
EpptEinK
Newbie
*
Offline

Activity: 8
Merit: 0


View Profile
July 07, 2014, 11:40:22 AM
Last edit: July 07, 2014, 01:02:28 PM by EpptEinK
 #222

Hey, thanks for the torrent. I downloaded the file and placed it in my btc folder. I open the client and it says its importing the block from the disk, but nothing hapens, it still says 0 out of 309616 blocks processed. It´s been like that for 1 hour. Is this normal? Thank you Smiley

EDIT: OK, its now syncronizing and the bootstrap.dat changed the name to bootstrap.dat.old, still at 0 blocks...

NEW EDIT: It doesnt sync, what should I do? Thanks.
sd
Hero Member
*****
Offline

Activity: 730
Merit: 500



View Profile
July 07, 2014, 12:31:12 PM
 #223

After checkpoints there is little need to get data from a torrent file instead of the network itself, since it is likely a CPU or disk IO bound operation, not network speed.

Also with this other torrent file, there is no way for me (other than downloading the whole file and decompressing it) to recreate + verify it locally.

Network speed is an issue, that's why we are using a torrent. There are currently 14620 blocks between the end of this torrent and the end of the blockchain. Bitcoind should download about as fast as torrent does, but sadly it doesn't.

Bitcoind verifies this data when it imports it, just like when it verifies data it pulls off random network nodes. The only possible attack is a 50%+1, and if an attacker could pull that off they would do it on the main network, or more likely they would just mine half the coins instead.

EDIT: For clarity
Sukrim
Legendary
*
Offline

Activity: 2618
Merit: 1014


View Profile
July 07, 2014, 01:23:02 PM
 #224

The verification speed when verifying blocks past the latest checkpoint is far slower than getting the blocks from the network usually.

https://www.coinlend.org <-- automated lending at various exchanges.
https://www.bitfinex.com <-- Trade BTC for other currencies and vice versa.
sd
Hero Member
*****
Offline

Activity: 730
Merit: 500



View Profile
July 09, 2014, 08:36:04 AM
 #225

The verification speed when verifying blocks past the latest checkpoint is far slower than getting the blocks from the network usually.

Experimentation being the basis of science I tested your theory. I used a AWS m3.xlarge with a 60GB SSD block device for the bitcoin client. I created a bootstrap.dat for block 305000 and compared the time it takes to update from block 295000 to 305000 via my bootstrap.dat and by pulling blocks from the live network. It turns out you are right.

Time to update from 295000 to 305000 with a custom bootstrap.dat: 3845 seconds
Time to update from 295000 to 305000 by connecting to the live network: 3848 seconds

As a 3 second difference on a 1 hour load is meaningless I can only conclude there is no reason to update this torrent. I'll keep throwing bandwidth at it.
DeeBo
Newbie
*
Offline

Activity: 52
Merit: 0


View Profile
July 10, 2014, 09:18:50 AM
 #226

The verification speed when verifying blocks past the latest checkpoint is far slower than getting the blocks from the network usually.

Experimentation being the basis of science I tested your theory. I used a AWS m3.xlarge with a 60GB SSD block device for the bitcoin client. I created a bootstrap.dat for block 305000 and compared the time it takes to update from block 295000 to 305000 via my bootstrap.dat and by pulling blocks from the live network. It turns out you are right.

Time to update from 295000 to 305000 with a custom bootstrap.dat: 3845 seconds
Time to update from 295000 to 305000 by connecting to the live network: 3848 seconds

As a 3 second difference on a 1 hour load is meaningless I can only conclude there is no reason to update this torrent. I'll keep throwing bandwidth at it.


One flaw in your test:  I'm pretty sure when you give the Bitcoin client a bootstrap.dat file it doesn't just "update" but instead rescans the entire blockchain database on your hard drive.  That's why the bootstrap.dat is meant to be used on a fresh install because if you already have a good chunk of the blockchain downloaded you're usually better off just letting the client update from the network.

Also not everyone has the bandwidth of an extra large AWS instance.  With my slow 3mbps/384kbps DSL line, I may be the exception in that I could possibly rebuild my local database faster than I could download the latest blocks.
sd
Hero Member
*****
Offline

Activity: 730
Merit: 500



View Profile
July 15, 2014, 01:40:00 PM
 #227

One flaw in your test...

I didn't go into exact details about how I tested because discussions like this tend to never end. If there was a significant advantage or disadvantage to using the torrent I think I would have seen it.

Please do treat my results with scepticism and test yourself.
giveBTCpls
Sr. Member
****
Offline

Activity: 322
Merit: 250


View Profile
July 15, 2014, 02:17:01 PM
 #228

Would be too much to ask if they added like, not daily but maybe weekly updated blockchain file? Everytime someone new comes to Bitcoin, the first thing they say is how much of a pain in the ass it is to sync, unless they are using 3rd party wallets that is, but im too used to the simple niceness of the classic wallet to change  Smiley

rav3n_pl
Legendary
*
Offline

Activity: 1361
Merit: 1003


Don`t panic! Organize!


View Profile WWW
July 15, 2014, 06:20:27 PM
 #229

Looks like update needed Smiley
I made a linear block chain up to 309000, 7zip it and put into torrent.
It is like 12,3GB.
My home upload is ~500kb/s so it will take some time to move it to better seeds... hopefully one day tops Smiley
magnet link:
Code:
magnet:?xt=urn:btih:8347fcc3df300e5a31e2304442c128e308256dfc&dn=bitcoin309000&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a80&tr=udp%3a%2f%2ftracker.publicbt.com%3a80&tr=udp%3a%2f%2ftracker.istole.it%3a80%2fannounce&tr=udp%3a%2f%2fopen.demonii.com%3a1337
Have fun Smiley

Looks like at least 10 ppl downloaded it but noone seed? :/

1Rav3nkMayCijuhzcYemMiPYsvcaiwHni  Bitcoin stuff on my OneDrive
My RPC CoinControl for any coin https://bitcointalk.org/index.php?topic=929954
Some stuff on https://github.com/Rav3nPL/
sd
Hero Member
*****
Offline

Activity: 730
Merit: 500



View Profile
July 15, 2014, 09:53:54 PM
 #230

Would be too much to ask if they added like, not daily but maybe weekly updated blockchain file? Everytime someone new comes to Bitcoin, the first thing they say is how much of a pain in the ass it is to sync, unless they are using 3rd party wallets that is, but im too used to the simple niceness of the classic wallet to change  Smiley

There isn't a 'they', there is only an 'us'. If you think it should be done then do it. I was going to but my tests showed that using a bootstrap file doesn't load data into bitcoind any faster than pulling the data from the live network for blocks beyond 295000. The recent blocks seem more complicated than earlier ones. Maybe a bigger bootstrap file would still be useful to some people.

I really think that most users should be using a light client like electrum anyway. Interested companies, organizations, and technical users should be running the full nodes, not every guy who wants to make small purchases.
georgem
Legendary
*
Offline

Activity: 1484
Merit: 1007


spreadcoin.info


View Profile WWW
July 15, 2014, 11:15:28 PM
Last edit: July 16, 2014, 06:27:58 AM by georgem
 #231

I am checking this out now.

A month ago it took me about a day to download the whole blockchain with the wallet.
I will see how fast the torrent and validating is.

At the moment it's very promising, like 25 MB/s download speed.

georgem
Legendary
*
Offline

Activity: 1484
Merit: 1007


spreadcoin.info


View Profile WWW
July 15, 2014, 11:26:43 PM
Last edit: July 16, 2014, 02:42:13 AM by georgem
 #232

Alright. It took 26 minutes to download.
Now starting the wallet..

It says it's importing data.... fine...

10 minutes in... it has imported about 170'000 blocks....

1 hour later ... 230k blocks.

1.5 hours later .... importing block 250k.... importing sure takes a lot of time...  Sad but we are getting there...

2 hours, 270k (that's end of november 2013)... at this rate it should take another hour to finish...

I was wrong... 3 hours, it's at 283k... I give up... I have to get some sleep... lol...

PS progress bar is at about 40%... but don't think its an accurate estimation of time needed.

But if it is, then it might take another 3-4 hours to stop.... will see tomorrow...

georgem
Legendary
*
Offline

Activity: 1484
Merit: 1007


spreadcoin.info


View Profile WWW
July 16, 2014, 06:21:26 AM
 #233

7 weeks to go.
it took 7 hours to get here.
It'll probably finish now in a 1 hour or so...

Now it switched to synchronizing with network for the remaining blocks that weren't in the torrent.

Muhammed Zakir
Hero Member
*****
Offline

Activity: 560
Merit: 518


I prefer Zakir over Muhammed when mentioning me!


View Profile WWW
July 16, 2014, 10:46:46 AM
 #234

Alright. It took 26 minutes to download.
Now starting the wallet..

It says it's importing data.... fine...

10 minutes in... it has imported about 170'000 blocks....

1 hour later ... 230k blocks.

1.5 hours later .... importing block 250k.... importing sure takes a lot of time...  Sad but we are getting there...

2 hours, 270k (that's end of november 2013)... at this rate it should take another hour to finish...

I was wrong... 3 hours, it's at 283k... I give up... I have to get some sleep... lol...

PS progress bar is at about 40%... but don't think its an accurate estimation of time needed.

But if it is, then it might take another 3-4 hours to stop.... will see tomorrow...

You can use this command for increasing syncing speed from your computer not from internet. Smiley
Code:
-dbcache=<n>           Set database cache size in megabytes (4 to 4096, default: 100)

You can use this command to decrease the connection to number of peers, if it is affecting your internet browsing.
Code:
-maxconnections=<n>    Maintain at most <n> connections to peers (default: 125)
IMO it won't affect in a great way but might affect in a very low speed networks.

Kindly,
        MZ

jgarzik (OP)
Legendary
*
Offline

Activity: 1596
Merit: 1164


View Profile
July 31, 2014, 02:39:36 PM
Last edit: July 31, 2014, 02:59:52 PM by jgarzik
 #235

Beta-testing and pre-seeding a new torrent at http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent

Please only join if you already have an old bootstrap.dat file, or you built the new one yourself using linearize.py and bitcoind.

Code:
magnet link:
magnet:?xt=urn:btih:682759b001fac5d3b0d8517d2a179af2e65ebed6&dn=bootstrap.dat&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.ccc.de%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A80

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
2112
Legendary
*
Offline

Activity: 2128
Merit: 1089



View Profile
July 31, 2014, 05:30:08 PM
Last edit: August 05, 2014, 02:31:18 AM by 2112
 #236

Does tracker.ccc.de work for anyone anywhere in the world besides .at, .ch & .de?

Maybe it should be changed to "udp://open.demonii.com:1337" like all the newer torrents? This new tracker is accessible worldwide.

Edit: After reviewing the recent statistics of reach-ability of demonii.com I no longer recommend it.

What about other trackers? Open, less politicized,  trackers, that aren't associated with thepiratebay.se? Or maybe differently politicized trackers associated with other organizations? While those 4 trackers are reliable and well defended against DDoS, they are also the ones that are most commonly blocked/censored/filtered by various software nannies.

Please comment, critique, criticize or ridicule BIP 2112: https://bitcointalk.org/index.php?topic=54382.0
Long-term mining prognosis: https://bitcointalk.org/index.php?topic=91101.0
jgarzik (OP)
Legendary
*
Offline

Activity: 1596
Merit: 1164


View Profile
July 31, 2014, 07:53:49 PM
Last edit: July 31, 2014, 09:46:17 PM by jgarzik
 #237

What about other trackers? Open, less politicized,  trackers, that aren't associated with thepiratebay.se? Or maybe differently politicized trackers associated with other organizations? While those 4 trackers are reliable and well defended against DDoS, they are also the ones that are most commonly blocked/censored/filtered by various software nannies.

I don't pretend to be a tracker expert, and actively solicit suggestions.

I would certainly bias towards less politicized + more reliable trackers.

Please post your tracker suggestions.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
ddrown
Newbie
*
Offline

Activity: 3
Merit: 0


View Profile
July 31, 2014, 09:22:37 PM
Last edit: July 31, 2014, 10:19:52 PM by ddrown
 #238

Since I was curious about the new torrent, here's some info:

new bootstrap.dat contains up to block height 313000 (July 8 )

The new block data (this is using the hexidecimal form) matches my bitcoind's:

$ ./blockheight -l 295001 < bootstrap-313000.dat | sha256sum
6cccd54b215799d91b6781b190903fb5673cf6413d2a93c5ebe26223e3b2de21  -
$ for((i = 295001; i < 313001; i++)); do  ./bitcoin-cli getblock `./bitcoin-cli getblockhash $i` false ; done | sha256sum
6cccd54b215799d91b6781b190903fb5673cf6413d2a93c5ebe26223e3b2de21  -

Previous data matches:

$ dd if=bootstrap-313000.dat bs=335633 count=52206 2>/dev/null | sha1sum
78a4b41ba7ed823c78cd9ca81a6de393c0ccc226  -
$ sha1sum bootstrap-295000.dat
78a4b41ba7ed823c78cd9ca81a6de393c0ccc226  bootstrap-295000.dat


Can you re-add a http:// based tracker?  The UDP trackers aren't supported by older transmission (2.13 from EPEL).  The exodus.desync.com tracker in the previous torrent seems to be working well.

Edit: UDP trackers actually do support IPv6, but support varies depending on your torrent software.
rme
Hero Member
*****
Offline

Activity: 756
Merit: 506



View Profile
July 31, 2014, 10:16:49 PM
 #239

Seeding new torrent 24/7
2112
Legendary
*
Offline

Activity: 2128
Merit: 1089



View Profile
July 31, 2014, 10:45:19 PM
Last edit: August 24, 2014, 02:16:39 AM by 2112
 #240

I don't pretend to be a tracker expert, and actively solicit suggestions.

I would certainly bias towards less politicized + more reliable trackers.

Please post your tracker suggestions.
I nominate two trackers that are open, differently politicized and seem to work reliably globally.

http://tracker.humblebundle.com:2710/announce

http://bttracker.crunchbanglinux.org:6969/announce

If you are changing trackers in your torrent, please replace remove ccc.de with demonii.com. In my opinion it doesn't make sense to use more than 6 classic trackers in addition to DHT and PEX that are used in a non-private torrent. You should be able to change the torrent without changing infohash, the trackers are in the "announce" section that isn't covered by the hash of the "info" section. There are apps and web sites that will help you do that change, although I don't have links handy.

From the other trackers that were used in the past: h33t.com domain is disabled by a complex legal issue and won't be back for a while.

I'm not a torrent&tracker expert either, but have fresh experience troubleshooting these problems for the others.

Edit:

The exodus.desync.com tracker in the previous torrent seems to be working well.

No, please don't. This is an example of dual-stack IPv4+IPv6 tracker that causes problems to many Windows users. Modern Windows versions support IPv6 out-of-the-box via Teredo tunnels and this just causes trouble and confuses the beginning users. The IPv4 side of this tracker is not globally accessible, some big European ISPs are on their block-list.

Edit2:

Upon further review of the recent torrent statistics in Windows I'm going to advise against using open.demonii.com. Again the problem is dual stack IPv4+IPv6 with bad accessibility from various places around the globe.

Edit3:

Since I've posted this humblebundle.com implemented registration & filtering in their tracker, so it won't work with 3rd party torrents.

Please comment, critique, criticize or ridicule BIP 2112: https://bitcointalk.org/index.php?topic=54382.0
Long-term mining prognosis: https://bitcointalk.org/index.php?topic=91101.0
Pages: « 1 2 3 4 5 6 7 8 9 10 11 [12] 13 14 15 16 17 18 19 20 21 22 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!