Trusted blockchain download source for mass installation

[poordeveloper]

I'm planning to setup around 20 full nodes, and in order to make the process as fast as possible I was looking for a way to speed the "bootstrap" process.

Last time I did this there was a torrent regularly updated I think, as well as a site with a public download.

That was quite a few years back - now I've found a few of them, but I'm unsure if any of them are maintained by the community / a trusted community member, as I'd obviously prefer to use a trusted data source.

[1714290685]

1714290685

[1714290685]

1714290685

[1714290685]

1714290685

[LoyceV]

From what I've seen, it's best to have Bitcoin Core download everything on it's own.
If you run 20 nodes in the same data center, it might be worth using something like rsync to transfer data from one server to all others.

[poordeveloper]

From what I've seen, it's best to have Bitcoin Core download everything on it's own.
If you run 20 nodes in the same data center, it might be worth using something like rsync to transfer data from one server to all others.

Thanks a lot for your reply.

The plan is for them to be located on as many different locations as possible.

What is the bottleneck with the current version for the download? I'm asking because if it's the network, I can get a 10G server to download the data and use rsync later as you said (some locations have quite limited ports, and I think rsync will be faster in those cases).

[LoyceV]

What is the bottleneck with the current version for the download?

From my experience: it's either CPU, or disk IO.

[Jet Cash]

My bottleneck was bandwidth and peer activity, but I was using public WiFi. I don't recommend this. Using an SSD rather than an HDD does make a big difference when synchronising.

[ABCbits]

There's too much risks downloading blockchain from 3rd party, you better download whole blockchain once on fast PC with fast internet connection then copy it to other 20 devices.

But if you like to live dangerously, there are few sites/thread about it :
1. https://bitcointalk.org/index.php?topic=1310261.0 (outdated)
2. https://getbitcoinblockchain.com/ (rarely see people mention this website)
3. https://bitcointalk.org/index.php?topic=2321650 (also outdated)

What is the bottleneck with the current version for the download? I'm asking because if it's the network, I can get a 10G server to download the data and use rsync later as you said (some locations have quite limited ports, and I think rsync will be faster in those cases).

Ordered by most-common reason :
1. Disk I/O
2. Sync speed
3. CPU
4. RAM

I doubt your 20 devices uses SSD (or have it in big capacity), so you definitely will experience bottleneck on Disk I/O.

[NeuroticFish]

I've read somewhere on Bitcointalk at some point, maybe more than one year ago, that downloading the blockchain bootstrap file has become meaningless because the bottleneck is usually not the internet connection (it's the HDD).

The only use would be to save some bandwidth if you have physical access to all those devices. But does it worth the risk?

[DaCryptoRaccoon]

https://bitcoin.org/bin/block-chain/

Found this torrent link on the bitcoin.org website might be useful.

Note the block height is 317000 for this.  I have no clue how this is maintained or updated someone else might have a better answer for you.

Code:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512


Torrent info hash: 36719ba2cecf9f3bd7c5abfb7a88e939611b536c
Torrent magnet link:
magnet:?xt=urn:btih:36719ba2cecf9f3bd7c5abfb7a88e939611b536c&dn=bootstrap.dat&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Fcoppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337&tr=http%3A%2F%2Fbttracker.crunchbanglinux.org%3A6969%2Fannounce

Block height: 317000
Filename: bootstrap.dat
Byte size: 22566124235
SHA1: 2e6e5dd20cd9c21960ba94f7265f42c4b6443758
SHA256: d05062052be4dd6fbaeac5f17598e52f7ad545f9f938acefaf7067ecce1d7b5a

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQEcBAEBCgAGBQJT+PcrAAoJEL8a4JGi25zKmj4H/i8xKIXW/xnw5i5Pc+ciUuc2
Zo+KxQd1YGRmJSvGsyTgTqK4IJUBD8s8tIGTtznCJ2/Q3wIdZmoqxkST7r4rqJly
cOyJ/n19K5beYvVekIgHVXFmhdJhZf8KNnyuV+DpHmmLKYyZtvkoY9b3hEBHVtSM
AkjuWyA9IEPeghxBX32JAToXOZGs9vZghNyIMg/fWbGLuv1oX7QCjAAua0NAMbLX
diSvVcLXskTRy38W4whz1DX6ruL8mPRYH5Ko6/cmtVqEkr53aEwYf1XJGHHAz3+J
nEHecxBUbiRhDwMdCVtqzKudXWVDtkgVQmRxrkKTw63NTVMge4eEziE2+7JQgq8=
=p9TY
-----END PGP SIGNATURE-----

[achow101]

https://bitcoin.org/bin/block-chain/

Found this torrent link on the bitcoin.org website might be useful.

Note the block height is 317000 for this.  I have no clue how this is maintained or updated someone else might have a better answer for you.

It is not maintained or updated at all anymore. That's the bootstrap.dat file. It will be slower to download that and sync off of it than it will be to let Bitcoin Core do its normal thing to sync.


The bottlenecks you will probably experience will be Disk I/O, CPU, and network speed. As mentioned by others, the fastest way for you would be to sync one node and then copy that to all of the other nodes. You can speed up that initial node's sync time by increase the dbcache if you have a lot of RAM available (>16 GB). Setting dbcache=8000 will yield the best results as the entire database will be able to be held in memory, thus reducing Disk I/O.

[StackGambler]

Sorry, I'm a little new here. I have two questions:

Firstly, to OP. What is the purpose of running 20 nodes? Is it for a bitcoin service, or just to help the network?

Secondly, to everyone. It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.

Thanks guys.

[ranochigo]

Secondly, to everyone. It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.

If you want, you can just copy out your whole Bitcoin data directory, then delete your wallet.dat from your copied data directory. Use a torrent program to share the file. You do need quite a number of seeders since the data directory (blockchain) is relatively big and a large number of seeders is required to spread the load evenly. It might be quite hard to get enough people to seed the file especially since it is not to their best interest to keep two blockchains on the same computer.

[jackg]

Secondly, to everyone. It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.

Thanks guys.

You can use utorrent to seed a torrent directory (or other torrenting software) and transmit it to others..

You'd have to put it into a different folder though and potentially compress it if possible to make the most of your bandwidth. If you advertise it remember to post a hash of it so people can verify it (normally these torrents ignore signatures that are normally required for network transmissions so a MITM attack is possible). The 7zip file manager has some nice hashing features.

[Subspace_Labs]

It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.

Thanks guys.

It sounds like a good idea but it's really not for many people. The problem is that you have to trust the creator of the torrent and trustlessness is one of the main features of Bitcoin. As others have said, downloading it via Bitcoin Core isn't much more resource intensive if you're using a desktop. If you're using some sort of low-power device like a Raspberry Pi, then it will take weeks at best to download and index. If you're using a device like that it's best to download it via Bitcoin Core on a desktop and then copy the data directory over to the Pi. In this case the bottleneck would be CPU and memory. With a fast enough desktop the bottleneck becomes bandwidth.

[HCP]

Secondly, to everyone. It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.

My advice would be "Don't Do it"... you would be simply perpetuating the "myth" that downloading the blockchain via a torrent will decrease the time required to sync... it won't. It is likely to actually be slower than just letting Bitcoin Core do it's thing and sync normally.

Remember... syncing the blockchain is NOT just simply downloading blocks.

For the record, achow101 is a very knowledgeable Bitcoin Core dev... he knows what he is talking about...

It is not maintained or updated at all anymore. That's the bootstrap.dat file. It will be slower to download that and sync off of it than it will be to let Bitcoin Core do its normal thing to sync.


The bottlenecks you will probably experience will be Disk I/O, CPU, and network speed. As mentioned by others, the fastest way for you would be to sync one node and then copy that to all of the other nodes. You can speed up that initial node's sync time by increase the dbcache if you have a lot of RAM available (>16 GB). Setting dbcache=8000 will yield the best results as the entire database will be able to be held in memory, thus reducing Disk I/O.

[jackg]

@hcp the psychology of doing the data download and thenstarting up core is a potential upside though. It's faster to scan the blockchain while you have a physical copy of it and you take out all the downloading to start with. Downloading and syncing via core may be faster but the two individual steps in their own are also faster and more psychologically appealing as it feels like the sync is going faster (from my own experience a few versions ago, it might be out of date entirely).