Bitcoin Forum
November 17, 2019, 02:41:23 PM *
News: Help collect the most notable posts made over the last 10 years.
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Trusted blockchain download source for mass installation  (Read 226 times)
poordeveloper
Hero Member
*****
Offline Offline

Activity: 896
Merit: 527


₿₿₿₿₿₿₿


View Profile WWW
March 09, 2019, 01:05:03 PM
Merited by DarkStar_ (3), vapourminer (1), LoyceV (1)
 #1

I'm planning to setup around 20 full nodes, and in order to make the process as fast as possible I was looking for a way to speed the "bootstrap" process.

Last time I did this there was a torrent regularly updated I think, as well as a site with a public download.

That was quite a few years back - now I've found a few of them, but I'm unsure if any of them are maintained by the community / a trusted community member, as I'd obviously prefer to use a trusted data source.

🎰 Bitcoin Casinos ⭐⭐⭐⭐⭐
.
🔵 Buy Bitcoin (Visa / Mastercard / SEPA / Bank Transfer / Western Union / MoneyGram / RIA)
1574001683
Hero Member
*
Offline Offline

Posts: 1574001683

View Profile Personal Message (Offline)

Ignore
1574001683
Reply with quote  #2

1574001683
Report to moderator
1574001683
Hero Member
*
Offline Offline

Posts: 1574001683

View Profile Personal Message (Offline)

Ignore
1574001683
Reply with quote  #2

1574001683
Report to moderator
1574001683
Hero Member
*
Offline Offline

Posts: 1574001683

View Profile Personal Message (Offline)

Ignore
1574001683
Reply with quote  #2

1574001683
Report to moderator
The Bitcoin Forum is turning 10 years old! Join the community in sharing and exploring the notable posts made over the years.
1574001683
Hero Member
*
Offline Offline

Posts: 1574001683

View Profile Personal Message (Offline)

Ignore
1574001683
Reply with quote  #2

1574001683
Report to moderator
LoyceV
Legendary
*
Online Online

Activity: 1666
Merit: 5060


Largest Merit Circle on BPIP!


View Profile WWW
March 09, 2019, 01:25:18 PM
Merited by LFC_Bitcoin (1)
 #2

From what I've seen, it's best to have Bitcoin Core download everything on it's own.
If you run 20 nodes in the same data center, it might be worth using something like rsync to transfer data from one server to all others.

poordeveloper
Hero Member
*****
Offline Offline

Activity: 896
Merit: 527


₿₿₿₿₿₿₿


View Profile WWW
March 09, 2019, 01:36:47 PM
 #3

From what I've seen, it's best to have Bitcoin Core download everything on it's own.
If you run 20 nodes in the same data center, it might be worth using something like rsync to transfer data from one server to all others.
Thanks a lot for your reply.

The plan is for them to be located on as many different locations as possible.

What is the bottleneck with the current version for the download? I'm asking because if it's the network, I can get a 10G server to download the data and use rsync later as you said (some locations have quite limited ports, and I think rsync will be faster in those cases).

🎰 Bitcoin Casinos ⭐⭐⭐⭐⭐
.
🔵 Buy Bitcoin (Visa / Mastercard / SEPA / Bank Transfer / Western Union / MoneyGram / RIA)
LoyceV
Legendary
*
Online Online

Activity: 1666
Merit: 5060


Largest Merit Circle on BPIP!


View Profile WWW
March 09, 2019, 01:49:27 PM
Merited by poordeveloper (2)
 #4

What is the bottleneck with the current version for the download?
From my experience: it's either CPU, or disk IO.

Jet Cash
Legendary
*
Offline Offline

Activity: 1428
Merit: 1795


Trying to preserve our heritage.


View Profile WWW
March 09, 2019, 03:21:29 PM
Merited by poordeveloper (2)
 #5

My bottleneck was bandwidth and peer activity, but I was using public WiFi. I don't recommend this. Using an SSD rather than an HDD does make a big difference when synchronising.

I've added a photography section to Fit to Talk -  The photography and content production board

Hopefully we can use this to help members earn from using budget photo kit whilst they practice their English
ETFbitcoin
Legendary
*
Offline Offline

Activity: 1820
Merit: 2086

Use SegWit and enjoy lower fees.


View Profile WWW
March 09, 2019, 06:16:16 PM
Merited by poordeveloper (2)
 #6

There's too much risks downloading blockchain from 3rd party, you better download whole blockchain once on fast PC with fast internet connection then copy it to other 20 devices.

But if you like to live dangerously, there are few sites/thread about it :
1. https://bitcointalk.org/index.php?topic=1310261.0 (outdated)
2. https://getbitcoinblockchain.com/ (rarely see people mention this website)
3. https://bitcointalk.org/index.php?topic=2321650 (also outdated)

What is the bottleneck with the current version for the download? I'm asking because if it's the network, I can get a 10G server to download the data and use rsync later as you said (some locations have quite limited ports, and I think rsync will be faster in those cases).

Ordered by most-common reason :
1. Disk I/O
2. Sync speed
3. CPU
4. RAM

I doubt your 20 devices uses SSD (or have it in big capacity), so you definitely will experience bottleneck on Disk I/O.

NeuroticFish
Legendary
*
Offline Offline

Activity: 2030
Merit: 1361


There are no mistakes. Only opportunities wasted.


View Profile
March 09, 2019, 08:00:58 PM
 #7

I've read somewhere on Bitcointalk at some point, maybe more than one year ago, that downloading the blockchain bootstrap file has become meaningless because the bottleneck is usually not the internet connection (it's the HDD).

The only use would be to save some bandwidth if you have physical access to all those devices. But does it worth the risk?

MagicByt3
Full Member
***
Offline Offline

Activity: 364
Merit: 186


View Profile
March 11, 2019, 08:59:57 PM
 #8

https://bitcoin.org/bin/block-chain/

Found this torrent link on the bitcoin.org website might be useful.

Note the block height is 317000 for this.  I have no clue how this is maintained or updated someone else might have a better answer for you.



Code:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512


Torrent info hash: 36719ba2cecf9f3bd7c5abfb7a88e939611b536c
Torrent magnet link:
magnet:?xt=urn:btih:36719ba2cecf9f3bd7c5abfb7a88e939611b536c&dn=bootstrap.dat&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Fcoppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337&tr=http%3A%2F%2Fbttracker.crunchbanglinux.org%3A6969%2Fannounce

Block height: 317000
Filename: bootstrap.dat
Byte size: 22566124235
SHA1: 2e6e5dd20cd9c21960ba94f7265f42c4b6443758
SHA256: d05062052be4dd6fbaeac5f17598e52f7ad545f9f938acefaf7067ecce1d7b5a

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - http://gpgtools.org

iQEcBAEBCgAGBQJT+PcrAAoJEL8a4JGi25zKmj4H/i8xKIXW/xnw5i5Pc+ciUuc2
Zo+KxQd1YGRmJSvGsyTgTqK4IJUBD8s8tIGTtznCJ2/Q3wIdZmoqxkST7r4rqJly
cOyJ/n19K5beYvVekIgHVXFmhdJhZf8KNnyuV+DpHmmLKYyZtvkoY9b3hEBHVtSM
AkjuWyA9IEPeghxBX32JAToXOZGs9vZghNyIMg/fWbGLuv1oX7QCjAAua0NAMbLX
diSvVcLXskTRy38W4whz1DX6ruL8mPRYH5Ko6/cmtVqEkr53aEwYf1XJGHHAz3+J
nEHecxBUbiRhDwMdCVtqzKudXWVDtkgVQmRxrkKTw63NTVMge4eEziE2+7JQgq8=
=p9TY
-----END PGP SIGNATURE-----

<3 Bitcoin
achow101
Moderator
Legendary
*
Offline Offline

Activity: 1946
Merit: 2902


bc1qshxkrpe4arppq89fpzm6c0tpdvx5cfkve2c8kl


View Profile WWW
March 12, 2019, 04:26:16 AM
Merited by LoyceV (2), poordeveloper (2), NeuroticFish (1), ETFbitcoin (1), MagicByt3 (1)
 #9

https://bitcoin.org/bin/block-chain/

Found this torrent link on the bitcoin.org website might be useful.

Note the block height is 317000 for this.  I have no clue how this is maintained or updated someone else might have a better answer for you.
It is not maintained or updated at all anymore. That's the bootstrap.dat file. It will be slower to download that and sync off of it than it will be to let Bitcoin Core do its normal thing to sync.



The bottlenecks you will probably experience will be Disk I/O, CPU, and network speed. As mentioned by others, the fastest way for you would be to sync one node and then copy that to all of the other nodes. You can speed up that initial node's sync time by increase the dbcache if you have a lot of RAM available (>16 GB). Setting dbcache=8000 will yield the best results as the entire database will be able to be held in memory, thus reducing Disk I/O.

StackGambler
Member
**
Offline Offline

Activity: 196
Merit: 66

YouTuber, gambler, and scam-buster.


View Profile
March 12, 2019, 10:18:08 AM
 #10

Sorry, I'm a little new here. I have two questions:

Firstly, to OP. What is the purpose of running 20 nodes? Is it for a bitcoin service, or just to help the network?

Secondly, to everyone. It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.

Thanks guys.

ETFbitcoin
Legendary
*
Offline Offline

Activity: 1820
Merit: 2086

Use SegWit and enjoy lower fees.


View Profile WWW
March 12, 2019, 04:20:09 PM
 #11

Firstly, to OP. What is the purpose of running 20 nodes? Is it for a bitcoin service, or just to help the network?

I'll bet it's not to help the network.

Secondly, to everyone. It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.

Go ahead, few people might be interested despite the risks or knowing it's not much different compared with just running Bitcoin Core.

ranochigo
Legendary
*
Offline Offline

Activity: 1834
Merit: 1195

Back online:)


View Profile WWW
March 12, 2019, 04:24:50 PM
 #12

Secondly, to everyone. It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.
If you want, you can just copy out your whole Bitcoin data directory, then delete your wallet.dat from your copied data directory. Use a torrent program to share the file. You do need quite a number of seeders since the data directory (blockchain) is relatively big and a large number of seeders is required to spread the load evenly. It might be quite hard to get enough people to seed the file especially since it is not to their best interest to keep two blockchains on the same computer.


jackg
Copper Member
Legendary
*
Offline Offline

Activity: 1554
Merit: 1351


https://bit.ly/2FR9nyn - free python tutorials


View Profile
March 12, 2019, 05:50:28 PM
 #13

Secondly, to everyone. It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.

Thanks guys.

You can use utorrent to seed a torrent directory (or other torrenting software) and transmit it to others..

You'd have to put it into a different folder though and potentially compress it if possible to make the most of your bandwidth. If you advertise it remember to post a hash of it so people can verify it (normally these torrents ignore signatures that are normally required for network transmissions so a MITM attack is possible). The 7zip file manager has some nice hashing features.

Subspace_Labs
Newbie
*
Offline Offline

Activity: 3
Merit: 2


View Profile
March 12, 2019, 08:14:48 PM
Merited by achow101 (1), ETFbitcoin (1)
 #14

It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.

Thanks guys.

It sounds like a good idea but it's really not for many people. The problem is that you have to trust the creator of the torrent and trustlessness is one of the main features of Bitcoin. As others have said, downloading it via Bitcoin Core isn't much more resource intensive if you're using a desktop. If you're using some sort of low-power device like a Raspberry Pi, then it will take weeks at best to download and index. If you're using a device like that it's best to download it via Bitcoin Core on a desktop and then copy the data directory over to the Pi. In this case the bottleneck would be CPU and memory. With a fast enough desktop the bottleneck becomes bandwidth.
HCP
Legendary
*
Offline Offline

Activity: 1148
Merit: 1851

<insert witty quote here>


View Profile
March 12, 2019, 08:18:05 PM
 #15

Secondly, to everyone. It seems like having a torrent copy of the blockchain (one that's regularly updated) is something people might be interested in. How would I go about creating one for people to use? I have a synced full node on Windows.
My advice would be "Don't Do it"... you would be simply perpetuating the "myth" that downloading the blockchain via a torrent will decrease the time required to sync... it won't. It is likely to actually be slower than just letting Bitcoin Core do it's thing and sync normally.

Remember... syncing the blockchain is NOT just simply downloading blocks.

For the record, achow101 is a very knowledgeable Bitcoin Core dev... he knows what he is talking about...
It is not maintained or updated at all anymore. That's the bootstrap.dat file. It will be slower to download that and sync off of it than it will be to let Bitcoin Core do its normal thing to sync.



The bottlenecks you will probably experience will be Disk I/O, CPU, and network speed. As mentioned by others, the fastest way for you would be to sync one node and then copy that to all of the other nodes. You can speed up that initial node's sync time by increase the dbcache if you have a lot of RAM available (>16 GB). Setting dbcache=8000 will yield the best results as the entire database will be able to be held in memory, thus reducing Disk I/O.

jackg
Copper Member
Legendary
*
Offline Offline

Activity: 1554
Merit: 1351


https://bit.ly/2FR9nyn - free python tutorials


View Profile
March 12, 2019, 09:43:40 PM
 #16

@hcp the psychology of doing the data download and thenstarting up core is a potential upside though. It's faster to scan the blockchain while you have a physical copy of it and you take out all the downloading to start with. Downloading and syncing via core may be faster but the two individual steps in their own are also faster and more psychologically appealing as it feels like the sync is going faster (from my own experience a few versions ago, it might be out of date entirely).

Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!