waspoza
|
|
October 13, 2012, 11:37:19 PM |
|
Seeding 24/h on my server siting on 100 Mbps link.
|
|
|
|
runeks
Legendary
Offline
Activity: 980
Merit: 1008
|
|
October 14, 2012, 01:52:51 AM |
|
I wonder if it could be generated from their existing blockchain (is it basically the same file?) which will presumably be pretty up-to-date.
Yes, this is open data and an open file format. Seeders may independently generate a byte-for-byte identical bootstrap.dat by running https://github.com/jgarzik/pynode/blob/master/mkbootstrap.pyEach time the blockchain torrent is updated, seeders may run that script to guarantee they have 100% of bootstrap.dat immediately. A nice, decentralized solution How do I use that script? Should some of these variables reference my .bitcoin directory somehow? The .bitcoin directory is for a different app. pynode is a full bitcoin client, separate from bitcoind. The script mkbootstrap.py requires access to the pynode database, after you have downloaded all the blocks. Sadly you do need to be a bit of a programmer to generate a bootstrap.dat file. I think I've got pynode synchronized with the network now. The blocks.dat file is 3.5 GB after letting it connect to a local instance of bitcoin-qt and waiting a bit. mkbootstrap.py is running now. I guess loading the config file should be separated out into a module that can be loaded by mkbootstrap.py. But I'm not sure how to do that elegantly (I'd just create a function that returns the settings dict given the path to the config file). EDIT: Looks like it succeeded. I'm now seeding the bootstrap.dat file as well.
|
|
|
|
jgarzik (OP)
Legendary
Offline
Activity: 1596
Merit: 1099
|
|
October 14, 2012, 07:10:43 PM |
|
Version 0.7.1, which just entered testing, includes a new feature: If the file "bootstrap.dat" is found in the bitcoin data directory, it will validate and import all blockchain data found in that file. So what is the difference vs this procedure? No need to use -loadblock and faster download, given there are many fast seeders? The difference between 0.7 and 0.7.1 is that 0.7.1 automatically runs "-loadblock=bootstrap.dat" at startup. The torrent will probably be a faster download... but if you have an ultrafast network peer, the regular download will be just as fast. This torrent is just adding option for users; it is not the New Official Recommends Means for getting the blockchain. As the OP emphasizes, this is an experiment.
|
Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own. Visit bloq.com / metronome.io Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
|
|
|
Digigami
|
|
October 14, 2012, 07:15:15 PM |
|
If the results of this experiment are positive, what are the chances we could modify the client so that it downloads the initial blockchain using this method instead of the current system? Is that a long term goal of this?
|
|
|
|
jgarzik (OP)
Legendary
Offline
Activity: 1596
Merit: 1099
|
|
October 14, 2012, 07:23:36 PM |
|
If the results of this experiment are positive, what are the chances we could modify the client so that it downloads the initial blockchain using this method instead of the current system? Is that a long term goal of this?
Not at this time. It is possible that somebody might create a bitcoin-firstrun.exe application, separate from bitcoin, that torrents the blockchain. For most users the initial blockchain download will be a one-time event, so there is little interest in directly adding bittorrent code to the bitcoin client. The import-bootstrap.dat feature was added so that bitcoin does not have to care about the source of the data. As long as you have the file, it will import it. Maybe you downloaded the file via torrent... or maybe HTTP. The point is, from the bitcoin client's perspective, it is agnostic to the download method.
|
Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own. Visit bloq.com / metronome.io Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
|
|
|
jgarzik (OP)
Legendary
Offline
Activity: 1596
Merit: 1099
|
|
October 14, 2012, 08:47:53 PM |
|
Torrent added to PirateBay! I hope you here or my friend - I used his abandoned account - will be OK with this move. Once PB picks some torrent, it usualy got spread to numerous other torrent sites in matter of few days, so expect (or not?) some more down/uploaders and/or participants here. In general, spreading the torrent far and wide is fine. But for trackers or accounts of friends, it is nice to ask [the tracker site / friend] first
|
Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own. Visit bloq.com / metronome.io Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
|
|
|
jgarzik (OP)
Legendary
Offline
Activity: 1596
Merit: 1099
|
|
October 14, 2012, 09:51:37 PM |
|
Torrent added to PirateBay! I hope you here or my friend - I used his abandoned account - will be OK with this move. Once PB picks some torrent, it usualy got spread to numerous other torrent sites in matter of few days, so expect (or not?) some more down/uploaders and/or participants here. In general, spreading the torrent far and wide is fine. But for trackers or accounts of friends, it is nice to ask [the tracker site / friend] first I haven't added any trackers to torrent. I just uploaded .torrent you linked at starting post, from my HDD, and copy/pasted some info. Yeah, post the .torrent file far and wide.
|
Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own. Visit bloq.com / metronome.io Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
|
|
|
K1773R
Legendary
Offline
Activity: 1792
Merit: 1008
/dev/null
|
|
October 17, 2012, 02:36:12 PM |
|
Off-topic, sort of: There's incredible amount of Anti-P2P activity lately. By the time I finished downloading file, 724 IP addresses known to be used by organizations and companies openly working against P2P were detected on DHT, and 720 of them tried to connect to my client, LOL! I was downloading and now seed just this file. Really, those people are very fast - and determined - to find out what's new on P2P. Level 1 filters those mentioned above: yea sure, ipv4 adress spaces higher than 255! lmao EDIT: or is the number a count of ipv4 addresses in this section? even touhg, ipfilter is a fine thing, but not really accurate at all. @jgarzik: need more seeds?
|
[GPG Public Key]BTC/DVC/TRC/FRC: 1 K1773RbXRZVRQSSXe9N6N2MUFERvrdu6y ANC/XPM A K1773RTmRKtvbKBCrUu95UQg5iegrqyeA NMC: N K1773Rzv8b4ugmCgX789PbjewA9fL9Dy1 LTC: L Ki773RBuPepQH8E6Zb1ponoCvgbU7hHmd EMC: E K1773RxUes1HX1YAGMZ1xVYBBRUCqfDoF BQC: b K1773R1APJz4yTgRkmdKQhjhiMyQpJgfN
|
|
|
K1773R
Legendary
Offline
Activity: 1792
Merit: 1008
/dev/null
|
|
October 17, 2012, 03:27:28 PM |
|
or is the number a count of ipv4 addresses in this section?
Yes, it shows number of different IP addresses for each list. You can check what exactly each of them is filtering (or allowing) here. well then, sry 4 that are you seeding from home? if so i can understand avoiding the 720 connections
|
[GPG Public Key]BTC/DVC/TRC/FRC: 1 K1773RbXRZVRQSSXe9N6N2MUFERvrdu6y ANC/XPM A K1773RTmRKtvbKBCrUu95UQg5iegrqyeA NMC: N K1773Rzv8b4ugmCgX789PbjewA9fL9Dy1 LTC: L Ki773RBuPepQH8E6Zb1ponoCvgbU7hHmd EMC: E K1773RxUes1HX1YAGMZ1xVYBBRUCqfDoF BQC: b K1773R1APJz4yTgRkmdKQhjhiMyQpJgfN
|
|
|
Gabi
Legendary
Offline
Activity: 1148
Merit: 1008
If you want to walk on water, get out of the boat
|
|
October 18, 2012, 12:00:28 PM |
|
Sadly you do need to be a bit of a programmer to generate a bootstrap.dat file. Of course gabi rage: is it so hard to have a program that you just clic it, tell it where the bitcoin folder with the blockchain is and then you press EXTRACT and it create bootstrap.dat?
|
|
|
|
K1773R
Legendary
Offline
Activity: 1792
Merit: 1008
/dev/null
|
|
October 18, 2012, 02:44:41 PM |
|
Sadly you do need to be a bit of a programmer to generate a bootstrap.dat file. Of course gabi rage: is it so hard to have a program that you just clic it, tell it where the bitcoin folder with the blockchain is and then you press EXTRACT and it create bootstrap.dat? you forgot that 95% of humanity nowadays is retarded, they will never be able to!
|
[GPG Public Key]BTC/DVC/TRC/FRC: 1 K1773RbXRZVRQSSXe9N6N2MUFERvrdu6y ANC/XPM A K1773RTmRKtvbKBCrUu95UQg5iegrqyeA NMC: N K1773Rzv8b4ugmCgX789PbjewA9fL9Dy1 LTC: L Ki773RBuPepQH8E6Zb1ponoCvgbU7hHmd EMC: E K1773RxUes1HX1YAGMZ1xVYBBRUCqfDoF BQC: b K1773R1APJz4yTgRkmdKQhjhiMyQpJgfN
|
|
|
jgarzik (OP)
Legendary
Offline
Activity: 1596
Merit: 1099
|
|
October 18, 2012, 03:03:22 PM |
|
Sadly you do need to be a bit of a programmer to generate a bootstrap.dat file. Of course gabi rage: is it so hard to have a program that you just clic it, tell it where the bitcoin folder with the blockchain is and then you press EXTRACT and it create bootstrap.dat? That's what we have here. "a program" is called a bittorrent client. Tell it where the bitcoin folder with the blockchain is, and then you press download.
|
Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own. Visit bloq.com / metronome.io Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
|
|
|
Gabi
Legendary
Offline
Activity: 1148
Merit: 1008
If you want to walk on water, get out of the boat
|
|
October 19, 2012, 07:20:43 PM |
|
Oh sorry, my post wasn't clear, i was refering to this: I wonder if it could be generated from their existing blockchain (is it basically the same file?) which will presumably be pretty up-to-date. Yes i know, downloading it is very simple and straightforward, what i was speaking about was about creating the file from my blockchain, not about downloading the file via bittorrent.
|
|
|
|
Syke
Legendary
Offline
Activity: 3878
Merit: 1193
|
|
October 20, 2012, 11:18:58 AM |
|
this data is the precisely the same data you see on the P2P network.
Then how is this different than connecting to high-speed bitcoin peers?
|
Buy & Hold
|
|
|
kjj
Legendary
Offline
Activity: 1302
Merit: 1026
|
|
October 20, 2012, 01:12:25 PM |
|
this data is the precisely the same data you see on the P2P network.
Then how is this different than connecting to high-speed bitcoin peers? Blocks are oddly sized, and their locations within the block files are pretty much arbitrary. If you don't have enough RAM to cache the whole thing, serving them as blocks one by one will thrash the hell out of your disks. Also, the way the stock client does the download now, it asks the first node it connects to for the whole chain, block by block. Bittorrent, on the other hand, doesn't understand the data at all, so it serves it in chunks that are always the same size, which are always aligned with the filesystem blocks. It also can manage multiple connections, grabbing different chunks from here and there. The result is that torrents are much more efficient at bulk transfer. They are faster for the client, and less stressful for the server. But, for various reasons, we don't like that we have to use an outside distribution method. The long term goal is to improve the software to make the initial block download faster and smarter. The bootstrap and torrent are just a temporary thing to lighten the load while work continues.
|
17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8 I routinely ignore posters with paid advertising in their sigs. You should too.
|
|
|
VEscudero
Donator
Sr. Member
Offline
Activity: 335
Merit: 250
Bitcoin, Ripple & Blockchain pioneer
|
|
October 20, 2012, 04:37:37 PM |
|
As a workaround it seems ok for me, so I'm seeding this file right now for helping others bootstrap.
|
|
|
|
Syke
Legendary
Offline
Activity: 3878
Merit: 1193
|
|
October 20, 2012, 11:54:24 PM |
|
Blocks are oddly sized
Makes sense. and their locations within the block files are pretty much arbitrary.
Even for someone starting from block 0? Also, the way the stock client does the download now, it asks the first node it connects to for the whole chain, block by block.
That sounds quite un-optimal. Bittorrent ... can manage multiple connections, grabbing different chunks from here and there.
We could update the Bitcoin client to grab blocks from multiple peers, right? With a few more patches, downloading the chain from peers should approach the speed of a torrent.
|
Buy & Hold
|
|
|
Pieter Wuille
Legendary
Offline
Activity: 1072
Merit: 1181
|
|
October 20, 2012, 11:57:38 PM |
|
The way the client currently downloads blocks from peers is hardly more than a hack that somehow frequently manages to work.
We could add even more hacks on top of it to improve it (and it wouldn't take much to improve it a lot), but the right solution in my opinion is doing it right from the ground up: synchronize the headers first, validate those, and then ask your peers to fill in the data for the blocks which you already know in advance are part of the best chain.
The just-merged rewrite of the database layout and validation engine should support such a mode of operation, but someone still has to implement it.
|
I do Bitcoin stuff.
|
|
|
kjj
Legendary
Offline
Activity: 1302
Merit: 1026
|
|
October 21, 2012, 02:38:01 AM |
|
Blocks are oddly sized
Makes sense. and their locations within the block files are pretty much arbitrary.
Even for someone starting from block 0? Yup, it is a consequence of the odd block sizes. Reading a single block may involve several disk reads. And getting the file offset to read will also take several reads in the index file.
|
17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8 I routinely ignore posters with paid advertising in their sigs. You should too.
|
|
|
TheButterZone
Legendary
Offline
Activity: 3038
Merit: 1032
RIP Mommy
|
|
October 21, 2012, 04:38:02 AM |
|
Ooh. Even though I don't use the client, I'm getting this to seed indefinitely (at least when my computer is on and BT is open). Except there's only 1 seed now, no peers. Will it be worth the sectors on my hard drive to keep this going?
|
Saying that you don't trust someone because of their behavior is completely valid.
|
|
|
|