Selfish Node: min. bandwdith needed for Bitcoin Core + tons of other data

[tvbcof]

I haven't really looked into VPS pricing.  After the linode hack (and the next one, and the next one, and the most recent one) probably the absolute best decision I have ever made related to Bitcoin is not using VPS.  I always use bare metal.  Either my hardware or rented hardware which is rented completely blank.  I always install the OS right onto bare metal using IPMI. 

The bad news is hardware rental or colocation is likely beyond your limit of what is reasonable.

Yes.  Bare metal is more expensive than I can justify for private dink-around use though I vastly prefer this approach, and I have always been quite anal about imaging such gear myself also.  I'll bet we find that many of the virtualization technologies are exploitable/exploited in the coming years.  OTOH, I'm expecting the same of various chipset components and CPUs.  If BTC goes another 10x then maybe I'll go with my preferred hardware route, but I an going to try my level best to remain a cheapskate no matter what the future holds.

Thx for the input.

[Kluge]

Where did you get the stats from?

I am running a relatively large node (200+ connections, 50 kbps upload, uncapped download) and would be interested in graphing something like that.  I probably will be moving my main node to datacenter and connecting to it via secure RPC calls.  The client locally will just be "dumb wallet".

I'm using http://www.netlimiter.com/ for my monitoring/throttling, and the pictures posted by Kludge look very much alike, I wouldn't be surprised he was using it too.

Yep. Wouldn't mind hearing about alternatives, though. Limiting is nice, but it'd also be nice to give applications (or even ports) "minimum guarantees."

[ineedit]

I have one other connection to bitcoind and that is to the client itself which consumes 23KBs down and 22KBs but to the same computer. Hopefully every took that connection into account when totalling their figures.

What client?  Are you running both bitcoin and bitcoind on the same machine?  If so why?

I am running Armoury on Win 7. If I use Bitcoin-QT then I get a similar network usage pattern.

[Kluge]

Updated image. Complete Nov. data and partial Dec. I now use "d" instead of "QT," so won't have "unified" data going forward.

I realized my flaw in this after wondering why the numbers were so freakin' consistent month-over-month. The "real" minimum is significantly lower than 9.5kB/s TCP. QT still downloads transactions to add to queue and relay, however. Since this transaction queue exchange appears to take significantly more bandwidth than block exchange, the download bandwidth limit is almost always reached. QT possibly prioritizes bandwidth, giving top priority to block exchanges instead of transaction exchanges (clever behavior if true).

I'll see if I can find the "truer" minimum. Will let it run in 24h spans at different limits and find the lowest possible without generating a block backlog. Stats in OP are fairly useless with that in mind.

[Kluge]

Good news! After some testing, it appears QT DOES prioritize block bandwidth over tx bandwidth (or blocks are just so relatively small, they're still downloaded within an acceptable time limit even when fully sharing bandwidth with the transaction queue process). This means QT can run at a MUCH lower throttle than I thought. I'm currently testing with a 2kb/s download throttle and will eventually start bumping down the upload throttle.

Because I know QT can function relatively normally on a 3kb/s leash, it probably can consume less than one gigabyte in data per month while still being able to operate like a full node (with caveats mentioned in OP).

[Kouye]

Was about to send a PM to kludge, but since I might not be the only one to have adopted netlimiter after his initial post...
I had 2 days left of free trial when I checked today.
Don't forget to buy a life-time licence if you find it useful and downloaded it after seeing this thread. (around 24-30€, depending on VAT)

[Kluge]

Was about to send a PM to kludge, but since I might not be the only one to have adopted netlimiter after his initial post...
I had 2 days left of free trial when I checked today.
Don't forget to buy

Oh. That's a good reminder. I've been using a pirated copy for ages, but it's saved me enough frustration where I'm going to go ahead and purchase at least a couple licenses.

[MoonShadow]

I would like to do this for my client/s on an Imac, any suggestions on how I should go about throttling the bandwidth?

[Kluge]

I would like to do this for my client/s on an Imac, any suggestions on how I should go about throttling the bandwidth?

As far as I can see, there isn't anything quite like NetLimiter for Mac. I've never used it, but WaterRoof has *some* of the features. It doesn't appear to support throttling by application, though, only IP address & port, which is kind of useless. They accept BTC, though. http://www.hanynet.com/waterroof/index.html If you set Bitcoin to use a specific, obscure port, maybe you can throttle traffic in and out of that port, so you effectively only throttle QT.

[Kouye]

Was about to send a PM to kludge, but since I might not be the only one to have adopted netlimiter after his initial post...
I had 2 days left of free trial when I checked today.
Don't forget to buy

My post was not as violent as that quote, just a "mis-click confirm" quote, fixed like 2 minutes later. But the idea is here.
I think this should be obvious options in QT.

Max upload BW
Max download BW

Was about to send a PM to kludge, but since I might not be the only one to have adopted netlimiter after his initial post...
I had 2 days left of free trial when I checked today.
Don't forget to buy a life-time licence if you find it useful and downloaded it after seeing this thread. (around 24-30€, depending on VAT)

And yes, please pay for what helps you.

[Kluge]

Update + ISP sync times and costs for no particular reason (bored, I guess):

NetLimiter 4 doesn't currently support stats, but will soon-ish. I won't have any new data until it's implemented.

Core currently does not require more than 5kb/s down once sync'd, possibly a little less, though I see no reason to risk going lower. Therefor, dial-up CAN (theoretically) still support Core running IF physically given an up-to-date blockchain and the connection is left on for the vast majority of the day without interruption. Satellite is a little more sketchy due to the increased overhead from high latency and dropped connections... I can't get experimental data for that (well, I can, but I'm not paying $80/mo for shit service just to see what happens ).

Since I personally get my connection from a 4G network (after buying a fairly expensive amplifier and antenna made directional with beer cans ), I'd like to organize some quick stats on US Internet speeds in comparison to sync time. Fun fact: 15% of Americans have no Internet connection.

Givens:
Current blockchain size: ~18.9GB (19818086.4kb)
One kb/s translates to ~0.00000095GB/s, which translates to .003433GB/hr. Therefor, the calculation to determine hours to download a 18.9GB file for a 7kb/s connection is (18.9/(.003433*7)).
Stay-sync'd bandwidth req.: 3kb/s
A month has roughly 2,592,000 seconds in it.
Pricing/speed data for DSL & cable assumes Jackson, MI location, FiOS assumes Chicago, 3G/4G data is national average for carrier as reported by opensignals.com
3G/4G assumes "pirate tether"
Promotional rates are not considered, but installation and ETF fees are ignored
Network reliability is not factored in
Minimum cost/gigabyte/mo assumes you use every byte you possibly can (unreasonably unlikely EXCEPT when letting Core sync 24/7) and achieve advertised speeds (unlikely)
(I should've thrown this in a spreadsheet, but wasn't intending to analyze this much info when I started)

Dial-up
Total US usage: 3%
Total US coverage: 97-99%? (unknown)
Examined: NetZero "Basic"
Cap: None
Price/mo: $10
Speed: 7kb/s
Max download capacity per month: 17.3GB
Min. cost/gigabyte/mo: $.578
Bitcoin min. sync time: ~32.77 days
Bitcoin theoretical sync cost: ~$10.92
Bitcoin theoretical stay-sync'd cost: ~$4.29/mo

Satellite
Total US usage:
Total US coverage: 100% (except for caves and enormous lead buildings)
Examined: Hughesnet "Power"
Cap: 20GB (not a typo)
Price/mo: $60 (not a typo)
Speed: 1,280kb/s
Max download capacity per month: 20GB
Min. cost/gigabyte/mo: $3
Bitcoin min. sync time: ~4.3 hours (in reality, probably over two billing cycles)
Bitcoin theoretical sync cost: $56.70
Bitcoin theoretical stay-sync'd cost: $22.24/mo

DSL
Total US usage: ~18-30%
Total US coverage: ~70%
Examined: AT&T "Pro"
Cap: 250GB
Price/mo: $40
Speed: 375kb/s
Max download capacity per month: 250GB
Min. cost/gigabyte/mo: $.16
Bitcoin min. sync time: ~14.68 hours
Bitcoin theoretical sync cost: ~$3.02
Bitcoin theoretical stay-sync'd cost: ~$.32/mo

3G
Total US usage:
Total US (home) coverage: ~60-80% incl. all carriers?
Examined: Sprint "Unlimited"
Cap: None
Price/mo: $80
Speed: 900kb/s (maybe they meant mbps??)
Max download capacity per month: 2,224.73GB
Min. cost/gigabyte/mo: $.036
Bitcoin min. sync time: ~6.12 hours
Bitcoin theoretical sync cost: ~$.68
Bitcoin theoretical stay-sync'd cost: insignificant

4G
Total US usage:
Total US (home) coverage: ~25-40% incl. all carriers?
Examined: Sprint "Unlimited"
Cap: None
Price/mo: $80
Speed: 5,100kb/s
Max download capacity per month: 12,606.81GB
Min. cost/gigabyte/mo: $.00635
Bitcoin min. sync time: ~1.08 hours
Bitcoin theoretical sync cost: insignificant
Bitcoin theoretical stay-sync'd cost: insignificant

WISP
Varies wildly, not included

Cable
Total US usage: ~31-50%
Total US coverage: ~70%
Examined: Comcast "Performance" (and fuck them!)
Cap: 300GB
Price/mo: $65
Speed: 3,200kb/s
Max download capacity per month: 300GB
Min. cost/gigabyte/mo: $.21
Bitcoin min. sync time: ~1.72 hours
Bitcoin theoretical sync cost: ~$3.78
Bitcoin theoretical stay-sync'd cost: inisignificant

FiOS
Total US usage: ~8%
Total US coverage: 10-15%??
Examined: Verizon " [image of a lightning bolt]"
Cap: None (don't you dare try to run a commercial server out of your house, though!)
Price/mo: $350
Speed: 64,000kb/s
Max download capacity per month: Exactly one fuck-load. (AKA 158,192.64GB)
Min. cost/gigabyte/mo: $.00553
Bitcoin min. sync time: ~5.16 minutes
Bitcoin theoretical sync cost: insignificant
Bitcoin theoretical stay-sync'd cost: inisignificant

US Average
Speed: 1,113.6kb/s
Bitcoin min. sync time: 4.94 hours

ETA: fixed FiOS max down/mo capacity

[2112]

Satellite
Total US usage:
Total US coverage: 100% (except for caves and enormous lead buildings)

Technically satellite would be a nearly perfect medium for Bitcoin. It would however take cooperation from both the provider and the core development team.

Provider would need to make a broadcast/multicast channel for Bitcoin blocks and transactions that all subscribers could receive simultaneously. This would produce tremendous savings both in bandwidth and in the latency. I can't really talk about cost, because I'm completely out of date on this aspect. Sending transactions could be made over dialup or cellular SMS or any other low bandwidth medium like postcards with the QR code mailed to the provider, or even carrier pigeons (with microSD cards) lent out by the provider. Edit: obviously normal satellite uplink is also acceptable, but not really required, because of the extremely low required uplink bandwidth: only to distribute own transactions generated by that node.

Core development team would have to adopt the Bitcoin over UDP, which I believe was prototyped by jgarzik in his picocoin implementation. This is more of a political issue than a technical one.

Edit: obviously this would centralize Bitcoin in the same way as pools centralize it: we would have to make sure that there are several satellite Bitcoin providers with overlapping footprints.

[Kluge]

Satellite
Total US usage:
Total US coverage: 100% (except for caves and enormous lead buildings)

Technically satellite would be a nearly perfect medium for Bitcoin. It would however take cooperation from both the provider and the core development team.

Provider would need to make a broadcast/multicast channel for Bitcoin blocks and transactions that all subscribers could receive simultaneously. This would produce tremendous savings both in bandwidth and in the latency. I can't really talk about cost, because I'm completely out of date on this aspect. Sending transactions could be made over dialup or cellular SMS or any other low bandwidth medium like postcards with the QR code mailed to the provider, or even carrier pigeons (with microSD cards) lent out by the provider. Edit: obviously normal satellite uplink is also acceptable, but not really required, because of the extremely low required uplink bandwidth: only to distribute own transactions generated by that node.

Core development team would have to adopt the Bitcoin over UDP, which I believe was prototyped by jgarzik in his picocoin implementation. This is more of a political issue than a technical one.

Edit: obviously this would centralize Bitcoin in the same way as pools centralize it: we would have to make sure that there are several satellite Bitcoin providers with overlapping footprints.

I remember seeing someone actually talking about launching a "bitcoin satellite" (maybe it was you). Cost is a crapshoot which could go into the tens of $millions annually if the company's feeling mean/"smart"... it's not like there are many companies you can talk to, so I'd be surprised if they even talked to anyone about it without first being bribed with a suitcase of money.

I'm unsure if centralization's really a worry... I've never heard of issues with any coins' bootstrap torrents when they're signed by the devs (though BTC's obviously much smaller than where it could go in the moon-future). Incidentally, I recently talked with a major pirate scene group about what they do to protect against impersonated scene releases, and they say it's really never been an issue even though they don't even sign what they push and actual publication of torrent files and magnet links are usually distributed via third-party sites like ThePirateBay. The only problem they have, and it's really just an "eyeroll problem," is when other people copy their work-arounds to DRM and claim it as their own. That's not "money," but an infected impersonation under the right circumstances could result in the creation of a massive infection of hundreds of thousands of computers, which can certainly cause $millions in damages. I don't think we should just wait for a problem to arise, but I don't see much issue trusting a blockchain maybe three devs have had verified against the "current majority blockchain" and sign; I think that's enough. I think it's either Jeff or Greg who maintain the current bootstrap torrent, so that person'd obviously be a good pick...

ETA: Wait... how are people supposed to pick the satellite signal up?

[tvbcof]

Just a note on satellite from a user:

I use 'Exede' and it is surprisingly usable.  I activate bitcoind from time to time when I need to access a cold storage wallet.  So far I only need to sync to the age of my cold storage wallet in order to spend the funds which means only a tiny fraction of the entire blockchain.  This is good because more than once the database has become corrupt and I needed to start the sync over.  I don't use UPnP and de-activate it in my router for security reasons (and didn't compile it into my binary) and I've not set up port forwarding.  In short, I've no idea if the latency is impacting my ability to serve blocks.

I get 15G/mo for $80.  That is double the standard 7G/mo.

I would not consider satellite to be at all decent for reliable peer-2-peer in an adversarial environment.  I believe that all satellites are dual-use civilian and military and military (including state sponsored surveillance needs) take precedence.  (I could be wrong.)  In any event, I have zero confidence that the satellite providers would do anything which is remotely adverse to the demands of the government, and if there was a mandate to interfere with peer-2-peer traffic of any nature I expect that the satellite operators would comply with enthusiasm.

It is also utterly absurd to hope that independent entities could control satellites.  Space is highly regulated and increasingly militarized.  The U.S., as a highly militarized and sole super-power relies heavily on space for sustained dominance in a variety of areas, and opposing forces (e.g., China) are putting significant efforts into systems of attacking space-based communications systems if need be.

To me it is a no-brainer to focus on devising a defense of crypto-currency value stores as concentric spheres of inward-increasingly robust solutions.  At the core would be a high latency no-frills system which would be very difficult to successfully attack on an extended basis.  Other more user-friendly systems could use it as a foundation.  I'd like to see Bitcoin evolve to be the inner core, but evolution seems to be going in the other direction to my chagrin.  OTOH, the idea of using Bitcoin proper as a 'reserve' and 'balancing' currency does seem to be gaining some traction of late.

[Kluge]

blk compression testing is done "good enough." Thanks to deslok for hassling me throughout. GoogleDocs has decided to suddenly stop publishing notes in their latest mandatory upgrade of spreadsheets because they're assholes, so I'll have to manually list notes here...

https://docs.google.com/spreadsheets/d/1NmgCVnLI8WBEdzjQvMhoe64LGXp23XiMGZU8Syz0lAI/pubhtml

Specs -
Phenom II x4 (965) 3.4GHz
16GB DDR3 RAM
7200RPM HDD
Win7 x64

blk00000
Nanozip 1 - .09a x86: optimum2, 512mb RAM allocated
Nanozip 2 - .09a x86: CM, 512mb RAM allocated
Nanozip 3 - .09a x64: optimum2, 4096mb RAM allocated
7zip 1 - 9.2 x64: LZMA2, ultra, 1024MB dic. size (word size 273), 3 threads
7zip 2 - 9.2 x64: LZMA2, ultra, 512MB dic. size (word size 192), 4 threads
7zip 3 - 9.2 x64: LZMA, ultra, 1024MB dic. size (word size 273), 2 threads
7zip 4 - 9.2 x64: BZip2, ultra, 900KB dic. size, 4 threads
7zip exe 1 - 9.2 x64: LZMA2, ultra, 512MB dic. size (word size 192), 4 threads, put into SFX

blk00000 + blk00001
7zip exe 2 - 9.2 x64: LZMA2, ultra, 512MB dic. size (word size 192), 4 threads, put into SFX

entire "blocks" folder
7zip 1 - 9.2 x64: LZMA2, ultra, 512MB dic. size (word size 192), 4 threads
7zip 2 - 9.2 x64: LZMA2, ultra, 1024MB dic. size (word size 273), 3 threads
nanozip 1 - .09a x64: optimum2, 7500mb RAM allocated


Overall, nothing particularly exciting discovered. 40-45% compression is to be expected (I never saw 45%, but I'm kind'a dumm), which is pretty critical for initial sync (for anyone using cable or worse as ISP) and makes a solid case for compressing bootstrap files, but not very useful outside. Unsure why nanozip's "optimum2" algorithm did so poorly with the extra blocks when they contain so much repeated info.

I'm working on that last 7zip archive, but it shouldn't be thrilling. With the test I did do for the "blocks" folder, assuming you have a 3200kb/s cable connection, even after a ~13 1/2 minute decompression time, you, overall, save about a half-hour in time, don't have to use up unnecessary bandwidth from your monthly allotment, and stress the infrastructure a solid deal less. With DSL, all those benefits are greater and you'd be saving a good many hours. With satellite ISP, you'd be able to download the entire blockchain in one billing cycle without buying ultra-expensive pay-as-you-go bandwidth. In "US average" conditions, the benefits are significant, too, and it'd also save hours of time.

Assuming an "average GB cost" of $.10 and 100,000 new full nodes per month, the monthly cost savings of compressing blocks for the initial sync would be right around $85k, or ~$1M annually. That's a pretty BS statistic, so fwiw. Additional RAM, CPU, and HDD load are all too minimal to really make a difference compared to that (and you may get a net save in electricity costs by not needing to have the PC running as long just to download the blockchain).

[Kluge]

Assuming an "average GB cost" of $.10 and 100,000 new full nodes per month, the monthly cost savings of compressing blocks for the initial sync would be right around $85k, or ~$1M annually. That's a pretty BS statistic, so fwiw. Additional RAM, CPU, and HDD load are all too minimal to really make a difference compared to that (and you may get a net save in electricity costs by not needing to have the PC running as long just to download the blockchain).

Here's a more useful tidbit from the info-gathering: spending an hour compressing the blockchain bootstrap torrent once would, assuming an average download speed of 1,100kb/s, with 100k new nodes per month, would save ~2.4 million hours globally in sync times every year. (rather, 274 "years" would be saved every year)

[Meuh6879]

on personal (and permanente) connexion of bitcoin-core ... i set 4 connexions max. = 5ko/s of UPLOAD
on p2pool server support with bitcoin-core, the minimal connexion must be set to 12. = 10-20ko/s of UPLOAD

[2112]

on personal (and permanente) connexion of bitcoin-core ... i set 4 connexions max. = 5ko/s of UPLOAD
on p2pool server support with bitcoin-core, the minimal connexion must be set to 12. = 10-20ko/s of UPLOAD

ko?

French...

Ah, ko == kilooctets.

Merci bien.