Bitcoin Forum

Estimated minimum TCP throttle: 3kb/s (likely lower, will continue testing)
Estimated minimum UDP throttle: 1kb/s (likely closer to .1kb/s)
Caveats: Because throttling will prevent you from truly acting as a full node and processing the transaction queue, you will likely not see transactions sent to addresses you control until it's included in a block. If you receive a low-priority transaction, it's possible your client won't notice it for a looong time (a 10kb/s throttle appears to keep up fairly well with the transaction queue). There may also be a delay when you send a transaction out, though temporarily unthrottling the connection would obviously fix this. This minimum WILL change due both to changes to Bitcoin's rules and the increasing blockchain size.


I figured someone may be interested in how much bandwidth running QT actually consumes per month in a scenario. It's right around 3GB down, 1GB up, assuming network is caught up AND you have severely throttled upload bandwidth allowed to QT (in these stats, QT has always been throttled to 1kb/s UL). December 10, '13 update: That statement is false. QT downloads transactions for queue as well as blocks. By throttling (for these stats, I set DL limit to 9.5kB/s) above the "true minimum" required, bandwidth is still going toward downloading transactions, which is non-essential. This makes these stats useless except to know QT will currently function (without accumulating a block backlog) with less than 9.5kB/s or ~3GB/mo allotted to it.

QT consumption stats:
https://i.imgur.com/WrpMe87.jpg
Grabbed on December 10, 2013

The stats file gets enormous, so I semi-regularly delete it and don't have older data - sorry.

For the sake of comparison (I'm not trying to make any kind of point with this), here is "real" consumption from a well-used Skype client without use of video/audio streaming:
https://i.imgur.com/5eVSOjI.jpg

ETA: Stats on Chrome are incomplete for some reason... so here're the stats for the download manager I use (every large file I download off the web):
https://i.imgur.com/vp0Og3I.jpg

Other data:
Sync times and theoretical "cost to sync": https://bitcointalk.org/index.php?topic=334778.msg6306363#msg6306363
Blockchain compression (not pruning!) methods compared: https://bitcointalk.org/index.php?topic=334778.msg6352393#msg6352393

Thank you very much for this analysis.

I stopped running nodes in mid-2012 due to excessive costs of bandwidth in rural Australia.

Thx.  Trying to decide whether to start my client back up on my satellite connection ($80 for 15GB/mo.)  Your info is valuable.

It is probably worth note that half a decade ago, Comcast (at least) was traffic shaping for economic reasons.  Fucking with P2P traffic.  They stopped because of economic, political, and legal pressures and not because of any particular technical challenges.

This is a key point to consider because the political and legal challenges could not only vanish but be completely reversed.  And it could happen at the flip of a switch, and probably across all providers who do not have actual ownership of part of the internet backbone.  That means practically everyone and every business could be impacted.  Those organizations who do have actual fiber and core routers can be brought under other pressures.

This is why the issue of transaction rate (block size) is so critical to me.  Smaller streams could potentially find nooks and crannies to wiggle through and keep the Bitcoin network functional in a more or less free form.  Larger streams feeding more sophisticated deployments (e.g., 'supernodes') could have a great deal of difficulty operating in a hostile environment.  Or at least operating free of coercion to honor various type of taint and privacy frameworks.  This is what I would call the 'WHG attack' after Waters, Hearn, and Guo.

That is surprising. I have my Bitcoin-Qt node running 24/7 at home. It states around 60 active connections, 8 of which are outgoing connections and the rest are incoming. For that reason, I would have thought that the up bandwidth would have been noticeably higher than the down bandwidth...

Not that I care that much. I have a flat rate monthly contract  :)

In most cases, upload bandwidth would be dramatically higher. In my case (made a little note in OP), I always have QT throttled to 1kb/s because I can't handle more and still be able to browse online. I also throttle download speed to 9.5kb/s, but that's more than enough to stay sync'd to the blockchain. Download speed throttle is only to even out strain of receiving new blocks so I don't notice it as much.

bump for curiosity

Thanks for the data, Kludge.
I'm curious about the throttle, though, what do you think it would look without it?
Anyway, I'll monitor bandwidth of QT client too, from now on...

I'd guess it'd be dependent on the total amount of bandwidth available unless you have a lot of slow peers (like me) connected, where they can't "eat" as much as they could be sent. Most wired connections are asymmetrical, so the user probably wouldn't notice if QT consumed the vast majority of their upload bandwidth unless they were uploading large files.

I'd guesstimate if you had a 756kbps up connection, many peers, and left QT on 24/7, you'd probably be looking at ~100-150gb/mo.

I don't have QT runnning 7/24, but I'll be able to do at least batches of weekly stats, will update.

20 minutes, almost 300MB upload. :o
Great clue to explain family complaining about sluggish internet...
I'll still run QT "unleashed" for some time, for truth, but I'll throttle it hard too, after. If not stickied, this should at least come up as a warning when you install QT.

After 24 hours, upload consumed by QT client has reached 22GB.
So we're talking about 650GB per month. :o
Guess it's time to throttle it.  ;D

I don't get your Skype comparison, either to mislead or you simply didn't read it properly, the Skype graph shows consumption in MB, not GB, so its still way smaller than the Bitcoin QT.

Where did you get the stats from?

I am running a relatively large node (200+ connections, 50 kbps upload, uncapped download) and would be interested in graphing something like that.  I probably will be moving my main node to datacenter and connecting to it via secure RPC calls.  The client locally will just be "dumb wallet".

mine has around 90 connections and sends around 10-15gbs out per day.

He was just using Skype as a common reference point to show how much more QT uses than many people may think.

I'm using http://www.netlimiter.com/ for my monitoring/throttling, and the pictures posted by Kludge look very much alike, I wouldn't be surprised he was using it too.
You can give it a try if your node is on a windows box, but it installs network drivers, so there's always a risk, be sure to backup before.

Interesting!

I am running bitcoind but only have 9 peers connected and they are consuming in total 1KBs down & 500Bs up. My total would be about 2.6GBs per month down, if I have my math right ;)


I have one other connection to bitcoind and that is to the client itself which consumes 23KBs down and 22KBs but to the same computer. Hopefully every took that connection into account when totalling their figures.

What client?  Are you running both bitcoin and bitcoind on the same machine?  If so why?

Anyone who has put significant time into looking around for VPC services I'd be interested to hear from.  Some of the features I would like:

 - jurisdictionally and physically located to be adverse to and likely belligerent towards fine grained analysis by 5-eyes.

 - allows me to run my own OS image (preferably a BSD variant.)

 - somewhat reasonable in pricing.

I haven't really looked into VPS pricing.  After the linode hack (and the next one, and the next one, and the most recent one) probably the absolute best decision I have ever made related to Bitcoin is not using VPS.  I always use bare metal.  Either my own hardware (colocation) or renting the datacenter's bare metal (dedicated server rental).  I always personally install the OS right onto bare metal using IPMI.   No backdoors, no superuser accounts, no password reset possibly by the hosting provider.

The bad news is hardware rental or colocation is likely beyond your limit of what is reasonable.

Yes.  Bare metal is more expensive than I can justify for private dink-around use though I vastly prefer this approach, and I have always been quite anal about imaging such gear myself also.  I'll bet we find that many of the virtualization technologies are exploitable/exploited in the coming years.  OTOH, I'm expecting the same of various chipset components and CPUs.  If BTC goes another 10x then maybe I'll go with my preferred hardware route, but I an going to try my level best to remain a cheapskate no matter what the future holds.

Thx for the input.

Yep. Wouldn't mind hearing about alternatives, though. Limiting is nice, but it'd also be nice to give applications (or even ports) "minimum guarantees."

I am running Armoury on Win 7. If I use Bitcoin-QT then I get a similar network usage pattern.

Updated image. Complete Nov. data and partial Dec. I now use "d" instead of "QT," so won't have "unified" data going forward.

I realized my flaw in this after wondering why the numbers were so freakin' consistent month-over-month. The "real" minimum is significantly lower than 9.5kB/s TCP. QT still downloads transactions to add to queue and relay, however. Since this transaction queue exchange appears to take significantly more bandwidth than block exchange, the download bandwidth limit is almost always reached. QT possibly prioritizes bandwidth, giving top priority to block exchanges instead of transaction exchanges (clever behavior if true).

I'll see if I can find the "truer" minimum. Will let it run in 24h spans at different limits and find the lowest possible without generating a block backlog. Stats in OP are fairly useless with that in mind.

Good news! After some testing, it appears QT DOES prioritize block bandwidth over tx bandwidth (or blocks are just so relatively small, they're still downloaded within an acceptable time limit even when fully sharing bandwidth with the transaction queue process). This means QT can run at a MUCH lower throttle than I thought. I'm currently testing with a 2kb/s download throttle and will eventually start bumping down the upload throttle.

Because I know QT can function relatively normally on a 3kb/s leash, it probably can consume less than one gigabyte in data per month while still being able to operate like a full node (with caveats mentioned in OP).

Was about to send a PM to kludge, but since I might not be the only one to have adopted netlimiter after his initial post...
I had 2 days left of free trial when I checked today.
Don't forget to buy a life-time licence if you find it useful and downloaded it after seeing this thread. ;D (around 24-30€, depending on VAT)

Oh. That's a good reminder. I've been using a pirated copy for ages, but it's saved me enough frustration where I'm going to go ahead and purchase at least a couple licenses.

I would like to do this for my client/s on an Imac, any suggestions on how I should go about throttling the bandwidth?

As far as I can see, there isn't anything quite like NetLimiter for Mac. I've never used it, but WaterRoof has *some* of the features. It doesn't appear to support throttling by application, though, only IP address & port, which is kind of useless. They accept BTC, though. http://www.hanynet.com/waterroof/index.html If you set Bitcoin to use a specific, obscure port, maybe you can throttle traffic in and out of that port, so you effectively only throttle QT.

My post was not as violent as that quote, just a "mis-click confirm" quote, fixed like 2 minutes later. But the idea is here.
I think this should be obvious options in QT.

Max upload BW
Max download BW


And yes, please pay for what helps you. :)

Update + ISP sync times and costs for no particular reason (bored, I guess):

NetLimiter 4 doesn't currently support stats, but will soon-ish. I won't have any new data until it's implemented.

Core currently does not require more than 5kb/s down once sync'd, possibly a little less, though I see no reason to risk going lower. Therefor, dial-up CAN (theoretically) still support Core running IF physically given an up-to-date blockchain and the connection is left on for the vast majority of the day without interruption. Satellite is a little more sketchy due to the increased overhead from high latency and dropped connections... I can't get experimental data for that (well, I can, but I'm not paying $80/mo for shit service just to see what happens :D).

Since I personally get my connection from a 4G network (after buying a fairly expensive amplifier and antenna made directional with beer cans ;D), I'd like to organize some quick stats on US Internet speeds in comparison to sync time. Fun fact: 15% of Americans have no Internet connection.

Givens:
Current blockchain size: ~18.9GB (19818086.4kb)
One kb/s translates to ~0.00000095GB/s, which translates to .003433GB/hr. Therefor, the calculation to determine hours to download a 18.9GB file for a 7kb/s connection is (18.9/(.003433*7)).
Stay-sync'd bandwidth req.: 3kb/s
A month has roughly 2,592,000 seconds in it.
Pricing/speed data for DSL & cable assumes Jackson, MI location, FiOS assumes Chicago, 3G/4G data is national average for carrier as reported by opensignals.com
3G/4G assumes "pirate tether"
Promotional rates are not considered, but installation and ETF fees are ignored
Network reliability is not factored in
Minimum cost/gigabyte/mo assumes you use every byte you possibly can (unreasonably unlikely EXCEPT when letting Core sync 24/7) and achieve advertised speeds (unlikely)
(I should've thrown this in a spreadsheet, but wasn't intending to analyze this much info when I started)

Dial-up
Total US usage: 3%
Total US coverage: 97-99%? (unknown)
Examined: NetZero "Basic"
Cap: None
Price/mo: $10
Speed: 7kb/s
Max download capacity per month: 17.3GB
Min. cost/gigabyte/mo: $.578
Bitcoin min. sync time: ~32.77 days
Bitcoin theoretical sync cost: ~$10.92
Bitcoin theoretical stay-sync'd cost: ~$4.29/mo

Satellite
Total US usage: ???
Total US coverage: 100% (except for caves and enormous lead buildings)
Examined: Hughesnet "Power"
Cap: 20GB (not a typo)
Price/mo: $60 (not a typo)
Speed: 1,280kb/s
Max download capacity per month: 20GB
Min. cost/gigabyte/mo: $3
Bitcoin min. sync time: ~4.3 hours (in reality, probably over two billing cycles)
Bitcoin theoretical sync cost: $56.70
Bitcoin theoretical stay-sync'd cost: $22.24/mo

DSL
Total US usage: ~18-30%
Total US coverage: ~70%
Examined: AT&T "Pro"
Cap: 250GB
Price/mo: $40
Speed: 375kb/s
Max download capacity per month: 250GB
Min. cost/gigabyte/mo: $.16
Bitcoin min. sync time: ~14.68 hours
Bitcoin theoretical sync cost: ~$3.02
Bitcoin theoretical stay-sync'd cost: ~$.32/mo

3G
Total US usage: ???
Total US (home) coverage: ~60-80% incl. all carriers?
Examined: Sprint "Unlimited"
Cap: None
Price/mo: $80
Speed: 900kb/s (maybe they meant mbps??)
Max download capacity per month: 2,224.73GB
Min. cost/gigabyte/mo: $.036
Bitcoin min. sync time: ~6.12 hours
Bitcoin theoretical sync cost: ~$.68
Bitcoin theoretical stay-sync'd cost: insignificant

4G
Total US usage: ???
Total US (home) coverage: ~25-40% incl. all carriers?
Examined: Sprint "Unlimited"
Cap: None
Price/mo: $80
Speed: 5,100kb/s
Max download capacity per month: 12,606.81GB
Min. cost/gigabyte/mo: $.00635
Bitcoin min. sync time: ~1.08 hours
Bitcoin theoretical sync cost: insignificant
Bitcoin theoretical stay-sync'd cost: insignificant

WISP
Varies wildly, not included

Cable
Total US usage: ~31-50%
Total US coverage: ~70%
Examined: Comcast "Performance" (and fuck them!)
Cap: 300GB
Price/mo: $65
Speed: 3,200kb/s
Max download capacity per month: 300GB
Min. cost/gigabyte/mo: $.21
Bitcoin min. sync time: ~1.72 hours
Bitcoin theoretical sync cost: ~$3.78
Bitcoin theoretical stay-sync'd cost: inisignificant

FiOS
Total US usage: ~8%
Total US coverage: 10-15%??
Examined: Verizon "??? [image of a lightning bolt]"
Cap: None (don't you dare try to run a commercial server out of your house, though!)
Price/mo: $350
Speed: 64,000kb/s
Max download capacity per month: Exactly one fuck-load. (AKA 158,192.64GB)
Min. cost/gigabyte/mo: $.00553
Bitcoin min. sync time: ~5.16 minutes
Bitcoin theoretical sync cost: insignificant
Bitcoin theoretical stay-sync'd cost: inisignificant

US Average
Speed: 1,113.6kb/s
Bitcoin min. sync time: 4.94 hours

ETA: fixed FiOS max down/mo capacity

Technically satellite would be a nearly perfect medium for Bitcoin. It would however take cooperation from both the provider and the core development team.

Provider would need to make a broadcast/multicast channel for Bitcoin blocks and transactions that all subscribers could receive simultaneously. This would produce tremendous savings both in bandwidth and in the latency. I can't really talk about cost, because I'm completely out of date on this aspect. Sending transactions could be made over dialup or cellular SMS or any other low bandwidth medium like postcards with the QR code mailed to the provider, or even carrier pigeons (with microSD cards) lent out by the provider. Edit: obviously normal satellite uplink is also acceptable, but not really required, because of the extremely low required uplink bandwidth: only to distribute own transactions generated by that node.

Core development team would have to adopt the Bitcoin over UDP, which I believe was prototyped by jgarzik in his picocoin implementation. This is more of a political issue than a technical one.

Edit: obviously this would centralize Bitcoin in the same way as pools centralize it: we would have to make sure that there are several satellite Bitcoin providers with overlapping footprints.

I remember seeing someone actually talking about launching a "bitcoin satellite" (maybe it was you). Cost is a crapshoot which could go into the tens of $millions annually if the company's feeling mean/"smart"... it's not like there are many companies you can talk to, so I'd be surprised if they even talked to anyone about it without first being bribed with a suitcase of money. :D

I'm unsure if centralization's really a worry... I've never heard of issues with any coins' bootstrap torrents when they're signed by the devs (though BTC's obviously much smaller than where it could go in the moon-future). Incidentally, I recently talked with a major pirate scene group about what they do to protect against impersonated scene releases, and they say it's really never been an issue even though they don't even sign what they push and actual publication of torrent files and magnet links are usually distributed via third-party sites like ThePirateBay. The only problem they have, and it's really just an "eyeroll problem," is when other people copy their work-arounds to DRM and claim it as their own. That's not "money," but an infected impersonation under the right circumstances could result in the creation of a massive infection of hundreds of thousands of computers, which can certainly cause $millions in damages. I don't think we should just wait for a problem to arise, but I don't see much issue trusting a blockchain maybe three devs have had verified against the "current majority blockchain" and sign; I think that's enough. I think it's either Jeff or Greg who maintain the current bootstrap torrent, so that person'd obviously be a good pick...

ETA: Wait... how are people supposed to pick the satellite signal up?

Just a note on satellite from a user:

I use 'Exede' and it is surprisingly usable.  I activate bitcoind from time to time when I need to access a cold storage wallet.  So far I only need to sync to the age of my cold storage wallet in order to spend the funds which means only a tiny fraction of the entire blockchain.  This is good because more than once the database has become corrupt and I needed to start the sync over.  I don't use UPnP and de-activate it in my router for security reasons (and didn't compile it into my binary) and I've not set up port forwarding.  In short, I've no idea if the latency is impacting my ability to serve blocks.

I get 15G/mo for $80.  That is double the standard 7G/mo.

I would not consider satellite to be at all decent for reliable peer-2-peer in an adversarial environment.  I believe that all satellites are dual-use civilian and military and military (including state sponsored surveillance needs) take precedence.  (I could be wrong.)  In any event, I have zero confidence that the satellite providers would do anything which is remotely adverse to the demands of the government, and if there was a mandate to interfere with peer-2-peer traffic of any nature I expect that the satellite operators would comply with enthusiasm.

It is also utterly absurd to hope that independent entities could control satellites.  Space is highly regulated and increasingly militarized.  The U.S., as a highly militarized and sole super-power relies heavily on space for sustained dominance in a variety of areas, and opposing forces (e.g., China) are putting significant efforts into systems of attacking space-based communications systems if need be.

To me it is a no-brainer to focus on devising a defense of crypto-currency value stores as concentric spheres of inward-increasingly robust solutions.  At the core would be a high latency no-frills system which would be very difficult to successfully attack on an extended basis.  Other more user-friendly systems could use it as a foundation.  I'd like to see Bitcoin evolve to be the inner core, but evolution seems to be going in the other direction to my chagrin.  OTOH, the idea of using Bitcoin proper as a 'reserve' and 'balancing' currency does seem to be gaining some traction of late.

blk compression testing is done "good enough." Thanks to deslok for hassling me throughout. GoogleDocs has decided to suddenly stop publishing notes in their latest mandatory upgrade of spreadsheets because they're assholes, so I'll have to manually list notes here... >:(

https://docs.google.com/spreadsheets/d/1NmgCVnLI8WBEdzjQvMhoe64LGXp23XiMGZU8Syz0lAI/pubhtml

Specs -
Phenom II x4 (965) 3.4GHz
16GB DDR3 RAM
7200RPM HDD
Win7 x64

blk00000
Nanozip 1 - .09a x86: optimum2, 512mb RAM allocated
Nanozip 2 - .09a x86: CM, 512mb RAM allocated
Nanozip 3 - .09a x64: optimum2, 4096mb RAM allocated
7zip 1 - 9.2 x64: LZMA2, ultra, 1024MB dic. size (word size 273), 3 threads
7zip 2 - 9.2 x64: LZMA2, ultra, 512MB dic. size (word size 192), 4 threads
7zip 3 - 9.2 x64: LZMA, ultra, 1024MB dic. size (word size 273), 2 threads
7zip 4 - 9.2 x64: BZip2, ultra, 900KB dic. size, 4 threads
7zip exe 1 - 9.2 x64: LZMA2, ultra, 512MB dic. size (word size 192), 4 threads, put into SFX

blk00000 + blk00001
7zip exe 2 - 9.2 x64: LZMA2, ultra, 512MB dic. size (word size 192), 4 threads, put into SFX

entire "blocks" folder
7zip 1 - 9.2 x64: LZMA2, ultra, 512MB dic. size (word size 192), 4 threads
7zip 2 - 9.2 x64: LZMA2, ultra, 1024MB dic. size (word size 273), 3 threads
nanozip 1 - .09a x64: optimum2, 7500mb RAM allocated


Overall, nothing particularly exciting discovered. 40-45% compression is to be expected (I never saw 45%, but I'm kind'a dumm), which is pretty critical for initial sync (for anyone using cable or worse as ISP) and makes a solid case for compressing bootstrap files, but not very useful outside. Unsure why nanozip's "optimum2" algorithm did so poorly with the extra blocks when they contain so much repeated info.

I'm working on that last 7zip archive, but it shouldn't be thrilling. With the test I did do for the "blocks" folder, assuming you have a 3200kb/s cable connection, even after a ~13 1/2 minute decompression time, you, overall, save about a half-hour in time, don't have to use up unnecessary bandwidth from your monthly allotment, and stress the infrastructure a solid deal less. With DSL, all those benefits are greater and you'd be saving a good many hours. With satellite ISP, you'd be able to download the entire blockchain in one billing cycle without buying ultra-expensive pay-as-you-go bandwidth. In "US average" conditions, the benefits are significant, too, and it'd also save hours of time.

Assuming an "average GB cost" of $.10 and 100,000 new full nodes per month, the monthly cost savings of compressing blocks for the initial sync would be right around $85k, or ~$1M annually. That's a pretty BS statistic, so fwiw. Additional RAM, CPU, and HDD load are all too minimal to really make a difference compared to that (and you may get a net save in electricity costs by not needing to have the PC running as long just to download the blockchain).

Here's a more useful tidbit from the info-gathering: spending an hour compressing the blockchain bootstrap torrent once would, assuming an average download speed of 1,100kb/s, with 100k new nodes per month, would save ~2.4 million hours globally in sync times every year. (rather, 274 "years" would be saved every year)

on personal (and permanente) connexion of bitcoin-core ... i set 4 connexions max. = 5ko/s of UPLOAD
on p2pool server support with bitcoin-core, the minimal connexion must be set to 12. = 10-20ko/s of UPLOAD

ko?


Ah, ko == kilooctets.

Merci bien.