Bitcoin Forum

I know this subject has been beaten to death but I still don't understand the technicals.

I would think that amount of transactions can go up with personal computer power. Since personal computer power grows exponentially, can amount of transactions also not increase exponentially?  

I also don't understand this expression:
"The block size is an intentionally limited economic resource, just like the 21,000,000-bitcoin limit. Changing that vastly degrades the economics surrounding bitcoin, creating many negative incentives. - Jeff Garzik"

Any help is appreciated.

There is a hard limit in the protocol that doesn't allow for blocks bigger than 1 MB.

Nodes would not accept Blocks that are bigger than 1 Mb and reject it. Changing this would require a hard fork, meaning all current version would not accept blocks of the new version as valid and therefore forking the network between a old version Blockchain and a new version Blockchain.


Meaning that changing that limit would affect Bitcoin in the same way as changing the 21 Mil Coin limit.

Jeff seems to have the opinion that "there will never be more than ~7 transactions per second" is as much a core principle of Bitcoin as "there will never be more then 21 Mil bitcoins" and changing either on of this numbers would destabilize Bitcoin and have people to loose confidence in it and maybe have some other negative effects.

This is per client?

just leaving this here

Sorry, I don't know what you mean?

1. Is this a hard cap in the client?

Yes, the current client would not acceppt blocks > 1 MB as valid and reject them.

2. Are the ~7 Transactions per second per client.

No, that's the (rough) total number of transactions that can be included in a block. If there would be more some would never be confirmed, no matter what fee they would pay.

But this topic has really beaten to death. Read this thread if you want more information about this: https://bitcointalk.org/index.php?topic=140233.0

Bitcoins for payments? You disapoint Visa guys. VisaNet processed an average of almost 11,000 transactions a second during the busiest minute of the year.

I'm still vetting the history on this, but, I believe the 1 MB limit was originally set arbitrarily as a way of protecting the block chain from attack. It was never intended to be a limit on real economic activity.

That said, from a technical perspective, it's easier for the network to deal with 7 tps (transactions per second) than 7000 tps. With the way the block chain is stored now, 7000 tps would make for some rather big blocks and rapid block chain growth. I'm uncertain how much magic can be invented by our mostly unpaid devs to implement the technically challenging pruning and compression that's oft written about (but it's a boat load of work and would cost a fortune in the real world).  

Anyhow, I view Bitcoin's value as paralleling the total # of tps. It's a relationship that's held generally true. Those who have bitcoins stand more to gain than those who don't by raising the max block size, IMHO. Those with no or few Bitcoins can make more money by devaluing Bitcoin by limiting the max block size and creating their own Bitcoin-like network for off chain transactions.

To me, it's just a question of what do you want to be more valuable, Bitcoin or off-chain-processors. I also believe the limited resource argument is FUD. Miners set the real limit no matter what the max block size is (well, not above it!).

If Bitcoin is to achieve mass adoption, the protocol will need to be handle way more than 7 tps. This change would be better performed sooner rather than later. The userbase is growing, and it will only become increasingly difficult to get everyone to update to the latest client with time.

and now reveals why bitcoin mainstreaming wont happen anytime soon.

1. starbucks does more then 7 transactions a second.. add on the transactions of walmart.. instantly with just 2 major retailers bitcoin cannot cope.

2. buying a $1 loaf of bread with a visa card would cost the customer $1 and the retailer would get 99c. with bitcoin he would only get 93c.. why?
well the transaction fee of 0.0005 just to get a fast confirm would cost 6c then add bitpay 1% fee to convert to fiat.

3. as bitcoins fiat value increases gavin andressens ignore dust feature will also add more costs onto small transactions.

so if you want to buy a drink, sandwich or coffee at a vending machine. don't look at bitcoin for your solution.

I think the blocksize limit is the ultimate hard proof that Bitcoin is not a Ponzi scheme-when you buy bitcoins, you actually invest on the infrastructure of the network, because a decentralized payment network and a permanent, always available tamper-proof database is useful to people, people are willing to pay miners to run it, if the upkeep is not enough to provide unlimited access to blockchain for everyone, the resource will then have to be rationed, which gives us the 7tps limit-my 2 cents on Jeff's words.

A lot of people are complaining that the Bitcoin network cannot scale to high processing capability, but you can't have it both ways, a system cannot be low-cost, highly-decentralized, and have high-availability all at the same time.

The federal wire network processes about $1.6 quadrillion USD worth of transactions annually and it works out to roughly 7tps.  Not saying the cap can't be raised but the idea that all economic transactions must be on blockchain or bitcoin "fails" is a dubious binary distinction.  It is also possible the limit will be raised but there still will be a limit (as opposed to unlimited block size).


Actually VISA routinely charges $0.30 PLUS 2% so that retailer selling a loaf of bread (not sure what bread is only $1) would get ~$0.68.  The 0.0001 (it was lowered in 0.8.2) min tx fee is just to avoid spam.  The market will decide the price of fast confirmations.  Also low value tx could be processed off blockchain so the merchant potentially could end up more.


There is no ignore dust rule, there is a DON'T ALLOW CREATION OF NEW DUST.  Of course the "rule" has a default value one that smart miners will decrease over time.  Newever versions of the client will almost certainly adjust the default value for nodes which don't override the default as well.  I would point out the min tx fee (for low priority txs) was initially 0.01 BTC that was lowered to 0.001 BTC, 0.0005, and now finally 0.0001 as the value of a BTC has risen. 


Maybe?  Bitcoin is an experiment in progress.  It may be that:
a) you can use on blockchain tx for these transactions but merchants would prefer (and thus offer a discount) if you use off-blockchain txs for.
b) you end up using an alt-coin backed by Bitcoin specifically designed for high volume low value transactions.  Bitcoin remains used for larger transfers of value.
c) you can't economically use a crypto-currency for these types of transactions and Bitcoin still becomes the worlds largest value transfer network in the history of the human race.

You can't simply change the client.  The existing protocol is incompatible with a larger blocksize.  Period.  You can't force that protocol to cease to exist.  Today we call that protocol Bitcoin.  The only option is a hard fork, an incompatible version of the protocol but that means the existing protocol still exists as well.  Essentially two incompatible Bitcoins.   The only viable solution is before making a hard fork you have such overwhelming support that a super super super majority of all users (not just miners but merchants, exchanges, service providers, full nodes, and active users) switch that the old fork dies off.  If you don't one of two things will happen:
a) most people won't leave and your forked Bitcoin will just die off
OR
b) both forks gain significant backing and people end up with differing wealth of both chains.  Both "camps" have a vested interest in their chain surviving so you end up with chaos.  Two Bitcoins both calling themselves Bitcoin but completely incompatible to each other.  (Yes this is the bad news scenario which is why hard forks should never be considered trivial changes).

Who do you think are paying the multi-billion dollar Visa makes every year then? And if Bitcoin really becomes that valuable, do you think the miners would not  just get off their lazy ass to change one or two lines of code(assuming devs won't do) to accept small-amount transactions?(Yes, it's that easy)

Yes, but the longer we wait, the harder it's going to become to gather overwhelming support. You think the average joe is going to care about this?

Bitcoin was always intended to scale to high transaction rates. Satoshi talked about it frequently, and in the very early days of Bitcoin every time somebody brought up the block size limit they were always told it would be raised when needed.

Originally there was no specific limit on block size except for a 32 MB message limit that applied to all node communications.

The block size was lowered to 1 MB in late 2010 because of a concern that an attacker could fill up everybody's hard drive with spam. Again this change was noted to be temporary by Satoshi, who posted about it on this forum.

Somewhere along the line some people decided they want the limit to stay at 1 MB forever. The reasons for this vary from person to person and appear to include: a desire to make alternate currencies artificially viable, they invented some type of off-blockchain transaction that nobody wants to use and so need some extra incentive to make people adopt it, hostility to Bitcoin's potential as a payment system that anyone in the world can use directly, a desire to see Bitcoin fail, or they invested in mining hardware and never want to be forced to upgrade or change their business model in order to remain profitable.

Frequently those who are in favor of keeping the block size at 1 MB will resort to lies of omission, misrepresentation, and fallacious economic arguments to make their case.

The average Joe will just see "There is a update available for your Bitcoin Wallet" and just install it. I would rather worry that this will make nefarious changes easier.

I think it is important to distinguish between A limit and the current limit.

So there are a couple scenarios:
a) limit totally removed from code. unlimited block size. Tomorrow a miner could produce a 80GB block if they desired to
b) limit remains but is raised to another limited value say 20MB.
c) limit remains but protocol is enhanced to provide a rising limit based on deterministic factors.
d) limit remains and is now and always shall be exactly 1 MB.

Personally I think a is impossible and has a lot of unintended consequences.  It would allow centralization through bloat.  Eventually a handful of companies would be the only ones with enough resources to run a full node.  This can be done by intentionally mining massive amounts of data to push the full node requirements (think TB per day of storage, the need for 24/7 high bandwidth low latency links, and 128GB of RAM) to kill off existing full nodes.  Eventually this cartel of full nodes could gain enough power to require governmental approval to add new full nodes (we will call this a charter) and the ability to charge users for access through their node.  Many users will deposit their coins directly at these full nodes.  Congratulations you just invented banking.

The last scenario (d) is the status quo.  I don't think it will remain "forever" as forever is a long time.  It is important to consider that the change is non-trivial.  Anyone thinking the solution is "easy" likely hasn't thought about it enough. 

The new fixed limit (b) gives Bitcoin some breathing room.  PayPal for example does about 50 tps so a 32MB block size would allow ~224 tps (or 5x PayPal).  At peak usage it would make Bitcoin one of the largest transfer networks in the world.  No it wouldn't be the one system for all transactions but I doubt anyone but the purists would consider Bitcoin a "failure".  It has the ugliness of a new larger "magic number".

The dynamic scenario (c) is the most complex and as such the one with the most potential for creating hard to troubleshoot scenarios that could cripple the network either through lack of understanding or by introducing a new attack vector that entities who wish to see Bitcoin destroyed could exploit.

Personally if this was a dictatorship I would go with something like:
* Announce unlimited blockchain is not viable and won't be attempted
* Keep 1MB limit until the next block halving at which point the limit would be raised to 32MB.
* Fund research into an optimal way to have block size expand deterministically.

Of course this isn't a dictatorship and Bitcoin requires consensus which makes things even more challenging.

Can the client not be updated now (or soon) to handle more TPS and larger block sizes but not yet create them?  Leave this change in place until needed - 1 or 2 years.  Then when miners actually implement larger block sizes most people will already have a client that can deal with it.

The average Joe isn't going to run a full node so his options are limited to what those running a full node do.

Even at 1MB limit sustain the blockchain will grow by 87GB annually.  That puts a limit on blockchain at >300GB by the time of the next subsidy halving.  A well connected nodes (say 24 peers) would need ~192Mbps of upload speed to share the block in a timely manner (~1 second).  The CPU load for synced nodes would be minimal but for bootstrapping nodes the number of tx to validate annually would be ~220 million.  So a new node around the time of the blokchain halving would need to download and validate roughly 1 billion transactions to get synced.  Even at 1MB sustained the requirements become a decent server (storage, memory, cpu) that has high speed, low latency links running 24/7.  Not impossible for an enthusiast or business but your average Joe isn't going to run a full node.

Really? If the 1 MB limit was lifted right now via a simultaneous upgrade of the entire network, what would actually happen if a miner tried to produce a 80 GB block 10 minutes from now?

Assuming the outputs can't be spent (remember, this was an evil miner doing this): 80GB must be stored forever. To put that in perspective, I was hopping to buy an 80GB SSD for my full node, and simply upgrade as the block-chain gets too large. I was hoping that would be on the order of a year or more, not 10 minutes :P

Edit: since 80GB would take a long time for most nodes to download, such a large block may actually get orphaned because it does not propagate fast enough.

I think from an economic standpoint, it makes sense to scale block size based on Tx fees. For example, at each difficulty adjustment, calculate average Tx fees in a block over all blocks since last adjustment, and scale block size based on that. That way, the extra cost to miners of storing larger blocks is made up by larger Tx fees. When competition for block space increases and people start paying more in Tx fees, the block size scales accordingly. This of course opens an attack vector where a miner mines many blocks himself, collecting his own Tx fees, to raise or lower the block size artificially. Also the equation to calculate block size from average block fees would still be an arbitrary/magic component.

Unless the miner controls a majority of hashing power such that his chain will always have the largest accumulated work, a block that large in today's network is guaranteed to be orphaned.

Even without a hard cap on block size 80 GB blocks will not be possible until the majority of mining nodes have sufficient bandwidth to move that much data in a timely fashion.

Really? Even you buy this story that without central planning we'll get an "evil cartel"?

You're very active in this forum so you're probably aware that miners could set their own soft limits to block sizes, and only accept a block beyond such limit if it's "deep enough" (that being whatever they want). This feature could be coded in bitcoind with some values by default, as the transaction fee policy, making it very easy for miners to perform such decentralized block size control.
So, how could this "centralization by bloat" actually happen?

Don't forget that the probability of being orphaned increases if you create extremely large blocks - even if we ignore the chance of hitting some soft limits, chance that we should not ignore. Just look at the examples we see today. I've heard some pools were voluntarily censoring SD just to avoid the extra propagation time.
Oh, and don't forget either that you generate storage costs for yourself too if you start bloating the chain.

Certain costs and risks in exchange of a hypothetical possibility to kick out a few pool operators with limited bandwidth? No way.
Actually, DDoSers have unfortunately already done much more damage than this hypothetical risk you claim could possibly do... many pool operators have to hide themselves behind DDoS resistant ISPs already, which means large bandwidth. And that doesn't seem to be creating any "unbreakable evil cartel". It's not that difficult to start a pool today.
You should check Hearn's calculations on bandwidth costs and bandwidth demands for running a full node, they are somewhere in this forum I think. You won't ever need millions of dollars to run full nodes. A few thousands per year would be enough, even with Visa-like volumes. That's far from being an insurmountable barrier of entry capable of protecting "evil cartels".

I'm with Gavin and Mike Hearn on this one: there's no reason for a hard limit. Even if such catastrophic scenario you describe ever materialize (it won't), it wouldn't be difficult to fix it by just going back and adding some limits.

Not only it would not propagate in time, but even if it did, it'd likely hit everybody's soft limits so everybody would just ignore this block and keep building on top of the previous one.

caveden I think you misunderstood I wasn't talking about miners but full nodes.  

Right now today there are x nodes.

If the average block size increased to 10MB the cost of those nodes (who receive no revenue) would increase.  It is likely the number of full nodes would thus decrease leading to an increased amount of centralization.
Now what if the average block size is 20MB, 50MB, 300MB, 1GB?  

I wasn't talking about miners.  Miners are already highly centralized.  I was talking about independent full nodes which the network requires a large number to operate effectively.  As the block size increases the (uncompensated) cost on those full nodes increases and the % of participants in the system running a full node (and thus paying the full cost) will decline in favor of SPV and other lite solution (which are "free").   If eventually the network handles 100,000 tps but the demands on a full node are so high that only the largest of the largest of companies operating in massive datacenters can run a full node then you might as well call those "full nodes" .... banks.  As the network grows those established large entities have an incentive to encourage more growth as it acts as a barrier to entry against smaller participants.  Barriers to entry = higher profits it is both rational for entities to attempt this and realistic to expect it.

The idea that miners would soft fork blocks which are "too big" is equally disturbing.  First of all it is unlikely it would work.  Nobody is going to make a block larger than what all but the tiniest fraction of miners says it "too big".  If 10% of miners say they will soft fork blocks over 5MB only an idiot miner would make a block larger than 5MB.  You are taking a 10% orphan chance by making the block even a few bytes over the limit.  Another element is that Bitcoin works on the concept (at least in theory) that miners are independent actors.  If various miners will soft fork at various levels you have created a disincentive to remain independent.  You NEED to know what all your peer miners are doing.  We shouldn't be building a system where independent miners (the desired state) is at a disadvantage to miners in coordination.  Lastly if various miners do have differing soft for levels than an attacker can exploit that to degrade the effective hashrate of the network.  This would open the network to a 51% style attack with less than 51% of hashpower.  What matters is effective haspower (i.e. hashpower applied to the longest chain).  If miner X has a soft fork level of x MB and miner y has a soft fork level of y then by planting various blocks of differing size the attacker could fragment the good miners into working on competing chains.  Anytime miners are on multiple chains the network is only as strong as the hashpower on the chain with the most hashpower.  Note miner in this case is the entity (pool for most hashers) that is making the strategic decision of what chain to extend and what tx to include.  Hash processors aren't truly miners in that they are already following an authority (the pool server).

Even at 100,000 tps, small companies could afford to run full nodes.

The current implementation of the P2P network is not particularly efficient in terms of bandwidth usage and storage requirements. There's a lot that can be (and is being (https://bitcointalk.org/index.php?topic=204283.0)) done to improve this and also as more businesses of all sizes become dependent on Bitcoin they'll have an incentives to operate full nodes and to promote the development needed to reduce the resources needed to run them.

I never said that that nobody could run a full node. Although 100,000 tps with 16 peers would require ~5 Gbps of bandwidth.  I doubt most small businesses could afford that type of server costs.  Even if they "could" the cost to run a SPV is "free" by comparison so you WILL see less full nodes than say a network capable of "only" 1,000 tps.

The average Joe is not running a qt client.

Small companies can afford that. This assumes 16 peers, where I think the average is now 8, bandwidth not getting cheaper, and SPV clients not paying full nodes microamounts for transaction data.

And that there will continue to be a difference in security between SPV and "full" nodes.

You'd think with all this processing power we could harness it a -bit- more efficiently

Have OP watched this: http://www.youtube.com/watch?v=cZp7UGgBR0I  ;)

This is always a political and economic discussion instead of a technical one. In one word: Scarcity create value, aboundant destroy value, as a currency you want it to have highest value possible, so transaction capacity should be scarce, people will adjust their behavior accordingly

VISA is transferring inflative fiat money for the purpose of spending, people will always spend inflative currency at the first place, so even bitcoin's network are capable of handle huge amount of transaction per second, it still will seldom be used to buy milk everyday

so in the end the 1mb limit is a factor in the evolution of the Bitcoin ecosystem.

We want as many people as possible to participate as possible with as little access as they have, so little bandwidth needed for a node to be run on a cell phone or a computer with a dial up connection, even making it effective to run over TOR or a mesh network effectively.

Maybe this limit creates better compression protocols for the transactions in the future? making them smaller squeezing more transactions into a block: Turning the average T from .36kb to .10 kb much easier for the network to send and increasing over all speed as well.

It can also bring the issue forward about the status of the network, giving it time for high speed mesh networks to be built or vpn corridors to be assembled, targeting nodes with high speed internet offers, etc. have you guys heard of hyperboria? that can be a solution as well free internet access to bypass the cost of running a node.

The economic aspects of the Bitcoin experiment could be enhanced by the limit as people begin to saturate not only the block chain limit but the 24 hour cycle so that no one without a fee can squeeze in, transaction acceptance by market decision, that aspect of the experiment is still left to be truly observed. The limit after all is 400,000 transaction per day, or 144,000 if we consider all the bloat from dust transactions. I do like satoshi dice by the way, that may be another target for optimisation :P

who knows how else a discussed and debated limit can create change.

What other limits does Bitcoin have?

Scarcity only matters to the extent that it contributes to value. The value of the currency comes from the demand to supply ratio. Demand will be limited if transactions are limited. The supply is 21 million BTC regardless of how many transactions are allowed, so more transactions = more demand for each unit of BTC.


Exactly.

I just thought of another reason to keep the block size limited.

Perhaps it's true that without a protocol limit the blockchain would grow to the point that even the most well funded datacenters would find that keeping the entire history all the way back to the Genesis Block was unfeasible.

That by itself wouldn't be a problem for Bitcoin - the only essential data in the blockchain is the UTXO set, plus enough recent history to handle a reorg event. If the blockchain got too large too quickly it would grow to a point at which pruning becomes essential and nobody is able to keep a permanent record of every transaction that has ever happened since 2009.

That would be wonderful for privacy and could be implemented with a negligible reduction in network security. I can understand why some people would be opposed to that.

We did just hard fork a few days ago...not sure many noticed :)  It _can_ be done relatively seamlessly.

Yes...that's more or less what satoshi suggested.

Will you change a bank just because one bank opens 24/7 and another bank open only 4 days a week? I guess the priority should be based on the security and the interest that banks provide. Currently bitcoin bank pay you an interest of 1000%+ per year, and you still complain that you have to wait in line to withdraw ;)

The value of a currency comes from trust and consensus, not supply and demand, this is the most significant difference between money's valuation and anything else's valuation. The demand for money is always endless, given enough trust for that money

Very true we have conceded to value this currency... could it be that the exchanges are hijacking it's value? we have had speculators flooding the market this year, but wow! have you seen how steady the price has been across all the exchanges lately?

so how do we concede to value Bitcoin then, rationally and backed by science? instead of scarcity?

The fees as well, their is a market on fees... but more importantly it seems to be guided by luck. Maybe your lucky and you get a ASIC miner to take care of your transaction instead of a efficient FPGA miner.

and what about the ecosystem from the other alt coins? how does that change it's valuation?

the factors for Bitcoin are complex and deep, but very entertaining to discuss. :)

I don't have answered just questions. ;D

Why the wink? :) Yes I saw that but there is a strong argument against this. As long as an ordinary personal computer can deal with it, it will remain decentralised. But since ordinary personal computers go up in power exponentially, so can the blocksize limit/amount of transactions go up exponentially, right?



I think that is a strong argument against bitcoin that initially came up for me. However this is counterbalanced by people always want to receive sound money in the first place.  Sound money like standardised gold and silver coins, came into widespread usage, not because people insisted on spending those, but because people insisted on receiving those.



Thanks for explaining that scarcity element applied to amount of transactions. I don't follow that reasoning though. Like many others I think the value of the currency goes up thanks to scarcity of amount of bitcoins, but goes down if there is a scarcity of amount of transactions. People will adapt by using another currency for smaller transactions, lowering the value of bitcoin and bitcoins.

Indeed I thought you were talking about miners. I don't view a smaller number of full nodes as a problem. A single honest full node is enough to spot any attempt of fraud. There'll always be more multiple honest full nodes running.


Come on. You know you'll never need a "largest of the largest company operating in massive datacenters" to run a full node.
100k tps? What's that? Visa apparently operates at 4tps according to the scalability page. How could Bitcoin possibly outgrow visa by 25 times in any reasonable time?

The world has ~7 billion people. A day has 86,4k seconds. With 100k tps, that'd be 8,64 billion transactions per day if I'm not mistaken. It's more than 1 transaction per person in the world per day (even if we consider population growth, it'd still be close to one transaction per person per day at least). I mean... Bitcoin would already be the king of currencies if it ever reach such amounts. It'd be something so freaking important in the world, that you can rest assured there would be more than enough honest nodes running. Remember: one single honest node can spot fraud (like breaking the 21M limit for ex.). And also, by that time, we can't even imagine how powerful hardware will be.


Why disturbing? Btw, that's not something that can be prevented by the protocol.


Yes, the blocksize would only be increased if there's a large consensus, or if the monetary incentives to take more transactions outcome the risks/losses from eventual "orphanage". In this last case, that's supply adapting to strong demand.
See, market coordination. ;)


It depends how you see it. Today there's a protocol rule imposing a 1Mb rule. That's 100% coordination, everybody follows the same rule. By dropping it, you allow a bit more of "independence". Some miners may try a limit slightly larger.

It's unavoidable: there must some planning concerning what's a "too big block", because otherwise the network is vulnerable to flooding. I personally prefer the "planning by the many", and that's miners coordinating what's reasonable and what's not. They will tend to adapt to actual demand. Attempts of "central planning" (elaborating some formula etc) might not produce ideal results, and, as noted yourself, are difficult and error-prone.


If I understand it correctly you're talking about a race attack with the intention to double-spend some payments, not a total overtake. Because an overtake wouldn't work out this way in the long run. It might give some higher chances of a race attack, but I doubt it'd be anything meaningful. By the way, I'm not sure I get how this attack would take place...
Say, I generate a block that I know somehow it will hit the soft limits of 50% of the network (in hashpower). That means I break the network, half of it is mining on top of my block, the other on the previous. Now if I have >25% of the network's hashpower (a lot!) I can outpace any of the competing chains. But the thing is that, whatever chain I choose not to mine on will likely outpace the other. Even if it's the chain with the large block, eventually it will be accepted by those who initially refused it. It would still be very difficult to get any double-spend on anything with a meaningful number of confirmations. High value transactions always should always wait for significant amount of confirmations. And double-spending small transactions is not worth the investment.

What?? You're mixing the value of the currency with the value (cost) of transacting it.

The scarcity of bitcoin is set on stone: 21M units. No more.

Making Bitcoin transactions more scarce - thus more expensive - is more likely to reduce the value of the currency due to its reduced utility (if it's more expensive to transfer it around, it's certainly less useful).

The personal computer's performance did not get any significant gain during recent years and it is already a pain to download the whole blockchain with an average block size less than 256KB. The bitcoin client's setup time is already magnitudes higher than it was 2 years ago but you won't have magnitudes higher harddrive capacity and network bandwidth in 2 years



An interesting view... I suppose that who have the money decide if they want to spend. Of course merchants will give some discount for bitcoin payment, maybe it will work. Then we back to the already-too-big blockchain problem



The VISA transaction fee is very expensive, but that won't affect the value of USD. Same for bitcoin, the blocksize limit is there to ensure the value of bitcoin (decentralized to maximum), but if transaction cost is high, people will just stop use it to do small transactions

Bitcoin's transaction fee being low is one of its major advantages over the dollar, which has to use relatively expensive third party payment processors like Visa for digital transactions.

If bitcoin transactions become too expensive for most uses, then bitcoin will be just like fiat currency, where you are forced to use large private third party intermediaries, like Visa, for transaction processing, who can hike up their fees because they are the only processors with large enough private networks to be useful.

Then what are they using? I think most people google for bitcoin and end up on bitcoin.org where they see this:

http://bitcoin.org/en/choose-your-wallet

So yeah they end up using bitcoin-qt. Then they complain that the blockchain download is taking ages :) I know that's exactly what happened to me.

This article says the inverse, that the amount of transactions can still go up dramatically for a standard home pc:
https://en.bitcoin.it/wiki/Scalability

"As we can see, this means as long as Bitcoin nodes are allowed to max out at least 4 cores of the machines they run on, we will not run out of CPU capacity unless Bitcoin is handling 100 times as much traffic as PayPal"

"Let's assume an average rate of 2000tps, so just VISA. ... That means that you need to keep up with around 8 megabits/second of transaction data. ... This sort of bandwidth is already common for even residential connections today, and is certainly at the low end of what colocation providers would expect to provide you with."

What do you think of the arguments and calculations presented?

That math is valid and the CPU really isn't the bottleneck.  If it ever became a bottleneck the use of OpenCL GPU accleration or even dedicated ASIC processors capable of highspeed ECDSA verification and SHA256 hashing are certainly a possibility (eventually the "easy money" for producing ASIC miners will dry up).  That still may happen anyways to reduce power consumption, heat, and processor load (much like many servers use dedicated ASICS for SSL or TCP offloading).

The bottleneck is more in this order (from most critical to least critical):
bandwidth (for residential connections the upload segment tends to be rather constrained)
memory (to quickly validate txs & blocks the UXTO needs to be kept in memory, sure pulls from disk are possible and ... panfully slow)
storage (as much as people worry about storage it is a free market unlike residential last mile and HDD capacities already have a massive "headstart")
cpu (with moore's law I don't see this ever being a problem but as pointed out non CPU solutions are possible)

The average Joe hasn't even started using Bitcoin today.  The requirements of a full node will only increase.  Today users are already encouraging new/casual users towards lite nodes and eWallets.  That trend will only accelerate.  The demands on a full node are certain manageable but there will always be a cost in being an equal peer in a global transaction network.  Many users will opt out of that cost by using lighter solutions.

I agree with your bottleneck order. Bandwidth will probably be the first, particularly with SSDs getting cheaper (you can store your UXTO in a SSD for better I/O performance). CPU can be dramatically improved as you say. Storage is not such a big deal. And if memory becomes a big deal, good caching strategies together with SSDs could circumvent it.

Let's talk bandwidth then... It seems people in Kansas City already have 1Gbit/s available in their homes, up and down. Assuming the 400bytes average for bitcoin transaction that I read somewhere, that's more than 300Ktps if I'm not mistaken. That's a shitload of transactions. Even if transaction sizes were to multiple by 3 due to more usage of multi-signature features (something that I hope will happen), that would still be more than 100Ktps. What's the average number of times a full node has to upload the same transaction? It shouldn't be much, due to the high connectivity of the network. But even if you have to upload the same transaction 10 times, Google Fiber would probably allow you to handle more transactions than Visa and Mastercard combined! We're obviously not hitting such numbers anytime soon. Until there, there might be much more than 1Gbit/s available for residential links.

All these desperate attempts to hold the block limit become ridiculous when we look at the numbers.


I'm not sure. The greatest issue for new users is having to wait for the initial sync. If the client were to operate as an SPV in the meanwhile, and switching to full mode once initial sync is complete, I guess many more people would be OK with having a full node. Well, some would complain about how slow their computer got after they've installed this bitcoin-thing, and might be turned off. But not that much as today.

What many people don't realise is that the bandwidth numbers quoted on the wiki and by you only apply to keep up with the block generation rate. An independant miner will need 100x - 1000x more bandwidth to mine at all.

1 MB block size produced ONCE every 10 minutes NOT over 10 minutes
If im a miner, i want to download that new block as fast as possible to reduce my idle time.
Lets use 1% idle time as your target(Means your entire mining farm sits idle for ~6s while you download the block)
To download 1MB over 6s, you need about 1.7MBPS connection (seems reasonable for most people in developed countries)
10MB block size, 17MBps (Even i do not have a 17MBPs connection at home though it is affordable enough if i need it)
100MB block size, 170MBPs (Most countries are atleast 5-10 years away from having affordable fibre internet)

And that is assuming 1% is the market determined edge you can afford to lose to remain profitable.

Just to clarify im not opposed to an increase in block size as long as decentralisation is not compromised by ensuring that the block size remains small enough for average residential broadband connections/commodity PCs to mine with.

This could be solved by pre-announcing blocks: As soon as a miner decides on a list of transactions to include in a block the start broadcasting this list in parallel with hashing, and then broadcast the nonce once they've found it.

Agreed higher bandwidth connections will be more common in the future however if 1% of potential users have a 1 Gbbps connection and that becomes the minimum then you have reduced the potential full nodes to <1% of the planet.  The numbers also aren't as rosy as they seem on first glance.  A node by definition needs connections to multiple peers so a node connected to 8 peers will rebroadcast a tx it receives to 7 peers.  Now 8 is the minimum for network security we really want a huge number of nodes with high levels of connectivity (20, 30, 500+ connections).  So lets look at 20.  1 Gbps / (400 bytes per tx * 19 peers to relay * 8 bits per byte) = ~16,000 tps.  Now 16,000 tps per second is still a huge number.  However that would limit full node participation to those with 1 Gbps.  However the real problem is real bandwidth vs marketing.

1 Gbps sounds great until you saturation your uplink at 1 Gbps 24/7 continually every second.  In no time flat the ISP is going to cut you off or throttle you.   Even if they have no hard bandwidth caps all ISP agreements have "reasonable usage" guidelines.  Residential bandwidth is shared.  No company could offer (even at cost) 1 Gbps for $100 per month if every user (or even a small minority) actually maxed it out.  If you want real pricing take a look at what most datacenters charge for bandwidth.  1 Gbps is going to cost a LOT more than $100 per month, probably more than $1,000 per month (although the cost does get cut in half every 12-18 months).

The last issue is what Nagato mentions above (although his numbers are low due to the need to broadcast to multiple peers).  For miners their outgoing bandwidth is "bursty".  A miner needs to broadcast his found block very quickly to as much of the network as possible.  Every 6 seconds in delay increases the orphan rate by ~ 1%.  If targeting a 3 second window to send a 10 MB block to 50 connected peers in 3 seconds we are looking at 10 MB * 8 bits per byte * 50 peers / 3 seconds = ~1,300 Mbps.   Lower connectivity will put the miner at a disadvantage to better connected miners.  If this barrier is too high you will see even more migration to the largest pools as they can afford the high levels of connectivity needed.  Slower pools will essentially have a 1% to 3% or more "hidden" oprhan tax.  As miners discover that they will migrate to the better paying pools.  


When the average user has "true" 1 Gbps connectivity at a reasonable cost and the average miner can obtain "true" 10 Gbps connectivity then maybe.  BTW despite the post I am bullish on Bitcoin, solutions can be found however those advocating dropping all limits because of "centralization" need to realize at the far extreme it just leads to another form of centralization.  When only a tiny number of players can afford the cost of running a mining pool (and 1, 10, 50 Gbps low latency connectivity) or run a full node you have turned the p2p network into a group of a few hundred highly connected peers.   Guess what modern banking IS a peer to peer network of a few hundred highly connected peers.   The fact that you can't be a peer on the interbank networks doesn't mean the network doesn't exist.  The barriers (legal, regulatory, and cost) just prevent you from becoming a peer.


Today maybe.  But lets look at just a 10MB block for a node with only 8 peers (dangerously low IMHO).  That requires about 64 Mbps sustained.  Due to the bursty nature for this peer to provide any value relaying blocks the peak bandwidth would need to be 10x higher (640Mbps).  The larger obstacle isn't sustained or peak speeds (more than acheivable from a technical standpoint).  The larger obstacle is how much burden it would put on ISP networks (which are generally massively oversubscribed).  Total bandwidth used for this peer is ~350 GB per month.   Most ISP will cap a user long before that.  The biggest ISP, comcast, IIRC starts throttling at ~200 GB per month (less on cheaper plans).  The first time a casual user either has his download speeds cut 80% or gets a warning from his ISP on having to pay overage fees he likely is going to pull the plug.  Maybe not every user but at least some users will.

Thank you DeathAndTaxes and all the others for your technical explanations.

Do you believe that bitcoin will continue to be used for microtransactions as it is today? Or do you think this will fade out over time due to technical limitations?

If you believe it will fade out, do you think another cryptocurrency will be used for microtransactions or do you think a top layer build on bitcoin with off chain transactions, will win out, for microtransactions?

I guess it depends on what you mean by micro transactions.  I mean look at the tempest in a teacup about setting the default dust limit at 5430 satoshis (~0.5 cents).   If you mean <$0.10 (in 2013 USDs) then probably not.  If you $0.10 to a couple bucks then it likely will be some time before those transactions are not economically viable. 

BTW I am not opposed to solving the "block size" problem I am just pointing out that the situation is slightly more complex then some make it out to be.  There is always some level of centralization, an unlimited blockchain simply pushes us towards a different kind of centralization.

How many tx the network eventually happens, what the costs will be, how big Bitcoin gets are all unknown.  My guess is likely no better than anyone elses.  I do believe a alt-coin built from ground up around low cost microtransactions could carve out a niche.  I also think off blockchain transactions aren't that scary. I don't like the idea of web wallets holding massive wealth but I could really careless of the security implications of using off blockchain transactions to buy a cup of coffee or some discount steam game.

That's not the case. If you were online since before that block started to be built, you already received all its transactions. They're all on your transaction pool. There's no actual need to download them again (that's a performance improvement suggested by the scalability page by the way).
To start mining on the next block, all you need is the header of the previous, and a sorted list of transaction hashes to build the Merkle tree. That's much less data then the entire block.

Unless of course the block contains lots of transactions that are not on the memory pool, in that case you'll have to download these unknown transactions.

And there you have it: an easy way to detect if a spamming attempt is in progress. If a sensible amount of transactions in the new block was not present on your memory pool, you should consider that block a spamming attempt by a miner and refuse to mine on top of it, unless of course it's already more than x blocks deep, in which case you accept it (soft limits).
If the spamming miner decides to broadcast his spamming transactions, he'd hit anti-spam fee policies, and end up needing to pay other miners in the network to include its spam.


Mostly everybody agrees with that. The argument is between those that think that an arbitrary formula should be invented and imposed via the protocol, and those who believe that spontaneous order (or p2p, free-market, freedom, pick your term) can implement a better and safer control on block size without the use of a centralized formula. Well, there's also a third group that thinks the 1Mb limit should be kept, but I can't take them seriously...
Not only I believe spontaneous order would reach better results, I also agree with D&T when he says that setting a formula is technically (and pollitically) complicated, and potentially error-prone (might introduce bugs).

Come on, D&T... I know that you know that a node should only need to broadcast a tx to all his peers if he's the very first to receive and validate it. Nodes can first send a "I have this new tx" message, which is small (tx hash size), and then upload the tx to the peers that requested it. Not all of your peers will request it from you - they're connected to other nodes too.

I used the amount 10 in a conservative way... I don't think a node would upload the same transaction 10 times in average, it seems a high number to me.

But it'd be interesting to see statistics on how many times a node has to upload a tx, proportionally to its amount of connections. I never saw any.


I've already answered Nagato above. (and I know that you knew that too...)


I'm confident that spontaneous order can easily tackle block size control. Miners can implement soft limits, not only on block size per se, but also on the percentage of unknown transactions in a block as I said above (normally you should have most transactions of the new block in your pool, if you don't, it might represent a spamming attempt).
Just look at miners today: they're already extra-conservative, only to ensure the fastest possible propagation.


Banking is an industry in symbiosis with the state. The problem with it are the regulations: that's the barrier of entry that makes it so hard to hop in. The cost of the business per se shouldn't be that high. Taking care of people's money (which is mostly digital today) has no reason to be a more costly business than a complex factory for instance.
Just take a look at the amount of competitors that show up in places where banking regulations are less burdensome, like Panama, and compare it with other places (relatively to the country's population and GDP sizes)

Yes there are ways to optimize traffic however they aren't implemented yet so TODAY those limits hold.  Also transmitting a hash (64 bytes) vs transmitting a tx (~400 bytes on average) while smaller isn't an order of magnitude smaller.  So take whatever realistic limit exists based on available resources and even hyper optimized the same limit still exists at 5x that transaction limit.

Don't get hung up on the details.  The point is as tx volume increases bandwidth is the tightest bottleneck.  As nodes can't or won't handle that the number of full nodes will decline.  Higher transaction volume when it breaches what is "reasonable" (and yes there is some gray area on what is reasonable) to the average node will result in a centralization as a result of the costs to run a full node.  Optimizations help but you can't get blood from a stone.   Nobody is going to be able to run a full node on the AVERAGE residential connection with a transaction speed of say 5,000 tps.


There are far more bitcoin nodes today than banks in Panama.  Regulation or not when the burden on a node rises there will be less nodes.  That is a form of centralization.  An open unlimited network "may" reach a good compromise or it may not and this is a billion dollar project.  Their are no "oops" guess it didn't lets hit the reset button.  Given that expanding Bitcoin is somewhat like conducting maintenance on an aircraft in flight it might be worthwhile to move cautiously.   

Transaction hashes are 32 bytes, but even only the first few bytes are enough to identify the tx in the mempool.  So it's actually about two orders of magnitude less data than sending full txs.

This.  Anyone who has worked in the aerospace/airline industry on mission critical systems knows that you don't make changes on gut feelings and optimism.  When people's lives are at stake, you are extremely conservative, assume the worst, triple check, and try to get 100% unanimity on any risky assessment.

The "hard 1MB" position perhaps has some adherents not necessarily out of irrational conservatism or ideological extremism, but to counterbalance people who appear to seriously want to dump the limit altogether in one go, and hope things will just work out, with no empirical evidence beyond thought experiments and forum debates.

In spite of the usual disclaimers, Bitcoin is no longer toy money.  Its stored value is now larger than the GDP of many countries.  Lots of simulation, hard data, and stress testing will be needed before prudent people will go along with changes that risk unintended consequences on a $1B+ economy.

I think most people would agree with that, but it's worth saying here that keeping the ceiling where it is as we get ever closer to it isn't necessarily the cautious move.

To stick with the aircraft analogy, we're flying at 1000 meters towards a 1500-meter-high mountain. If we do nothing, we hit the mountain. There may be some legitimate concerns about whether the plane will be OK at higher altitude, but there are also serious concerns about what happens when the plane hits the mountain. The fact that we've been flying along happily at this altitude over open sea for quite a while before we actually got to the mountain doesn't mean that everything will be OK at the same altitude when we do hit it.

Specifically, we've always had reasonably low transaction fees. We have no idea what would happen to the economy with high transaction fees, and there are some very plausible worst-cases, like Bitcoin losing in the marketplace to an alt-coin or another technology, and most of that $1B+ going up in smoke. The people advocating keeping the limit low are generally optimistic on how great it might be to hit it (we'll land softly in a blossoming forest of off-chain solutions!), and also of the opinion that we're going to hit a mountain sooner or later anyway so it may as well be sooner. They may even turn out to be right. But they're bold, not cautious. The cautious move is not to hit the mountain.

They also tend to drastically underestimate exponential adoption rates. We don't have a few years before we hit the mountain. The right time to start addressing this was last year.

Sigh...
It's not a "hope", it's an aprioristic certainty. You don't need central planning to avoid "nefarious market cartelization", just study economics if you don't believe me.

Talking Bitcoin specifics: it's easy to spot an attempt of spamming by another miner. Its blocks will contain a large percentage of unknown transactions. So, just create soft limits to censor blocks with many unknown transactions. Say, if a block contains more than 10% of unknown transactions, don't build on top of it unless it's already 1 block deep. If it contains >20%, wait for it to be 2 blocks deep etc. Obviously the percentages and depths should be configurable.
You can also add such limits on the block size itself. Larger than X, wait for Y depth at least. Multiple (X,Y) configurable pairs.
Oh, and as a bonus, such soft limits would create an incentive for miners to relay and broadcast transactions they receive - today they have no incentive other than the counter-incentive of having to patch the software. If they keep transactions they receive from SPV nodes for themselves, they might get their block orphaned.

It's quite visible that miners would only slightly raise their limits when they believe the gains from adding more paying transactions would outcome the potential losses from orphanage. That's spontaneous regulation, transaction space supply adapting to its demand.
And it's absurd to claim that a remote, unclear chance of kicking out very low bandwidth miners would be so attractive as to make large bandwidth miners take the risk of losing their entire block on orphanage.

Please, people, being p2p is the greatest feature of Bitcoin. P2P is all about spontaneous order - an actual verifiable fact, not a mere "hope". How can you claim to support the first and largest p2p currency and yet be against spontaneous order?

Except that the block size issue is not akin to free market dynamics.

In a free market, when someone makes a purchase, the only parties directly affected are the purchaser and the seller.

In the blockchain world, when someone publishes a 1GB block, the price is paid by every full node while the reward is only
collected by the publisher. This dynamic is closer to private profit, public loss.

There is much incentive for a well connected miner to publish a large block for 3 reasons
i) He and only he gets more txn fees. The larger the block, the more revenue he gets.
ii) Price for large block is paid by everyone (read: my competition today and tomorrow)
iii) In the event the huge block is not orphaned, he gets a headstart to mine the next block while the rest of the network sits idle trying to download his block. This leads to him having a higher probability than his actual hashing power to find the next block(go to i).

Now lets look at empirical evidence.

P2Pool has an insignificant share of hashing power even though miners get to keep 100% of all earnings vs mining pools which take a cut or txn fees.

Why?
Because the cost of running a full node outweighs the the revenue loss from mining with a pool.
And this is with the average block being <250KB.

There is no question that mining with a pool brings down mining cost because you don't need a fat pipe and a decent PC with a large HD/RAM to run bitcoind.
Decentralised systems have redundancies which leads to much higher costs when compared with centralised systems and the free market may eventually force centralisation in Bitcoin as well. The question is whether we want Bitcoin to centralise towards off-chain solutions for smaller transactions(low blocksize with many miners) or centralise the currency itself to allow on-chain txns for everyone(big blocksize with few large miners).

Personally i think keeping the Bitcoin protocol decentralised to be much more important than keeping its direct transactional capabilities decentralised. However i understand that there is a risk that people will just start using off-chain solutions as money just as paper notes were used instead of gold.

Ideally, the community takes the middle ground and increases the block size slowly to keep pace with bandwidth increases. There is probably an inflection point for bandwidth where blocks will be large enough for the global population, but we are not there yet.

Dude, this a repeated scenario in economic theory. The "fear" that free markets cannot internalize costs when there's the possibility of a tragedy of the commons. It's similar to security in the meatspace.
And guess what? It's perfectly possible to eliminate the tragedy of the commons risk through spontaneous order, as long as property rights are established and respected.

And btw, private profit and public loss only happens when the state comes into the scene.


Nagato, if there are real transactions paying large fees to get included, this represents real demand. Miners better attend it or Bitcoin jams!

And as I explained twice on this thread already, the risk of hitting some soft limits would make miners be prudent on this. They would only increase their blocks when there's enough consensus, or when the demand is so strong that it's worth the risks. In both cases we are fine.


Please, I and many others run a full node without getting nothing in return.

AFAIK P2Pool is not very popular because it allegedly has large stale rates. I don't know if this claim is factual.


Both things will always be the case, if you remove the hardcoded constant limit.


But that's precisely what I'm saying! Block size should be controlled by everybody, with their choices and plannings, not by a centrally imposed formula.

Personally I think the solution is more than one coin.

Bitcoins for storing value long term and bigger payments.

Litecoins for day to day transactions and purchases under $100. 

If nearly all the actual commerce was happening in Litecoins, and Litecoins had solved the scaling problems that Bitcoin hadn't, it's not really clear that people would still want to hold Bitcoins.

The normal thing to do when you run into serious scaling problems is to shard. Sharding is always a big PITA, so you don't do it until you absolutely have to. (Half the payment has confirmed, but the other half hasn't yet, because you're spending coins from two different shards...)

But right now Bitcoin is scaling fine. It's not radically decentralized, and hasn't been for a long time, but that ship sailed when people started mining on expensive, dedicated hardware rather than using the spare CPU cycles on boxes they were running anyway. Capital-intensive commodity businesses have huge economies of scale. They are never cottage industries. You can't run a profitable steelworks in your garage. We may not like these economic facts, but tinkering with irrelevant things like the block size isn't going to change them.