How a floating blocksize limit inevitably leads towards centralization

[Peter Todd]

However, with no limit on block size, it effectively becomes miners who are in control of _everyone_'s block size. As a non-miner, this is not something I want them to decide for me. Perhaps the tragedy of the commons can be avoided, and long-term rational thinking will kick in, and miners can be trusted with choosing an appropriate block size. But maybe not, and if just one miner starts creating gigabyte blocks, while all the rest agrees on 10 MiB blocks, ugly block-shunning rules will be necessary to avoid such blocks from filling everyone's hard drive (yes, larger block's slower relay will make them unlikely to be accepted, but it just requires one lucky fool to succeed...).

Well, read my initial post at the top; that larger blocks don't propegate to the network as a whole is actually benefits the miner because provided the blocks propagate to more than 50% of the effective hashing power, the part that doesn't get the block is effectively wasting their mining effort and taken out of the competition.

Additionally even if miners see a rational reason to keep block-sizes low, which I already doubt, allowing them to control the size gives irrational miners who are trying to actively damage Bitcoin another way to do so. Right now all that an evil miners can do is either help double-spend attempts, easily defeated with confirmations, or launch a 51% attack, so far defeated with large amounts of hashing power. We don't want to give yet more ways for malicious people to damage Bitcoin, especially ones which are actually profitable in the short term.

My suggestion would be a one-time increase to perhaps 10 MiB or 100 MiB blocks (to be debated), and after that an at-most slow exponential further growth. This would mean no for-eternity limited size, but also no way for miners to push up block sizes to the point where they are in sole control of the network. I realize that some people will consider this an arbitrary and unnecessary limit, but others will probably consider it dangerous already. In any case, it's a compromise and I believe one will be necessary.

I mean, I'm not totally against a one-time increase if we really need it. But what I don't want to see is an increase used as a way to avoid the harder issue of creating alternatives to on-chain transactions. For one thing, Bitcoin will never make for a good micropayments system, yet people want Bitcoin to be one. We're much better off if people work on off-chain payment systems that complement Bitcoin, and there are plenty of ways, like remote attestation capable trusted hardware and fidelity bonds, that allow such systems to be made without requiring trust in central authorities.

I would hate to see the limit raised before the most inefficient uses of blockchain space, like satoshidice and coinad, change the way they operate. In addition I would hate to see alternatives to raising the limit fail to be developed because everyone assumes the limit will be raised. I also get the sense that Gavin's mind is already made up and the question to him isn't if the limit will be raised, but when and how. That may or may not be actually true, but as long as he gives that impression, and the Bitcoin Foundation keeps promoting the idea that Bitcoin transactions are always going to be almost free, raising the block limit is inevitable.

Anyway, there are plenty of good reasons to have off-chain transactions regardless of what the block limit is. In particular they can confirm instantly, so no waiting 6 blocks, and they can use chaum tokens to truely guarantee that your transactions are private and your personal financial information stays personal.

[solex]

Great posts from Mike and Gavin in this thread. There's indeed no reason to panic over "too much centralization". Actually, setting an arbitrary limit (or an arbitrary formula to set the limit) is the very definition of "central planning", while letting it get spontaneously set is the very definition of "decentralized order".

Also, having fewer participants in a market because these participants are good enough to keep aspiring competitors at bay is not a bad thing. The problem arises when barriers of entry are artificial (legal, bureaucratic etc), not when they're part of the business itself. Barriers of entry as part of the business means that the current market's participants are so advanced that everybody else wanting to enter will have to get at least as good as the current participants for a start.

Removing the block cap means a hard fork, and once we decided to do that we may as well throw in some "no brainer" upgrades as well, like supporting ed25519 which is orders of magnitude faster than ECDSA+secp256k1. Then a single strong machine can go up to hundreds of thousands of transactions per second.

That's cool. Please core devs, consider studying what other hard fork changes would be interesting to put in, because we risk hitting the 1Mb limit quite soon.

+1
Fully agree.

Further. In the OP an example of slow bandwidth for a political reason was given. We can't let bitcoin be handicapped because of local problems affecting just a small part of the network. It can't operate like a kindergarten going only at the speed of the slowest pupil.

[caveden]

However, with no limit on block size, it effectively becomes miners who are in control of _everyone_'s block size. As a non-miner, this is not something I want them to decide for me.

Only miners (and by that I mean solo miners + pool operators, not pooled miners) need to be a full node. They're your "everyone" there.

Please people, understand one thing: you can't run a full payment network used by millions as a hobby. The day Bitcoin becomes something really serious (i.e., thousands of transactions per second), being a full node will necessarily be a professional task. It won't be something you can do on your laptop, "just for fun".

EDIT: And, as Mike said, the idea of converting Bitcoin into some replacement to SWIFT with $20 fees for transactions, which would force people to use bank-like institutions for daily transfers, just because you want "ordinary people to verify transactions", totally turns me off. Bitcoin can be much more than that. If you actually want it to remain this censorship-resistant currency that it is, it has to remain suitable for small transactions like buying some plugin from Wordpress. If you want Bitcoin to remain an alternative for those seeking financial privacy, you have to keep it suitable for SR users and alike - otherwise all these "bank-like" payment processor would ruin your privacy. If you want Bitcoin to remain an alternative for those trying to protect their purchasing power from inflation, you have to keep it suitable for those who want to protect their daily money on their own, without having to use a bank just for storage purposes which would recreate the incentive for fractional reserves. The list can go on. Bitcoin has the potential to be much more than SWIFT 2.0. But for that, processing transactions will have to become a professional activity (it kinda already is actually).

But maybe not, and if just one miner starts creating gigabyte blocks, while all the rest agrees on 10 MiB blocks, ugly block-shunning rules will be necessary to avoid such blocks from filling everyone's hard drive (yes, larger block's slower relay will make them unlikely to be accepted, but it just requires one lucky fool to succeed...).

Succeed in what? Killing everybody else? Do you realize that would likely require more than 50% of the network processing power, otherwise the "unacceptably-gigantic" block would always be an orphan? Miners would likely reject blocks way too large, specially if it's filled with transactions never seen before (i.e., a likely attempt of flooding).

There is of course wide spectrum between "I can download the entire chain on my phone" and "Only 5 bank companies in the world can run a fully verifying node", but I think it's important that we choose what point in between there is acceptable.

I'm glad you see it's not so black-and-white. I'm sad though that you think you can actually "choose what point between there is acceptable". Such point cannot be found arbitrarily, even because it is not fixed: it varies all the time according to different demands for security, transaction space, speed etc, and different supply of resources. Basically, you'll never be able to even list all data that is necessary to take such decision, as it involves subjective opinions of thousands and eventually millions. Let alone calculate anything.

The curious task of economics is to demonstrate to men how little they really know about what they imagine they can design.

Then I think you misunderstand what a hard fork entails. The only way a hard fork can succeed is when _everyone_ agrees to it.

Only if you want to totally avoid the fork actually. Otherwise, you may have both chains living in parallel for some time. Then eventually one might "beat" the other, as people migrate. Or both keep alive "forever", that's a possibility too.
The more I see these discussions, the less I believe in totally avoiding the fork.

EDIT: And also, as a general comment on the discussion, you people fearing "too much centralization", as in "too few market participants", should realize that, at most, what would happen would be a few pool operators, like we have now. Pool operators do not own the processing  power. Such processing power will remain scattered among thousands of people, who may easily migrate to different pools if they feel like. Pretty much like what already happens. Current pools need to have some "professional bandwidth" if only for protecting against DDoS, It already require professional resources to run a mining pool.

[cjp]

I mean, I'm not totally against a one-time increase if we really need it. But what I don't want to see is an increase used as a way to avoid the harder issue of creating alternatives to on-chain transactions. For one thing, Bitcoin will never make for a good micropayments system, yet people want Bitcoin to be one. We're much better off if people work on off-chain payment systems that complement Bitcoin, and there are plenty of ways, like remote attestation capable trusted hardware and fidelity bonds, that allow such systems to be made without requiring trust in central authorities.

This is exactly how I see it too!

In this thread I'm a bit disappointed in Gavin. I used to see him as a very conservative project leader, only including changes when there's community consensus about it and no doubt about its security implications. And I liked that, even though it meant that some of the changes I support are not going to be included. For a monetary system, trust and stability are essential, and I hope Gavin will continue to provide that trust and stability, so hopefully he just considers abandoning the transaction limit as an academic "thought experiment", and not something he is planning to actually put into the code in the near term.

I'm afraid Gavin still has this "low-fee micropayment" idea of Bitcoin. Gavin, if you read this: please abandon that idea. It stops you from seeing clearly where Bitcoin is useful and where it is not useful, so it stops you from developing Bitcoin in a direction that will make it most successful. If you steer towards "low-fee micropayment", you will find yourself in a niche with more efficient competing technologies, where Bitcoin will die. Bitcoin's strength is in store of value; the bulk of microtransactions can and should be done off-chain with other systems.

[Mike Hearn]

I think there are some things we can all agree on.

We're all keen to see efficient protocols built on top of Bitcoin for things like micropayment channels (which allow lots of fast repetitive satoshi-sized payments without impacting the block chain), or trusted computing (which allows offline transactions to be carried around in long chains until final resolution). Also the payment protocol should eliminate the most absurd abuses of micropayments like SDs messaging system. These things fall into the class of "no brainers" and were discussed for a long time already.

Other more exotic ideas like Ripple-style networks of payment routers using contracts don't seem against the spirit of Bitcoin if they keep the low trust aspects of the system.

At the same time, as evidenced by the disagreement on this thread, there are too many unknown variables for us to figure out what will happen ahead of time. The only way to really find out is to try it and see what happens. If Bitcoin does fail to scale then the end result will be a smaller number of full nodes but lots of people using the system - this is still better than Bitcoin being deliberately crippled so it never gets popular because even if the number of full nodes collapses down to less than 1000, unknown future advances in technology might make it cheap enough for everyone to run a full node again. In the absence of a hard-coded limit the number of full nodes can flex up and down as supply and demand change. But with a hard-coded limit Bitcoin will fail to achieve popularity amongst ordinary people and will eventually be forgotten.

And now some specific replies:

Heck, while we're playing this game, find me a single major O(n^2) internet scaling problem that's actually been solved by "just throwing more hardware at it", because I sure can't.

I already gave you one. The internet backbone is a great analogue for Bitcoin because you have billions of users who connect up hierarchically through to the backbone which is a P2P broadcast network - exactly like what we have where users will run SPV nodes and connect up to a backbone of full nodes.

You say, oh, but people were worried about routing table scalability. They took measures to improve things. Yes, of course they did, just like we're doing with Bitcoin. Nobody is arguing that scalability improvements can be ignored, far from it. Along the way there will be problems and bottlenecks and rewrites and protocol improvements to change the slopes of the graph, but fundamentally the backbone scaled because routers got a lot more powerful (in particular, high speed RAM got a lot cheaper).

And when the internet backbone hit its own "block size limit" (the 16 bit ASN width), they expanded it to 32 bits. There was no talk of backbone scalability being an O(n^2) problem and who needs more than 65k networks anyway. They just bit the bullet, grew the system and here we all are today sending tiny messages over it. Success!

It's interesting you say that, I'm guessing you don't think there was a need for a Bitcoin Foundation then do you?

For funding development? Not really. But I am glad Gavin has the certainty of a salary rather than having to raise money all the time via assurance contracts. Also, giving him a salary means he has free reign to work on what he thinks is most important rather than what community members feel like funding, which is important when so much of the work he does is somewhat intangible (like testing infrastructure).

The Foundation is however very important for presenting a professional and united front to organizations that are genetically programmed to require one, like governments or the media. It's also a good organizational focal point. So I am a member for that reason.

[Peter Todd]

Please people, understand one thing: you can't run a full payment network used by millions as a hobby. The day Bitcoin becomes something really serious (i.e., thousands of transactions per second), being a full node will necessarily be a professional task. It won't be something you can do on your laptop, "just for fun".

Right now the capacity of Bitcoin is about half a million transactions per day. So you can participate in that level of transactions as a hobby. The value of those transactions can be as high as required. If Bitcoin does become a widespread store of value, blocks will probably be transferring hundreds of million of dollars worth of value each, tens of billions every day.

But after all, it's just information, so yes, participating will be perfectly possible as a hobby, and for a fairly affordable fee, you'll be able to even make transactions directly on the world's decentralized value transfer service, the same system big banks will use.

EDIT: And, as Mike said, the idea of converting Bitcoin into some replacement to SWIFT with $20 fees for transactions, which would force people to use bank-like institutions for daily transfers, just because you want "ordinary people to verify transactions", totally turns me off. Bitcoin can be much more than that. If you actually want it to remain this censorship-resistant currency that it is, it has to remain suitable for small transactions like buying some plugin from Wordpress. If you want Bitcoin to remain an alternative for those seeking financial privacy, you have to keep it suitable for SR users and alike - otherwise all these "bank-like" payment processor would ruin your privacy. If you want Bitcoin to remain an alternative for those trying to protect their purchasing power from inflation, you have to keep it suitable for those who want to protect their daily money on their own, without having to use a bank just for storage purposes which would recreate the incentive for fractional reserves. The list can go on. Bitcoin has the potential to be much more than SWIFT 2.0. But for that, processing transactions will have to become a professional activity (it kinda already is actually).

Absolutely. But the solution isn't to make access to the core Bitcoin network, the thing that actually keeps Bitcoin inflation free and secure, require such a huge investment in computer hardware that only big banks and other large institutions can afford access. The solution is to keep blocks small, and build payment systems that work on top of the block chain.

Remember that if the blockchain is kept small enough that validating it is affordable, you don't have to trust the payment processors very much. The protocols will be designed in ways that allow anyone to prove fraud automatically and warn the whole world. The client software people use will see these fraud proofs, and immediately stop using the payment processor, putting them out of business. Yet at the same time, using technologies like chaum tokens, those payment processors can't even know where payments are going too; you're privacy is even more protected than with on-chain transactions, because the links connecting one transaction to another are severed with unbreakable mathematics.

Do you think the banking crisis would have happened if banks were forced to have all their bank-to-bank transactions publicly recorded for the whole world to see? Keeping the blocksize limited does exactly that.

But maybe not, and if just one miner starts creating gigabyte blocks, while all the rest agrees on 10 MiB blocks, ugly block-shunning rules will be necessary to avoid such blocks from filling everyone's hard drive (yes, larger block's slower relay will make them unlikely to be accepted, but it just requires one lucky fool to succeed...).

Succeed in what? Killing everybody else? Do you realize that would likely require more than 50% of the network processing power, otherwise the "unacceptably-gigantic" block would always be an orphan? Miners would likely reject blocks way too large, specially if it's filled with transactions never seen before (i.e., a likely attempt of flooding).

Ok, so a 10GiB block is unacceptably large. What about a 5Gib block? Or a 1GiB block? Or a 500MiB block? At some point the block will be confirmed by a large fraction of the hashing power, but not all the hashing power. The hashing power that couldn't process that gigantic block in time has effectively dropped off of the network, and is no longer contributing to the security of the network.

So repeat the process again. It's now easier to push an even bigger block through, because the remaining hashing power is now less. Maybe the hashing power has just given on on Bitcoin mining, maybe they've redirected their miners to one of the remaining pools that can process such huge blocks, either way, bit by bit the process inevitably leads to centralization.

EDIT: And also, as a general comment on the discussion, you people fearing "too much centralization", as in "too few market participants", should realize that, at most, what would happen would be a few pool operators, like we have now. Pool operators do not own the processing  power. Such processing power will remain scattered among thousands of people, who may easily migrate to different pools if they feel like. Pretty much like what already happens. Current pools need to have some "professional bandwidth" if only for protecting against DDoS, It already require professional resources to run a mining pool.

Pool operators do own hashing power if the miners contributing the hashing power can't effectively validate the blocks they mine.

If running a validating node requires thousands, or even tens of thousands, worth of expensive equipment, how exactly do you expect to even find out that you've been mining at a dishonest pool? If >50% of the people mining and running validating pools decide to get together and create bogus transactions creating coins out of thin air, you won't even know they've been defrauding everyone.(1) If running a node requires tens of thousands of dollars worth of equipment, and it will to support Visa-scale transaction volumes, only a small handful of large banks are going to run nodes. I think you can see how collusion between half-a-dozen large banks becomes not just possible, but likely.

1) Yes, you can try to create automated fraud proof mechanisms to detect it - I wrote about the idea here - but implementing the software to process fraud proofs is extremely complex, much more complex than applying the same idea to keeping off-chain banking services honest. I also have little hope that those mechanisms will actually get written and tested before the much more simple step of just lifting the block limit is taken.

In this thread I'm a bit disappointed in Gavin. I used to see him as a very conservative project leader, only including changes when there's community consensus about it and no doubt about its security implications. And I liked that, even though it meant that some of the changes I support are not going to be included. For a monetary system, trust and stability are essential, and I hope Gavin will continue to provide that trust and stability, so hopefully he just considers abandoning the transaction limit as an academic "thought experiment", and not something he is planning to actually put into the code in the near term.

I agree %100. Increasing the block limit seems like a conservative change - it's just one little number - but the long-term implications are enormous and have the potential to drastically change what Bitcoin is. It may be a conservative change for the small number of big businesses that are heavily invested in the current system, and can afford the network power to process large blocks, but it's not a conservative change for the rest of us.

[gmaxwell]

... and at this point your (and gmaxwell's) imagination seems to run out, and you throw up your hands and say "We Must Limit Either T or P."
Really?

This may, in fact, be the first time in my life that I've been accused of not having enough imagination. First the grey hairs, now this... I guess I'm finally getting old.

But perhaps you've not been paying attention where I've been throwing ideas against the wall. My imagination has far from run out. (Nor has PeterTodd's, as he's independently come up with some of the same ideas— as well as many that I haven't thought of...)

…Not to mention a zillion other less workable ideas that are waiting for some breakthrough or another to make viable… mostly I try to spare the public the more inane stuff... and certainly I've had no shortage of imagination for all the technical and economic failure modes that can result from removing part of the scarcity that drives a cryptocurrency.  We've even had non-blockchain scarce altchains— Liquidcoin (for speculation)—, perhaps you don't remember because it imploded into complete non-convergence before half the people who wanted to run it even got it installed (though its 'innovation' was uncapping the block rate instead of uncapping the blocksize).

Generally the ideas to improve Bitcoin's scalability seem to only change around the constant factors in costing equation— N*∝N is still quadratic even if you multiply in only a fraction validating. Or at least, so far, the in-blockchain-ideas don't allow unbounded scaling (as blockchain external thing like fidelity chaum-banks, however, do). As far as I can tell, if we're to have a system which is practically decentralized and secure— then there must be limits for technical and economic reasons.  I _hope_ that if we're clever enough we can improve scale without compromising decentralization, but examples from existing currencies and the state of the technology make me more confident in both centralized and decentralized off-chain approaches— which we'll have to have no matter what happens with blockchain scaling.   (And it bears bolding out: Off-chain DOES NOT MANDATE CENTERALIZED, though it seems that plenty of people love centralization, and its hard to argue against for low value things).

I think we should put users first. What do users want? They want low transaction fees and fast confirmations. Lets design for that case, because THE USERS are who ultimately give Bitcoin value.

I want a pony. Made of cake. Special healthy cake that makes you lose weight. If I can choose without compromise of course I'll want all the features and none of the costs.

The users of Bitcoin— with their understandable desire for fast transactions and low fees— are, above all, users of _BITCOIN_.  If fast transactions and low fees were their utmost priority they would be using VISA and the USD, which among numerous other advantages, have deep and fundamental advantages over direct Bitcoin transactions for achieving low transaction costs and high speed.  Decentralization is costly as heck, and if you punt on it you can scale much better (go look at what ripple is doing— even though it has gone the rather uninspired blockchain-ish global consensus like bitcoin route— it has replaced a scarce resource consumption consensus with trusted verifiers that produce blocks as fast as they achieve a majority).

Blockchains are just simply bad at fast and cheap. They are a zero information hiding global flooding network (==costly) that depends on achieving a worldwide consensus (==slow). When a butterfly in china buys a dumpling with Bitcoin, processors in Nebraska must whirl, and everyone must wait until we've had time to hear back about all the whirling or risk having their transactions reversed. And in spite of being slow, costly, and with a heaping helping of the market-failure-maker of externalized costs... it turns out that that blockchains can actually do a whole lot. An O(N^2) algorithm can be quite fast when N is small. And with enough thrust (compromise) even a pig can fly (scale). But we shouldn't mistake that for scaling without compromise— because, at least as of yet, no one has figured out how.

I believe that Bitcoin plus a nice mixture off-chain transaction processing can be all things to all people: It can provide unbreakable trust based on cryptographic proof, it can provide infinitely scalable transactions. The chain doesn't scale great, but it can scale enough to handle high value transactions (I pay $30 for an international wire today) and scale enough to interlink infinitely scalable additional mechanisms.  People can pick their blend of security vs cost by choosing how they transact.  If, instead, we try to make the blockchain do it all directly, we get nothing: We lose the zero-trust but would still not be as fast or cost effective as systems which don't pretend to be decenteralized. In doing so we'd force people— against their will and interests— to accept the system changing out from under them and forcing them to trust other parties. They may ask rightfully ask— "When fast and cheap transactions 'require' pegging the subsidy and creating inflation, will respecting the promises that were made to me once again be considered negotiable for the greater good because I'm not a core developer or big mining bank?"

... But at the same time, this is all greatly benefited by making Bitcoin scale better.  Any scaling improvement is equally a improvement of decenteralization at a given level of scale, and I absolutely am working my brain overtime on how to make it better, and open to listening to every idea presented.

Decenteralization— offering an alternative monetary system which doesn't require all that messy and misplaced trust— is the fundamental purpose of Bitcoin: it is the one thing Bitcoin does which nothing else can hold a candle to.  It is messy and political and, at times, morally ambiguous. It is a fantastic experiment. It is the BIG IDEA that makes this all worth doing. This is why we're all here. But, it doesn't always neatly translate into quarterly profits for Bitcoin startups, or simply explained day-to-day benefits, it drags in messy politics when you just want to shut up and code. Of course, Bitcoin actually has to _work_ for people but if we can't make it work and be decenteralized we might as well give up and go home.  The trust in classical currencies isn't eroded because of evil— it's eroded by successive application of pragmatic local decision making which leaves people with nothing they can count on to hold true for long.  Bitcoin might show us a way out of that— with its awful suicide pact of inhuman razor-sharp mathematics made nearly immutable through decentralization— but only if we're ready to deal with the consequences.

At the same time, as evidenced by the disagreement on this thread, there are too many unknown variables for us to figure out what will happen ahead of time. The only way to really find out is to try it and see what happens

That I can agree with— but I think not in the way you intended it. Some of the argument about the uncapping risk are mooted if you only increase the maximum blocksize once its apparent to everyone the transaction load is some multiple of it. This way you don't get the fees race to the bottom.   You argue that removing the limits is okay because hopefully we won't commit suicide, I counter that it's even more okay to not back them off them until its apparent to all that they must be backed off.  Without a credible claim for long term decentralization Bitcoin will still retain many of its flaws but have no selling point to the ordinary people you wish to attract. We can't have both by uncapping the chain, but we can have both with the additional systems.

Only miners (and by that I mean solo miners + pool operators, not pooled miners) need to be a full node. They're your "everyone" there.

What you're describing there is a distributed system, not a decenteralized system. If only some small number of wealthy nodes do all the validation— instead of pools, why not call them Central Banks?— then everyone else is forced by economic reality to trust them to behave honestly, and there is no economic force beyond a consensus of a small number of like-motivated-parties not to make helpful "optimizations" according to earnest beliefs the greater good or in response to coercion.

In Bitcoin, as it exists today, running a full node is dirt cheap and would remain dirt cheap forever— unless and until we have a hard fork with the consent of the users— and so you're not forced to trust anyone or anything beyond your ability to audit source code and your understanding of software and mathematics.  All modern banking is distributed, and Bitcoin as a distributed system isn't especially inspired. It's fragile and has high overhead.  In site of this, because of decentralization we have something better. I hope we don't lose it because the rush to make MAD MONEY with speculation has left us with too large of a community who doesn't realize how powerful the ideas behind the system actually are— and how those ideas are the fundamental that make Bitcoin valuable, not how many transaction per second the system can handle (how many transaction per second does a gold bar process?).

[gmaxwell]

In this thread I'm a bit disappointed in Gavin. I used to see him as a very conservative project leader, only including changes when there's community consensus about it and no doubt about its security implications.

I'm not disappointed. Although I don't currently agree with Gavin on this, I believe he has clearly behaved in a  praise-worthy manner: His responses are understated and considerate— which must be difficult where the opposition (e.g. me) might be read as saying "OMG YOU'RE CENTERALIZING BITCOIN YOU MONSTER". This is hard stuff with long term consequences that none of us can truthfully say we completely understand and, at the moment, without the kind of immediate urgency (or even running code) which would help crystallize thinking.

Above all other things, I'm glad that he's taking some time to spend adding some thoughts to this when it isn't an immediate issue.  A lesser, not-consensus-driven developer would just rest quietly comfortable that there would eventually be an emergency that would allow pushing their way though... Gavin isn't like that, which is why you even get to hear what he's thinking on this when it's still just a philosophical debate.

[Ari]

So...  I start from "more transactions == more success"

I strongly feel that we shouldn't aim for Bitcoin topping out as a "high power money" system that can process only 7 transactions per second.

If bitcoin operated at the scale of PayPal, ~87 transactions per second, that would require roughly 20MB blocks.  That's large, but not unmanageable.

People paid PayPal US$ 5.6 billion in fees last year.  There's no way the bitcoin network is going to cost that much.

[Technomage]

Very interesting discussion. Similar debates have been going on for a long time for sure, and there is clearly some conflict on what Bitcoin is supposed to be.

Personally I find it ridiculous that we would even be thinking of handicapping Bitcoin simply because it allows us to be "maximally decentralized". I have long since accepted the fact that not everyone will be able to run a full node, nor can everyone achieve profitability in mining. Being less centralized does not mean it's not decentralized. I believe there will have to be some sort of a compromise.

We should not accept that Bitcoin is going to become this network for high value transactions alone. That is basically the last option. Bitcoin is about decentralization, yes, but it's also a tool for making _small_ international payments affordably. With that being said, I will comment on some specific issues relating to the debate.

First of all, being a full node and being a miner is already more restricted than it was before. Running a full node is already a pain in the ass (0.8 helps, true, but still), GPU mining is becoming a thing of the past, etc. Bitcoin backbone even now isn't something everyone can help maintain, it's quite understandable that it will be even less so in the future. This doesn't mean Bitcoin will become centralized, that is simply not true. It will become more centralized, but that is different.

I find it ridiculous that we would intentionally handicap the technology so that everyone can fully help maintain the backbone of the network, or keep miners on equal basis. Miners are never on equal basis. Some people have more know-how and they tweak their hardware and software to better performance. Some have lower electricity costs. In the future the bandwidth will also be a factor, so what?

The whole bandwidth and storage question is mind-bogglingly stupid. I've had a 10mbps+ connection for over 10 years. In fact almost everyone in this country can get a 100mbps connection fairly cheaply, which is something I have now. I used to have 200mbps. Gigabit home connections are basically around the corner. Storage space is getting cheaper every day. CPU is apparently not the problem, so what is the problem? I have not seen any proof that Bitcoin is becoming a "bank only" technology, I have not seen that these higher requirements amount to needing a super computer.

So in conclusion, running a full node, or being a competitive miner, might have increased requirements in the future. So fucking what? Mining is a highly competitive market, isn't that supposed to happen? If Bitcoin becomes as big as PayPal, is running a full node something everyone should be able to do? Of course not, that is ridiculous. Handicapping Bitcoin so that everyone can, is even more ridiculous.

All of this being said, I'm not sure the block limit should be lifted entirely. There are many issues with it, one being the transaction fee structure and the entire profitability of mining (which is crucial). I believe making conservative raises in the limit should be the goal for now, until a robust method of automatically adjusting block limit has been agreed upon. An automatically adjusting block limit is the best solution long term, but until we have a good method for doing that, it should not be done.

Finally, I have a few things to say about the profit motive. Zeilat is the only one who got it right in his first post. Miners have a great incentive to not fuck with the transactions and screw with the blocks. The more they do that, the more it would handicap Bitcoin usage, thus likely leading to a decrease in Bitcoin price, which would in turn decrease profits. That's profit motive for you guys.

The whole profit question is often like the blind leading the blind. The only situation where there would likely be significant problems is if miners can 1) get very significant short term advantage by screwing things up and 2) not get caught and 3) have a low IQ. If this is the case, I can admit there is a problem. Otherwise I don't see it. Especially when we're talking about a more centralized mining network. That means there will be more and more actual mining companies and major projects, which are concerned for their longer term profits as well.

Making more on the short term just to fuck up the long term doesn't make much business sense, does it? Especially if the short term advantage isn't enormous and if the miner is big enough to actually know that his own actions will affect the network in itself. It would not be rational, it would be in fact against the profit motive.

From what I've seen, a large majority of miners are thinking long term and are fairly rational, which leads me to the conclusion that short term profit exploits are not such a major concern that we should put too much weight on them when thinking about how we develop the technology.

[Technomage]

Making the system less usable by artificial restrictions while allowing more people to maintain it. This is what we're basically talking about. Just making sure everyone knows.

As I said though, the block size limit is a difficult question because there are other issues with it besides the requirements of being a full node, and the status of the mining market. Personally I think that the incentive of mining itself is in question long term if there is no scarcity at all with the block sizes, but I could be wrong. That is basically the final test of proof of work, and basically a different issue, but still.

[TierNolan]

So we need a maximum block size that is high enough that the vast majority of nodes are comfortable with it, and isn't so big that it can be used to manipulate the difficulty by artificially slowing propagation accross the network with massive blocks. With the help of the maintaining of the propagation window through it's difficulty, we may be able to determine whether the propagation of blocks is slowing and whether the max_blocksize should be adjusted down to ensure the propagation window remains stable.

A measure of how fast blocks are propagating is the number of orphans.  If it takes 1 minute for all miners to be notified of a new block, then on average, the number of orphans would be 10%.

However, a core of miners on high speed connections could keep that down and orphans are by definition not part of the block chain.

Maybe add an orphan link as part of the header field.  If included, the block links back to 2 previous blocks, the "real" block and the orphan (this has no effect other than proving the link).  This would allow counting of orphans.  Only orphans off the main chain by 1 would be acceptable.  Also, the header of the orphan block is sufficient, the actual block itself can be discarded.

Only allowing max_block_size upward modification if the difficulty increases seems like a good idea too.

A 5% orphan rate probably wouldn't knock small miners out of things.  Economies of scale are likely to be more than that anyway.

Capping the change by 10% per 2 week interval gives a potential growth of 10X per year, which is likely to be at least as fast as the network can scale.

So, something like

Code:

if ((median of last 2016 blocks < 1/3 of the max size && difficulty_decreased) || orphan_rate > 5%)
 max_block_size /= 8th root of 2
else if(median of last 2016 blocks > 2/3 of the max size && difficulty_increased)
 max_block_size *= 8th root of 2 (= 1.09)

The issue is that if you knock out small miners, a cartel could keep the orphan rate low, and thus prevent the size from being reduced.

[rini17]

Ok, so a 10GiB block is unacceptably large. What about a 5Gib block? Or a 1GiB block? Or a 500MiB block? At some point the block will be confirmed by a large fraction of the hashing power, but not all the hashing power. The hashing power that couldn't process that gigantic block in time has effectively dropped off of the network, and is no longer contributing to the security of the network.

So repeat the process again. It's now easier to push an even bigger block through, because the remaining hashing power is now less. Maybe the hashing power has just given on on Bitcoin mining, maybe they've redirected their miners to one of the remaining pools that can process such huge blocks, either way, bit by bit the process inevitably leads to centralization.

This requires conspiration of biggest pool owners with majority of hash power. But if such conspiration is going to happen, it will result in abuse of current rules (and any updated rules) anyway. So I consider this argument a strawman. With any competition, issuing 5GiB block out of the blue takes serious risk of it becoming orphaned and Gavin's 5-second proposal supports exactly that.

[markm]

Bitcoin is just the primary chain in a whole family of merged-mine-able blockchains; it provides, primarily, the world's most difficult proof of work.

Anything other than providing the world's most difficult proof of work can be delegated out to one or more of the secondary chains, most of which currently seem to in any case be mostly "backed" by bitcoins thus not only have a vested interest in keeping the world's most difficult proof of work difficult but also in keeping the coins/tokens of value of that primary chain, which provide most of the funding for the massive proof of work, valuable.

More transactions per day, faster transactions, anything other than sheer proof of work power and sheer (store-able?) value (or, at least, some means of funding that difficult work) are secondary thus can reasonably be delegated to secondary chains, surely?

-MarkM-

[gmaxwell]

Making the system less usable by artificial restrictions while allowing more people to maintain it. This is what we're basically talking about. Just making sure everyone knows.

"Making"? The system already has the scarcity built in, and the suggested removal of it would require an incompatible hardfork no technically different than changing the inflation schedule. Bitcoin has never been changed like that.

If you're going to go for the hyperbole: I'll accept that and raise you—  Violating the contractual agreement embodied in the Bitcoin software in order to remove some of the scarcity which is currently necessary for making the act of honestly expending resources to secure it economically rational. This would be a change which has a universally anticipated consequence of increasing centralization, forcing an unknown number of participants to trust third parties, and removing all reason from the long future security arguments of the system... Peter outlines an argument that the equilibrium of an uncapped blocksize is a single miner. All for the sake of increasing short term transaction capacity when we are no where near utilizing the levels currently available and when that capacity could be provided by other means. This is what we're basically talking about.

... I think this is horribly over-stated, but it's at least more pedantically correct than suggesting that anyone is proposing adding any limitations.

This requires conspiration of biggest pool owners with majority of hash power.

Huh? It's the arguments that the pools (note: miners are not even invoked here, the idea that the authority is handed over to a couple pools is already so easily accepted) will reject "too large" blocks that demands a conspiracy.  Peter Todd's point is that a single miner can push off all the smaller ones, all on their own— unless there is a conspiracy to stop them from including these otherwise valid blocks.  As you argue such a conspiracy "will result in abuse of current rules", and I agree.  Worse, if we as a community can't figure out a criteria to conspire on and put it into the system— where there is no risk of defection— how can we expect a pool-conspiracy to do better when they have less powerful tools to enforce conformance?  Also, while the network is choking on blocks which will be rejected users are left around waiting longer times for confirmation because its not obvious when a chain is going to get orphaned because it violated the invisible cartel rules.  Finally, even if the cartel is successful at preventing single miners from breaking the system, its in each member's individual best interest to mine the largest possible blocks, so we should expect a slow evaporation which has the same conclusion: a single primary miner/validator.

A measure of how fast blocks are propagating is the number of orphans.

Not quite— only if you assume miners don't directly peer (either intentionally or by connecting to everyone). They do.  Besides— a single megaminer has no orphans. Miners and non-mining users are different, counting orphans only count the impact on miners. If you disenfranchise the non-miners from validating you're handing control to a special interest group with high fixed costs— the kinds of blocks you can get away with mining are very different if you're trying to appease all users broadly vs just miners.

Using orphans may well be a valid way to dynamically control block speed without a fixed time constant, but it doesn't address the concerns around the economic motivations of mining, or the disenfranchisement of the non-mining users whos voice is embodied by the strict rules of the system which constrain miners absolutely no matter how powerful a miner conspiracy they have.

Anything other than providing the world's most difficult proof of work can be delegated out to one or more of the secondary chains,

I'm interested in hearing more thoughts about how to safely integrate other blockchain like systems... e.g. what protocol extensions they need to share more security and facilitate cross chain trades. I've generally been a bigger fan of non-chainlike alternatives because I think the diversity has the most potential to make a big marginal improvement— but I also think there is room to add value with chainlike systems too.

[Technomage]

More transactions per day, faster transactions, anything other than sheer proof of work power and sheer (store-able?) value (or, at least, some means of funding that difficult work) are secondary thus can reasonably be delegated to secondary chains, surely?

Where do you think the value in bitcoins comes from? They have no use beyond their capabilities as a medium of exchange and store of value. What happens to the value of bitcoins if the ability to use bitcoins as a medium of exchange is severely handicapped? Not good things, I'd reckon.

It is true that Bitcoin can be used together with various other means and maybe that is how it will eventually be, but that is not the most efficient goal. Bitcoin has the promise of being universal money, which means it can be more or less, a jack of all trades. There is obviously a large amount of skepticism regarding this. I agree that it may not be possible.

However, I do not agree in not trying. It is a massive step down in efficiency if Bitcoin has to be used together with other systems to be usable for anything else than value storage. That is not what we should be aiming for, far from it.

[Technomage]

"Making"? The system already has the scarcity built in, and the suggested removal of it would require an incompatible hardfork no technically different than changing the inflation schedule. Bitcoin has never been changed like that.

If you're going to go for the hyperbole: I'll accept that and raise you—  Violating the contractual agreement embodied in the Bitcoin software in order to remove some of the scarcity which is currently necessary for making the act of honestly expending resources to secure it economically rational. This would be a change which has a universally anticipated consequence of increasing centralization, forcing an unknown number of participants to trust third parties, and removing all reason from the long future security arguments of the system... Peter outlines an argument that the equilibrium of an uncapped blocksize is a single miner. All for the sake of increasing short term transaction capacity when we are no where near utilizing the levels currently available and when that capacity could be provided by other means. This is what we're basically talking about.

... I think this is horribly over-stated, but it's at least more pedantically correct than suggesting that anyone is proposing adding any limitations.

I think you know well that the original creator of Bitcoin, Satoshi, has clearly stated that he intended for the block limit to be raised or removed eventually. This means that for all intents and purposes, Bitcoin was not complete regarding this issue.

It's pretty much the same thing as the fee structure put place in Bitcoin. Everyone knows it's not the final system, it was just something that was put there to make it work for now.

This shouldn't be compared with changing the inflation schedule. That is absolutely false. The inflation schedule is a part of Bitcoin that is complete, and has been complete, ever since Bitcoin started. I also think that pretty much everyone agrees with that.

[Technomage]

I just don't like the fact that there is deliberate attempt to steer Bitcoin towards being a gold-like settlement currency. It can retain a high value like that, true, because it still offers significant advantages to gold. However, it would lose all appeal as a day to day money transfer mechanism.

Maybe centralized systems need to handle that, or other blockchains, that is possible. The problem I have is that I see no problem in higher requirements to run a full node. Nor do I see a problem with the mining market, which is bound to become more centralized regardless (thanks to the cost structure of ASIC manufacturing). Based on the posts in this thread there are many possible methods of making it difficult for miners to start seriously screwing it up too much.

I can accept that Bitcoin won't be used for super micro transactions, that can be done elsewhere. With this I mean transactions worth cents. However, paying the equivalent of $1 very cheaply, internationally, should and needs to at least be a goal for Bitcoin. That's what I think.

[ElectricMucus]

For the blocksize limit removal proponents: What are you afraid of?

That you cannot make transactions for basically free anymore? Don't you get it?
The limit is there for a reason... some of which were already mentioned in this thread.

Maybe once we reach the limit there will be finally some steady income for miners (read not related to the initial block reward).
If the limit is reached consistently there would finally be a market for transactions which wasn't there before. You preach free markets and shit, but when it comes down to it you are afraid of them.

[markm]

Where do you think the value in bitcoins comes from? They have no use beyond their capabilities as a medium of exchange and store of value. What happens to the value of bitcoins if the ability to use bitcoins as a medium of exchange is severely handicapped? Not good things, I'd reckon.

The ability to frivolously use the primary chain being "severely handicapped" need not be a "bad thing", it could actually be a "good thing". Blockchains are not ideal anyway as payment networks, so trying to compete in the payment network space is probably doomed to failure from the start. Storing value securely and moving significant value securely should not be sacrificed for the ability to tell the whole world about every penny-ante gambler's every wager and every child's purchase of a popsicle.

Let the transaction fees on the primary chain go up to whatever the market pushes them to, lower value transactions can use the first secondary chain for its presumably lower transaction fees (possibly also with lower security if less miners bother to merge that chain), people wanting to transact for free can migrate to the least of the secondary chains as long as there remain enough chains in the family for one or more of them to still offer space for free transactions.

Throw in Ripple to make exchanges between the chains transparent, or clients that do atomic cross-chain exchanges, to hide the underlying many-chains "complexity" from users who prefer not to think about the nitty gritty technical details of how money is implemented. Add more chains as needed.

-MarkM-