review of proposals for adaptive maximum block size

[mughat]

Then maybe it really is best to just let whoever can manage to build petabyte blocks do so as soon as possible to raise the bar as high as possible as fast as possible?

Its not like there aren't alt-chains nations of various sizes, technology-levels and industrial base can use if the G7's ideal uber-network is a bit too much for them to install/use domestically.

-MarkM-

You don't make sense.
There should be a limit to the growth rate but it makes sense to let the miners that do the work deside what direction and how fast it should increase within some limits that prevents an attacker messing to much.

The miners have incentives to do the right thing so why not let them do it?

[markm]

Huh? Isn't that what I just said?

Let them use petabyte blocks if they want, what the fuck, who cares, if some nations/people's technology or infrastructure can't handle that there are plenty of alt-coins they can use instead.

For example anyone who thinks anything more than 60 megabytes per ten minutes would be a bit too much for them can choose geistgeld, with its 60 one-megabyte blocks per 10 minutes (one block per ten seconds).

Anyone who thinks any more than four megabytes per ten minutes would be a bit much for them to handle can choose a one block per two and a half minutes coin (is litecoin one of the 2.5 minutes blocks ones?)

Anyone who thinks five megabytes per ten minutes is better than four, but more than that might be a problem can use the two minute blocks coin, I am fairly sure there is one I just cannot recall which one it was. Unless it was merely someone carelessly saying two minutes when they meant two and a half.

Basically if the megacorps / superpower-nations, or the places/people with the most-massive highest-capability infrastructure want to remove the limit so the vicious dog eat dog world of ruthless competition can get to the most massive, highest-bandwidth system as fast as possible to ensure it is absolutely bigger than any other in the world, so big no other network could ever hope to compete with it, there are plenty of alternative blockchain based currencies anyone with less interest in or desire for an infrastructure-tech bubble can choose.

-MarkM-

EDIT: "We don't care what your fat cat banker's nubile daughter spent on individual doses or even sixpacks of bubblegum, but we do want to account correctly for our local currency you spend on the shiploads of bubblegum we export to you."

[Garrett Burgwardt]

Been very interested in this debate, and I'm interrupting mughat and markm's conversation, and I apologize, but I'm sure they'll deal.

There was one proposal, which I can't recall now (somewhere deep in that giant thread) about having one unlimited/very large block every two weeks or so, to clear the backlog of transactions.

I think it warrants more discussion since most people seem to have passed it by. But perhaps I'm just missing some obvious flaw.

Thoughts?

[MoonShadow]

Been very interested in this debate, and I'm interrupting mughat and markm's conversation, and I apologize, but I'm sure they'll deal.

There was one proposal, which I can't recall now (somewhere deep in that giant thread) about having one unlimited/very large block every two weeks or so, to clear the backlog of transactions.

I think it warrants more discussion since most people seem to have passed it by. But perhaps I'm just missing some obvious flaw.

Thoughts?

That sounds like my proposal, to permit each re-target block to be unlimited.  The re-target block comes once every 2448 blocks (or so?), and is intended to be roughly every two weeks.  This would make the deliberate padding of blocks to force out small players ineffective, reward honest miners with an expecially profitable block if they are able to handle it, and preserve the market for rapidly confirming transactions for the remainder of the two week period.  Any small players who were overwelmed by a huge block would simply have to write off the next couple of blocks while they caught back up with the rest of the network.  It's also provide an outlet for free transactions and fee paying transactions that simply don't have enough to get included into a block, so the backlog of unconfirmable transactions (and thus the transaction queue) won't grow into infininty.

However, there could be, and probably are, some unintended consequences that this could create if this were the only change here.  The first one that I can think of is that there still wouldn't be any way to compel miners to include old transactions, free or not, and then such free transactions might never clear out regardless.

[MoonShadow]

An alternative that I also offered is to have a special case, wherein a miner could produce a particularly large block (probably not good to be unlimited) if all the transactions included were free; as evidence that the miner doing the processing is either doing so altruisticly, or is being compensated by an out-of-network agreement.

However, I'd modify this proposal a bit that wouldn't require that all transactions be free, but that all of them were part of the transaction pool for at least a week.

[Mashuri]

I think dree12 had the most logical argument for doing away with the limit altogether.

https://bitcointalk.org/index.php?topic=145636.msg1551002#msg1551002

Removing the limit does not imply useless transactions. It simply means the market will decide what is a useless transaction. So long as clients prefer smaller blocks over larger ones, the miners will decide among themselves what a suitable block size is. If it's too high, small-block miners will earn more and blocks will get smaller. If it's too low, large-block miners will earn more and blocks will get larger.

Isn't this what Bitcoin's about? All we need to do is make sure the system cannot be exploited, test it on testnet, and then not need to worry about block sizes ever again.

Appendix:
1. What changes are needed?
A: The "longest chain" code needs to increase the length of shorter blocks and decrease the length of longer ones. Clients also need to set a bandwidth quota so that they download and propagate the best currently-downloaded chain.
2. Couldn't a miner just spam the network with a huge block?
A: No. Like transactions, blocks need "confirmations". If other miners think the block is too big, they will refuse to confirm it and a smaller block will replace it (as they orphan large blocks).
3. Wouldn't blocks be too big?
A: No. Miners will take an unacceptable risk if they mine large blocks, as they are likely to get orphaned. Higher demand and higher transaction fees will make taking that risk more attractive, so block size can scale with usage.
4. Wouldn't miners just mine empty blocks?
A: Eventually, no. This is a concern in the short term, but as soon as transaction fees overtake subsidies empty blocks will no longer be attractive.
5. Wouldn't this allow DOS attacks?
A: Yes. However, DOS attack prevention code can be written to give up downloading a particularly large block if a competing chain is running at not far below network hashrate.
6. Isn't this a hard fork?
A: Done properly, not any more than raising the block limit. The length code only acts as a tiebreaker, which we already have in "first block wins".

My own question: The bitcoin network has operated well under the limit so far.  If arbitrarily increasing the block size were so advantageous to certain miners, why hasn't it already been done?  We should have seen the limit maxed a long time ago, yes?

[BitcoinAshley]

Dree12's proposal sounds reasonable. Bitcoin is like the little baby bird in the nest and we're like the big  protective momma bird. We don't want to let her out to fly in case she's not ready and she plops to the ground!
For a while the network was way too small and we worried about attacks. And of course with ASICs starting to go online there are concerns about too few people with too much mining power but soon ASICs will be widespread so that won't be an issue. There should come a point where we recognize Bitcoin's maturity and let it decide for itself what the block size is. The important thing is to do it soon because a hard fork takes time. I feel like it should have already been done... if BTC keeps growing at this rate, we're not going to have even a year to introduce a hard fork!
TBH and IMHO it matters more that we do something than what we actually do... keep draggin' the feet and there will be scalability problems in 6 months and people will migrate to alt-coins.
(Is a hard fork necessary for removing the block size limit altogether?) </noob>

We should have seen the limit maxed a long time ago, yes?

I also would like to know why people aren't just pumping out 1MB blocks non-stop which is apparently what will happen if we remove the limit.

[Sukrim]

My own question: The bitcoin network has operated well under the limit so far.  If arbitrarily increasing the block size were so advantageous to certain miners, why hasn't it already been done?  We should have seen the limit maxed a long time ago, yes?

Increasing it beyond 1 million bytes would require a hard fork. We are close to reaching the 250k bytes "soft limit" of bitcoin-qt however, mostly "thanks" to Satoshi's dice.
Writing custom rulesets requires work and this was not necessary so far, there are actually no real specialized "mining" bitcoind implementations out there as far as I'm aware.

Everyone running a full node has to store currently ~2.5 GB of SD transactions alone.

Currently the rules in bitcoin-qt are quite tight concerning "spammyness", also quite a few pool operators are as far as I see it rather interested in the long run than risking something by making a quick change for a few extra coins (=accepting more spammy transactions with and maybe even without fees).

[MoonShadow]

.
(Is a hard fork necessary for removing the block size limit altogether?) </noob>

Yes, but only because the max blocksize is coded into the rules that define the validity of a block, and anyone that refuses to upgrade to the new set of rules (on purpose or simply by ignorance of the issues) would force the blockchain to split into two competing versions.  Thus, for a time, there will literally be two different versions of the truth, and that cannot persist.  Bitcoin is designed to manage relatively short splits that result from temporary conditions, such as bandwidth issues for entire sections of the Internet.  However, it's not really designed to be able to recover from a split that lasts more than a day.

We should have seen the limit maxed a long time ago, yes?

I also would like to know why people aren't just pumping out 1MB blocks non-stop which is apparently what will happen if we remove the limit.

No one has tried to game the system, in part, because the presence of the max_blocksize rule would make any such attempt futile.  If we do remove that limit altogether, that kind of criminal calculation changes, and attempts might be made.

[misterbigg]

...doing away with the limit...

Did you skip over the explanation for how an unlimited block size drives fees to nothing?

[hazek]

</noob>

You should have placed that at the end of your question too! The question has an obvious answer.. The reason big miners aren't stuffing blocks right now is because 1Mb blocks can't be used to attack smaller miners into quitting. Quite fking obvious isn't it?

[Mashuri]

...doing away with the limit...

Did you skip over the explanation for how an unlimited block size drives fees to nothing?

Did you skip over dree12's explanation as to why it wouldn't?  If you disagree with his explanation, can you refute it specifically?

[misterbigg]

If you disagree with his explanation, can you refute it specifically?

Of course I can, and this has been explained MULTIPLE times. Miners want to maximize their revenue. Users want to minimize their fees. If miners always include every transaction regardless of fees, there's no incentive for users to pay fees. The only way for a market for fees to emerge is if there is competition for transactions in a block. That means that miners have to leave some transactions out. But which transactions should they drop? Obviously...the ones with the lowest fees per kilobyte. Or to put in other terms, a market for fees can only emerge when block size is limited.

I think that when people come on here and post claims that there should be no limit on block size that means either they did not bother to get acquainted on the subject by reading the exist posts, or they simply do not have the capacity to understand regardless of the amount of explanation provided.

[hazek]

...doing away with the limit...

Did you skip over the explanation for how an unlimited block size drives fees to nothing?

Did you skip over dree12's explanation as to why it wouldn't?  If you disagree with his explanation, can you refute it specifically?

I already did that. It's the same reason why in a market regulated strictly by consumption i.e. a free market there can be no racists businesses because they'd drive themselves out of business by discriminating. Same goes for miners who would discriminate against those paying less fees than such a miner would demand.. that "business" wouldn't simply be gobbled up by some other miner and so the competition for putting as many transactions into a block paying as much as possible would force all miners to accept ever decreasing transactions until we Bitcoin ends up with only a handful of miners providing not nearly enough security.

[misterbigg]

Did you skip over dree12's explanation as to why it wouldn't?  If you disagree with his explanation, can you refute it specifically?

Couple of posts that explain things well:

We know that the equilibrium when not restricted by block size is minimal fees per transaction (even free transactions) because that is the reality now.

The block size is an intentionally limited economic resource, just like the 21,000,000-bitcoin limit...Changing that vastly degrades the economics surrounding bitcoin, creating many negative incentives.

[Mashuri]

I understand that reasoning.  Why do you think this proposal wouldn't work to mitigate such an outcome?

1. What changes are needed?
A: The "longest chain" code needs to increase the length of shorter blocks and decrease the length of longer ones. Clients also need to set a bandwidth quota so that they download and propagate the best currently-downloaded chain.

[misterbigg]

Why do you think this proposal wouldn't work to mitigate such an outcome?

I don't even understand that proposal but from the little I do understand, I don't see how it relates to changing the maximum block size. Maybe you could rewrite it in more clear, formal terms? Here's an example of my proposal for changing the maximum block size:

1) A boolean flag is added to each block. The flag represents the block solver's yes or no vote for increasing the block size. The independent miner or mining pool sets this flag according to their preference for an increase.

2) Every time the difficulty is adjusted, the number of yes votes is counted from the last adjustment. If the number of yes votes is greater than 90%, then the block size is increased by 1%. Both percentages are baked-in constants, requiring a hard fork to change.

See how it's concise? Try to mimic that.

[Sukrim]

If you disagree with his explanation, can you refute it specifically?

Of course I can, and this has been explained MULTIPLE times. Miners want to maximize their revenue. Users want to minimize their fees. If miners always include every transaction regardless of fees, there's no incentive for users to pay fees. The only way for a market for fees to emerge is if there is competition for transactions in a block. That means that miners have to leave some transactions out. But which transactions should they drop? Obviously...the ones with the lowest fees per kilobyte. Or to put in other terms, a market for fees can only emerge when block size is limited.

I strongly disagree with this, as limited block size is not the only limitation in the system.

I agree that miners want to earn fees and users want to pay as few fees as possible. Miners however in your (limited block size scenario) not only can leave transactions out of blocks, they have to leave them. Even if there are more transactions out there that the miner would like to include, he has to choose as subset of the available ones.

Even if miners only can ignore transactions, they will do so - including more transactions means slower block propagation times and as already said in a different thread, bandwidth is still a limiting factor in an unlimited/virtually unlimited block size scenario. People are getting worried that miners might start to inflate blocks to drive other miners out of the network. Still having a bunch of miners that can only send blocks with a 100% used 10 GBit link between them would just mean they'd have the longest chain but nobody can use it, as it is not accessible to anyone outside any more (sending data out of the mining network would mean the miner gets driven out of business by his colleagues that have no more need for him). I'd like to call it the miner circlejerk scenario.

Anyways, since larger blocks = higher risk for orphans = fees need to be higher to migitate that risk (e.g. 5% orphan risk = needs 5% more fees than needed with 0% orphan rate), miners still can and will create a market for fees, even with (practically) limitless blocks.

[markm]

So it'd be like the search engine circle-jerk back when the most relevevant possible page, in the engine's eyes, was its own output.

So we got what, a big three in each part of the world?

Modulo the fact that blocks are not cultural/linguistic the way web pages are, so maybe the big three can be the same big three world-wide, no "you don't even speak our language, you don't understand the nuances of our pages, thus your searches suck" effect helping very-different cultures to prop up a big one of their own (do they even get to have a big three, or do the regions where the big three aren't the biggest only have a big one of their own not their own whole big three?)

Way to not centralise. Are the big number two and big number three still shrinking relative to the big one or are they becoming more equal?

-MarkM-

[mp420]

I think misterbigg's voting proposal is the best one thus far. My reasoning: Miners are more likely to keep on mining if they can maximize their real revenue. And if over 90% of mining power thinks that increasing the max block size will increase their revenue, I'd say it's very very likely that this is indeed the case.

Of course this will mean big boys will drive hobbyist miners out of the market. But they can do this regardless: Large scale ASIC operations with industry level electricity contracts (and located in places where energy is cheap) can drive difficulty so high that hobbyist rigs can never even dream of turning profit.