Elastic block cap with rollover penalties

[solex]

I really like Meni's elastic block cap proposal.

Obviously increasing the block size requires a hard fork, but the fee pool part could be accomplished purely with a soft fork.  

Absolutely. It's worth clarifying that the actual mechanics of how the pool fee works, (particularly how it is calculated), does not have to be on the critical path to resolving the 1MB problem.
All that is necessary today for Meni's proposal is to agree that a pool fee, of some make-up, can usefully exist.

Then it is possible to look at a simple implementation plan which overcomes the urgency of dealing with the 1MB, does not raise the limit too much, and allows time for an elastic cap with rollover penalties to be fully worked out, modeled, developed, and tested. As mentioned before, the pool fee could incorporate a function of block delta utxo and sigops.

Phase I:
Hard Fork to increase the max block size to 2T, e.g. via block version 4.
2T might be in the region of 6 or 8MB which also scales at a fixed percentage each year, say 20%, or a fixed multiple (e.g. 4x) of the recent average 144 or 2016 blocks. However, the difference this time is that no miner will be able to mine a block larger than T without paying a pool fee, but this won't be possible because it requires a supermajority on version 5 blocks to vary the pool fee from zero.

Phase II:
Soft fork to implement the full elastic cap, effective by supermajority, e.g. via block version 5, when blocks between T and 2T can then be mined.

Advantages:
Decoupling dealing with the hard-limit, from the making of a graceful decay in network performance as the limit is approached.
No urgency on how to best to set the pool fee, lots of time for debate and modelling.
A yearly scaling percentage can be more approximate because it should be easier to schedule hard-fork revisions to this as and when changes in global computing technology dictate.

[1714024579]

1714024579

[1714024579]

1714024579

[1714024579]

1714024579

[jonny1000]

Meni

Thanks for this great proposal.  Please could you explain how the rollover fee pool helps out and why the penalty cant just be burnt? Which would be more simple.

There is a major shortcoming I can see in the rollover fee pool suggestion: miners are incentivized to accept fees out of band so they can obtain all the highest fees instantly, thus defeating the entire purpose of that feature.

This is only (potentially) a problem in my 2012 rollover fee proposal. Here, tx fees don't go into the pool - fees are paid to the miner instantly. It is only the miner's block size penalty that goes in to the pool, and he must pay it if he wants to include all these transactions.

Just so I can understand properly, your penalty is related to a formula that excludes the transaction fees, therefore paying fees out of band won't reduce the penalty.  Is that how the problem is solved?

I think this idea seems really good, but I need to think a bit more.

[johnyj]

Nice proposal. It is exciting to hear some carefully thought, incentivize based design


I think the current design already incentivize smaller blocks: Smaller blocks get broadcasted much faster and become less possible to be orphaned. If you consider that there are 25 coins to compete for, you would like to broadcast your block as fast as possible once you find it

With bigger blocks cap, it becomes more favorable to mine smaller blocks. 10MB blocks have a very high risk of being orphaned by 1MB blocks, since the time needed to broadcasting them will be much longer than 1MB blocks, maybe several minutes longer

However, if everyone is aiming for the smallest block possible, then most of the transactions will not be included. So far we have not run into this problem because the broadcasting speed is still relatively fast at current block size. But if one day it happens, a natural result is the bigger blocks will ask for more fee due to higher risk of being orphaned, how mathematically it is formulated is difficult to say without some real world cases. I guess it will be very similar to what OP described, above certain threshold, the fee will increase exponentially due to the risk of being orphaned also increase exponentially

While searching for block propagation data, I found out an article from Gavin. His Invertible Bloom Look-up Tables proposal will incentivize all the miners to include similar set of transactions to speed up the block propagation time (miners do not need to broadcast transactions that other peers already have in their memory pool, only block header and some extra transactions), but that seems to be a major change further down the road

https://gist.github.com/gavinandresen/e20c3b5a1d4b97f79ac2

[molecular]

I like this proposal. It takes away much of the destructiveness of hitting the block size limit. Makes 'impact' softer and actually allows for a market to function: with higher aggregate fees (higher tx demand), economically usable blocksize can actually increase, which is not the case with the current 'hard' implementation: no amount of higher aggregate fees will increase block space. That doesn't facilitate a market.

It would be interesting to hear from some core devs. Sounds to me the proposal could be acceptable to Peter Wuille and maybe even Peter Todd, for example, since the solution still offers to deliver economic incentive to develop scaling solutions.

Maybe this really could be (worked into) something with broad consensus.

I understand we will still argue about T, but it will be much easier because the consequences of choosing it wrongly aren't as dire.

[pvz]

Why not consider % of transactions with a fee?

This way users have direct influence in block size by paying a transaction fee (above median).
With paying a transaction fee users have a vote in the 'free rider effect'/spam and if current levels are acceptable.

1. small block size -> only transactions with fee will be processed fast (others get parked in mem. pool)
2. big block size -> a lot of transactions have a fee and a lot transactions without a fee (spam?) are accepted
3. median block size -> a balance between transactions with and without a fee

Also added a bigger scale (due to near future (holiday) transaction peaks).

I used Gavin's proposal as starting point:

Code:

max block size = 10 * median size of last 144 blocks * % of transactions with a fee higher than median transaction fee for last 144 blocks (e.g.0.36) * 2

[klondike_bar]

Code:

max block size = 10 * (median size of last 144 blocks) * (% of transactions with a fee higher than median transaction fee for last 144 blocks * 2)

Code:

max block size = 20 * (median size of last 1200 blocks) * (% of transactions with a fee higher than median transaction fee for last 600 blocks)

I think the above simplifies your equation slightly, and uses better timeframes (~200hrs and ~50hrs) to compute the averages.

my biggest concern is what would happen in a spam attack, which could easily last several days, to as much as a week, if a quick-changing algorithm can be played to drastically increase blocksize during that time.

lets look at your original equation: assume that blocks are 0.5MB for the last 144 blocks (thus the block limit will probably be set to something like 5-10MB), but then spam attacks start. you see every block filled with 5MB of spam, 95% of transactions below the median for fees. after 144 blocks of this (1 day), you get something like this:

Code:

max block size = 10 * (5MB) * (0.05 * 2) = 50*0.1 = 5MB

which seems to resist growing any larger.

however, the median transaction fee drastically drops as a result, and in a continued attack soon you could see that ~10% of transactions pay more than the median fee.

Code:

max block size = 10 * (5MB) * (0.1 * 2) = 10MB

   If this attack persisted for 3-4 days you could see the blocksize increase to >20MB very fast, and >60MB within a week (would likely require very well-funded spammers though)

longer timeframes are absolutely necessary, bare minimum should be 600 blocks for calculations.

[JeromeL]

I really like Meni's elastic block cap proposal.

Obviously increasing the block size requires a hard fork, but the fee pool part could be accomplished purely with a soft fork.  

Absolutely. It's worth clarifying that the actual mechanics of how the pool fee works, (particularly how it is calculated), does not have to be on the critical path to resolving the 1MB problem.
All that is necessary today for Meni's proposal is to agree that a pool fee, of some make-up, can usefully exist.

Then it is possible to look at a simple implementation plan which overcomes the urgency of dealing with the 1MB, does not raise the limit too much, and allows time for an elastic cap with rollover penalties to be fully worked out, modeled, developed, and tested. As mentioned before, the pool fee could incorporate a function of block delta utxo and sigops.

Phase I:
Hard Fork to increase the max block size to 2T, e.g. via block version 4.
2T might be in the region of 6 or 8MB which also scales at a fixed percentage each year, say 20%, or a fixed multiple (e.g. 4x) of the recent average 144 or 2016 blocks. However, the difference this time is that no miner will be able to mine a block larger than T without paying a pool fee, but this won't be possible because it requires a supermajority on version 5 blocks to vary the pool fee from zero.

Phase II:
Soft fork to implement the full elastic cap, effective by supermajority, e.g. via block version 5, when blocks between T and 2T can then be mined.

Advantages:
Decoupling dealing with the hard-limit, from the making of a graceful decay in network performance as the limit is approached.
No urgency on how to best to set the pool fee, lots of time for debate and modelling.
A yearly scaling percentage can be more approximate because it should be easier to schedule hard-fork revisions to this as and when changes in global computing technology dictate.

Since the blocksize issue is controversial and may take some time to settle, we are better off implementing this elastic cap right now with a softfork (your phase II) and skip the hardfork part (phase I).

We could do that by choosing T=0.5MB (2T = 1MB = current maxblocksize).

[DumbFruit]

I think it's interesting to switch the conversation over to a soft-failure rather than trying to find the appropriate answer to the block size for now.

That said, a problem with any kind of rollover fee is that it assumes that moving fees to future blocks is also moving fees to different nodes.

Put differently; centralizing nodes is a way of avoiding the penalties you're trying to introduce with this protocol.

Put differently again; Paying fees over consecutive blocks gives a competitive advantage to larger mining entities when making larger blocks.

Put triply differently; A node that can reliably get another block within X blocks is less penalized than a node that cannot, where "X" is the number of blocks that the rollover fee is given.

So if the goal is to avoid centralization, then the protocol does the opposite of the intention. If the goal is to make Bitcoin fail-safe, I'm not convinced that Bitcoin isn't already. When blocks fill, we will see higher transaction fees, potentially lengthier times before a transaction is included in a block, and as a result more 3rd party transactions.

Rereading Mike Hearns article¹, changing bitcoin to include highest fee transactions into the mempool should result in the behavior I described. An end user might see delays before their transaction is included in a block, but I wouldn't call that a "crash landing", considering that the sorts of transactions that would be done at these rates are not as concerned about the speed of confirmation.

TLDR: How does a fee over "X" blocks not incentivize a centralization of nodes?

¹https://medium.com/@octskyward/crash-landing-f5cc19908e32

[pvz]

however, the median transaction fee drastically drops as a result, and in a continued attack soon you could see that ~10% of transactions pay more than the median fee.

And if we use:

Code:

% of transactions with a fee higher than (median transaction fee of transactions with a fee)

This excludes zero fee transactions in the calculation of the median transaction fee.
In that case a large quantity of zero fee transactions (and spam) does not have influence on the median transaction fee.

[klondike_bar]

however, the median transaction fee drastically drops as a result, and in a continued attack soon you could see that ~10% of transactions pay more than the median fee.

And if we use:

Code:

% of transactions with a fee higher than (median transaction fee of transactions with a fee)

This excludes zero fee transactions in the calculation of the median transaction fee.
In that case a large quantity of zero fee transactions (and spam) does not have influence on the median transaction fee.

perhaps a threshold? The problem is that spam is usually sent as a single transaction with 0.001btc fees and hundreds of recipients 0.0000001BTC that get. There should be protection in place that ensures sending thousands or millions of dust payments becomes prohibitively expensive, without limiting its use for colored coins or increasing fees to the regular user.

right now, the only sort of filter for this is the miners deciding whether to include or not include a transaction, and will include a spam transaction if theres a justifiable fee to do so.


IMO, the simplest way to specify the max size is as

Code:

Max Block Size = [2.50 * (average size of last 6000 blocks)] + [0.50 * (average size of last 600 blocks)]

That provides a max size thats 3x the size of the average block over the past 1000hrs (41days). If demand suddenly rises, the max size can double within a week.

say current block is ~0.5MB on average. The new size cap under this formula is 1.5MB (50% larger than it is now)

assume by the end of 2015 theres now 2MB/block on average, with business hours being around 5MB/block.

Code:

Max Block Size = [2.50 * (2)] + [0.50 * (~3MB during a busy week)] = 6.5MB

assume by the end of 2016 theres now 12MB/block on average, with business hours being around 30MB/block.

Code:

Max Block Size = [2.50 * (12)] + [0.50 * (~20MB)] = 40MB

assume that in the following week bitcoin gets popular, and suddenly there are 30-40MB blocks almost 24/7. within 4 days, the blocksize will increase to

Code:

Max Block Size = [2.50 * (~14)] + [0.50 * (~35MB)] = 52.5MB

[NewLiberty]

Isn't it precisely what is implemented in Monero? (except you don't have a rollover pool, the penalty is simply deducted from the block reward for good).

No idea what happens in Monero, but if so, more power to them.

Apparently, neither does Gavin.
He said he didn't want to talk to you until there was working code that does it?
Such code has been working for years, but people forget where the experimentation is occurring, the alts.
A good place to start:
https://github.com/monero-project/bitmonero/blob/54fbf2afb3bc029823ed6c200e08bd21fe42ac10/tests/unit_tests/block_reward.cp
and
https://github.com/monero-project/bitmonero/blob/c41d14b2aa3fc883d45299add1cbb8ebbe6c9ed8/src/cryptonote_core/blockchain.cpp#L2230-L2244

Can you explain a bit about the mechanism wherein the miner pays into the rollover pool, and why that is different from the 'original proposal'?  It is not obvious why this dictinction makes a difference.  It seems to still incentivize out of chain payments to miners for transaction inclusion regardless of whether it is paid by the miner, or deducted from the miner's reward, both are dependent on the fees in the block (which aren't there in out of block payments schemes).

[DumbFruit]

Isn't it precisely what is implemented in Monero? (except you don't have a rollover pool, the penalty is simply deducted from the block reward for good).

No idea what happens in Monero, but if so, more power to them.

Apparently, neither does Gavin.
He said he didn't want to talk to you until there was working code that does it?
Such code has been working for years, but people forget where the experimentation is occurring, the alts.

Can you explain a bit about the mechanism wherein the miner pays into the rollover pool, and why that is different from the 'original proposal'?

The difference is quantitative. In this version the rollover effects only blocks that exceed a threshold of size.

[NewLiberty]

Can you explain a bit about the mechanism wherein the miner pays into the rollover pool, and why that is different from the 'original proposal'?

The difference is quantitative. In this version the rollover effects only blocks that exceed a threshold of size.

OK, so its similar to Monero.
But there is a difference between this proposal and what Monero does that appears that it might need to be addressed.

The rollover pool creates an incentive for the miner to not use the fee pooling, and instead contract directly with the TX creators.
If implemented as written, this could become a problem.  Large TX creators and large miners would have an incentive to cartel because of the way this rollover pool works.

Monero avoids this problem, but most of the rest of this proposal has been implemented and running for quite a long time.  Its not new, or novel, except in ways that it is not as good.
An examination of the prior art is warranted.

[molecular]

Can you explain a bit about the mechanism wherein the miner pays into the rollover pool, and why that is different from the 'original proposal'?  It is not obvious why this dictinction makes a difference.  It seems to still incentivize out of chain payments to miners for transaction inclusion regardless of whether it is paid by the miner, or deducted from the miner's reward, both are dependent on the fees in the block (which aren't there in out of block payments schemes).

I think the penalty is not dependent on the fee:

The miner of a large block must pay a penalty that depends on the block's size.

So it still has to be paid regardless of wether or not the tx fee payment was on- or off-chain.

[NewLiberty]

Can you explain a bit about the mechanism wherein the miner pays into the rollover pool, and why that is different from the 'original proposal'?  It is not obvious why this dictinction makes a difference.  It seems to still incentivize out of chain payments to miners for transaction inclusion regardless of whether it is paid by the miner, or deducted from the miner's reward, both are dependent on the fees in the block (which aren't there in out of block payments schemes).

I think the penalty is not dependent on the fee:

The miner of a large block must pay a penalty that depends on the block's size.

So it still has to be paid regardless of wether or not the tx fee payment was on- or off-chain.

This is penalty then ONLY taken from the coinbase reward and not also TX fees?
How would this method survive the successive halvings (much less the eventual cessation)?

If it includes the TX fees (which likely it must), then to the extent that fees are off chain, they are not subject to the penalty.

[molecular]

Monero avoids this problem, but most of the rest of this proposal has been implemented and running for quite a long time.  Its not new, or novel, except in ways that it is not as good.

When Gavin says "I need to see working code", he probably means code he can directly deploy to his test setup within the framework of bitcoin-core. I can relate to this demand and find it reasonable.

[NewLiberty]

Monero avoids this problem, but most of the rest of this proposal has been implemented and running for quite a long time.  Its not new, or novel, except in ways that it is not as good.

When Gavin says "I need to see working code", he probably means code he can directly deploy to his test setup within the framework of bitcoin-core. I can relate to this demand and find it reasonable.

Sure, there are quite a few edits that would be needed.
However, it has also been deployed and shown to work over time without pernicious economic effects. 

[molecular]

Can you explain a bit about the mechanism wherein the miner pays into the rollover pool, and why that is different from the 'original proposal'?  It is not obvious why this dictinction makes a difference.  It seems to still incentivize out of chain payments to miners for transaction inclusion regardless of whether it is paid by the miner, or deducted from the miner's reward, both are dependent on the fees in the block (which aren't there in out of block payments schemes).

I think the penalty is not dependent on the fee:

The miner of a large block must pay a penalty that depends on the block's size.

So it still has to be paid regardless of wether or not the tx fee payment was on- or off-chain.

This is penalty then ONLY taken from the coinbase reward and not also TX fees?
How would this method survive the successive halvings (much less the eventual cessation)?

The penalty will be deducted from the funds he collects in the generation transaction

I read that as "from reward, tx fees and rollover pool input".

If the penalty exceeds the miner's income of tx fees + minted coins + his share of the current rollover pool, the block is invalid.

I stumbled accross this problem, too and I remember it was adressed upthread, not very far from OP. I just searched, but couldn't find it and I also can't remember why (or wether) this wasn't a problem. I think I wasn't convinced or didn't think it through.

But thinking about that a bit, assuming 0 block reward is reached. A block with no transactions in it can always be mined because it doesn't carry any penalty at all. As soon as there's a penalty to be payed there need to be some transactions in the block and the fees they pay must exceed the penalty (otherwise the block is invalid). Is this a problem? I'm not sure, seems to me that depends on the particularities of the penalty function...


 

[NewLiberty]

Can you explain a bit about the mechanism wherein the miner pays into the rollover pool, and why that is different from the 'original proposal'?  It is not obvious why this dictinction makes a difference.  It seems to still incentivize out of chain payments to miners for transaction inclusion regardless of whether it is paid by the miner, or deducted from the miner's reward, both are dependent on the fees in the block (which aren't there in out of block payments schemes).

I think the penalty is not dependent on the fee:

The miner of a large block must pay a penalty that depends on the block's size.

So it still has to be paid regardless of wether or not the tx fee payment was on- or off-chain.

This is penalty then ONLY taken from the coinbase reward and not also TX fees?
How would this method survive the successive halvings (much less the eventual cessation)?

The penalty will be deducted from the funds he collects in the generation transaction

I read that as "from reward, tx fees and rollover pool input".

Thanks for that, it was my reading also.
Thus TX fees that are not in the block but paid out of band are not subject to penalty...

Another difference, is that in the Monero method, the penalty can not reduce the block reward to below zero and make the block invalid.
There are many similarities, but also some important differences with the time tested Monero method and the current proposal.

[molecular]

The penalty will be deducted from the funds he collects in the generation transaction

I read that as "from reward, tx fees and rollover pool input".

Thanks for that, it was my reading also.
Thus TX fees that are not in the block but paid out of band are not subject to penalty...

No, that wasn't my understanding (the bolded part doesn't follow from what's said above, does it?). Fees aren't subject to penalty at all. The penalty depends purely on size of block (if I understand correctly). Even if the tx fee was payed out of band, the tx will still contribute to the penalty through increased block size. Yes, it's deducted from the tx fees (can be other tx fees), but it depends (in amount) on the fact that the tx takes up space in the block.

In other words: it's not the fee that's penalized (neither in block nor out of band), but the space it takes up in the block.

Again: if I understand correctly.

EDIT: regarding the discussion about what happens with block reward going to 0: this means that you can't mine a block with only transactions that had their fees paid out of band, because they would take up space and thus effect a penalty, but there'd not be enough on-chain fees to pay for it.