Elastic block cap with rollover penalties

[NewLiberty]

I see your point in that the amount of the fees are not part of the calculation of the penalty.
However, why would anyone mine a block for which they will not be paid?  
The incentive would remain to move fees off chain, especially as the coinbase reward decreased.
Off-chain fees would not be subject to the seizure and redistribution of the rollover.
Better for the miner to guarantee positive revenue and get it on the side.

[molecular]

I see your point in that the amount of the fees are not part of the calculation of the penalty.
However, why would anyone mine a block for which they will not be paid?  
The incentive would remain to move fees off chain, especially as the coinbase reward decreased.
Off-chain fees would not be subject to the seizure and redistribution of the rollover.
Better for the miner to guarantee positive revenue and get it on the side.

I still don't follow you.

Why does it matter wether the fee is payed on- or off-chain. The penalty is the same and the miner has to pay it either way.

[NewLiberty]

I see your point in that the amount of the fees are not part of the calculation of the penalty.
However, why would anyone mine a block for which they will not be paid?  
The incentive would remain to move fees off chain, especially as the coinbase reward decreased.
Off-chain fees would not be subject to the seizure and redistribution of the rollover.
Better for the miner to guarantee positive revenue and get it on the side.

I still don't follow you.

Why does it matter wether the fee is payed on- or off-chain. The penalty is the same and the miner has to pay it either way.

By miner has to pay it, we mean "not paid to miner but to rollover pool instead"?  They miner is never paying anything, they are simply not given the reward, yes?

[molecular]

I see your point in that the amount of the fees are not part of the calculation of the penalty.
However, why would anyone mine a block for which they will not be paid?  
The incentive would remain to move fees off chain, especially as the coinbase reward decreased.
Off-chain fees would not be subject to the seizure and redistribution of the rollover.
Better for the miner to guarantee positive revenue and get it on the side.

I still don't follow you.

Why does it matter wether the fee is payed on- or off-chain. The penalty is the same and the miner has to pay it either way.

By miner has to pay it, we mean "not paid to miner but to rollover pool instead"?  They miner is never paying anything, they are simply not given the reward, yes?

No, by "miner has to pay it", we mean he always has to pay it, even if he receives no on-chain fees.

Edit: Saying again for clarity - In the current proposal, fees from transactions will be paid to the miner of the current block instantly and in full. The miners can't gain anything by accepting tx fees out of band. The one paying into the rollover pool is the miner himself, as explained below.

[Meni Rosenfeld]

...

I'll try to examine your proposal in more detail later.

There is a major shortcoming I can see in the rollover fee pool suggestion: miners are incentivized to accept fees out of band so they can obtain all the highest fees instantly, thus defeating the entire purpose of that feature.

This is only (potentially) a problem in my 2012 rollover fee proposal. Here, tx fees don't go into the pool - fees are paid to the miner instantly. It is only the miner's block size penalty that goes in to the pool, and he must pay it if he wants to include all these transactions.

Just so I can understand properly, your penalty is related to a formula that excludes the transaction fees, therefore paying fees out of band won't reduce the penalty.  Is that how the problem is solved?

Looks correct, but I would say there was no problem to begin with.

I think the current design already incentivize smaller blocks: Smaller blocks get broadcasted much faster and become less possible to be orphaned. If you consider that there are 25 coins to compete for, you would like to broadcast your block as fast as possible once you find it

This is correct but as far as I know, the magnitude of this effect is very small, and not enough to keep block size in check.

That said, a problem with any kind of rollover fee is that it assumes that moving fees to future blocks is also moving fees to different nodes.

This is correct, and I hadn't given enough thought to this problem prior to posting.

Now that I've given it more thought, I think it can be significantly alleviated by making collection from the pool span a longer period, on the time scale of years. Relative hashrate is likely to change over these period, so it may not be the best plan to publish excessively large blocks hoping to reclaim the fee (and publishing typical-sized blocks does not give big miners an advantage). Also, with a different function you can change the balance between the marginal and total penalty so that the actual penalty is small, nullifying the effect (it will require the cap to be a bit harder, but still more elastic than what we have now).

I agree that this calls for more analysis.

Can you explain a bit about the mechanism wherein the miner pays into the rollover pool

Not much to explain. The penalty is deducted from the amount of bitcoins he can credit himself in the generation transaction; The network keeps track of the total pool size, and allows miners to add a fraction of it to the bitcoins they credit themselves.
Instead of
total outputs = new coins + tx fees
You have
total outputs = new coins + tx fees + collection from pool - penalty

 It is not obvious why this dictinction makes a difference.  It seems to still incentivize out of chain payments to miners for transaction inclusion regardless of whether it is paid by the miner, or deducted from the miner's reward, both are dependent on the fees in the block (which aren't there in out of block payments schemes).

The penalty doesn't depend on the fees of txs in the block. It depends on the size of txs in the block. Out-of-band payments are irrelevant - the miner must include the transaction to perform the service he would be paid for. If he includes it he pays a penalty for it, and expects a fee for it - it doesn't matter to either him or the user if the payment is out-of-band or not (except, of course, the extra friction of out-of-band).

The rollover pool creates an incentive for the miner to not use the fee pooling, and instead contract directly with the TX creators.
If implemented as written, this could become a problem.  Large TX creators and large miners would have an incentive to cartel because of the way this rollover pool works.

This is simply false, as explained above. It seems many people are confusing the current suggestion, with the suggestion I brought up back in 2012 and referenced here.

Thanks for that, it was my reading also.
Thus TX fees that are not in the block but paid out of band are not subject to penalty...

It's an additive deduction, not multiplicative.

You seem to be thinking the miner's reward is:
(1 - penalty) * (minted coins + tx fees + collection from pool)
Where it is really:
minted coins + tx fees + collection from pool - penalty

Having more fees in the txs included doesn't increase the penalty. There is no difference between adding 1 mBTC to the fee and paying 1 mBTC out-of-band.

Another difference, is that in the Monero method, the penalty can not reduce the block reward to below zero and make the block invalid.

The miner chooses how many transactions to include. The penalty starts at 0 and stays there for a while (or alternatively, has a derivative starting at 0). He chooses the set of txs so that his profit is maximal, not so that it is negative...

(PS I apologize for the multiple comments on the same issue, I'm trying to respond to all claims in the order I see them...)

, and why that is different from the 'original proposal'?

I'll need to examine the Monero implementation in more detail to answer that intelligently. One difference is that Monero, apparently, already has a mechanism to defer rewards to future miners, so it has no need to introduce the extra concept of a rollover pool. Additionally, I suggest a different scaling function (hyperbolic rather than quadratic). It has a different set of tradeoffs (which I currently believe is better overall). I could suggest a different function based on the requirements - finding the best functions to manifest given intuitive concepts is what I'm good at.

Monero avoids this problem, but most of the rest of this proposal has been implemented and running for quite a long time.  Its not new, or novel, except in ways that it is not as good.
An examination of the prior art is warranted.

I agree with the last part. I, too, am upset when people try to reinvent the wheel. But it's difficult to know whether some feature exists in some system somewhere. I've read several discussions of block size and didn't see a mention it. I started with privately chatting with Gavin and Mike, and they seemed unaware of it either. I don't think it was reasonable for me to do anything but post about it and see what people think, which I did. If I had known about the situation in Monero I may not have made this post.

I'm glad that I did, though. As we can see from the lively discussion here and on reddit, there are obviously many people who are interested in this kind of solution but who, like me, were not previously aware of Monero's system. I think wider awareness and discussion of these issues is a good thing.

[Meni Rosenfeld]

By miner has to pay it, we mean "not paid to miner but to rollover pool instead"?  They miner is never paying anything, they are simply not given the reward, yes?

No, by "miner has to pay it", we mean he always has to pay it, even if he receives no on-chain fees.

You're both correct, it's a matter of terminology.

The miner has to pay it. But he's not paying by spending any of his previous coins, he's paying by having the payment deducted from the reward he collects. If the reward is not sufficient (negative reward after deduction), the block is invalid.

So the miner will not include so many transactions that the penalty will make the block invalid.

See my previous comment and I hope everything is clearer now. I apologize for not intervening sooner, I had a busy day, maybe this whole exchange could have been avoided...

[molecular]

I think the current design already incentivize smaller blocks: Smaller blocks get broadcasted much faster and become less possible to be orphaned. If you consider that there are 25 coins to compete for, you would like to broadcast your block as fast as possible once you find it

This is correct but as far as I know, the magnitude of this effect is very small, and not enough to keep block size in check.

I agree. Also with IBLT or other block propagation bandwidth conservation techniques (some of which seem to be used by pool interconnects already) this effect can (and will) be nullified.

[molecular]

By miner has to pay it, we mean "not paid to miner but to rollover pool instead"?  They miner is never paying anything, they are simply not given the reward, yes?

No, by "miner has to pay it", we mean he always has to pay it, even if he receives no on-chain fees.

You're both correct, it's a matter of terminology.

The miner has to pay it. But he's not paying by spending any of his previous coins, he's paying by having the payment deducted from the reward he collects. If the reward is not sufficient (negative reward after deduction), the block is invalid.

So the miner will not include so many transactions that the penalty will make the block invalid.

See my previous comment and I hope everything is clearer now. I apologize for not intervening sooner, I had a busy day, maybe this whole exchange could have been avoided...

Thanks for clearing it up. I can see now where NewLiberty was probably coming from (either the monero mechanism or your earlier proposal referenced in OP, both of which I didn't look at). Your reward calculation formulas cleared things up.

[DumbFruit]

This is correct, and I hadn't given enough thought to this problem prior to posting.

Now that I've given it more thought, I think it can be significantly alleviated by making collection from the pool span a longer period, on the time scale of years. Relative hashrate is likely to change over these period, so it may not be the best plan to publish excessively large blocks hoping to reclaim the fee (and publishing typical-sized blocks does not give big miners an advantage). Also, with a different function you can change the balance between the marginal and total penalty so that the actual penalty is small, nullifying the effect (it will require the cap to be a bit harder, but still more elastic than what we have now).

I agree that this calls for more analysis.

A longer time period over which the reward is given doesn't help, as the larger nodes or entities will still get a larger ratio of the rolling over fees, by definition.
Actually, making the rollover fees only extend over a couple of blocks would more likely mitigate the problem, but if you roll over the fee for about 3 blocks or so, then it might be worth it for a miner to hold blocks and release 2 at a time, depending on fee-over-time. This, in turn, might exacerbate the selfish miner exploit¹. The natural monopoly condition that already exists in Bitcoin² seems to be exacerbated either way.

Getting around this would be tricky, if it's possible.

¹http://fc14.ifca.ai/papers/fc14_submission_82.pdf
²https://bitcointalk.org/index.php?topic=176684.msg9375042#msg9375042

An examination of the prior art is warranted.

Pointing to Monero as an examination of prior art is asking a bit much. Are you expecting us to dig through the Monero source code? How do they get around the problem?

This is not very helpful;

The Basics
A special type of transaction included in each block, which contains a small amount of monero sent to the miner as a reward for their mining work.

https://getmonero.org/knowledge-base/moneropedia/coinbase

[JeromeL]

Monero avoids this problem, but most of the rest of this proposal has been implemented and running for quite a long time.  Its not new, or novel, except in ways that it is not as good.

When Gavin says "I need to see working code", he probably means code he can directly deploy to his test setup within the framework of bitcoin-core. I can relate to this demand and find it reasonable.

It's a polite way to say :"i am not interested in your design, i dont want to analyse it, f**k off. "

[goatpig]

Monero avoids this problem, but most of the rest of this proposal has been implemented and running for quite a long time.  Its not new, or novel, except in ways that it is not as good.

When Gavin says "I need to see working code", he probably means code he can directly deploy to his test setup within the framework of bitcoin-core. I can relate to this demand and find it reasonable.

It's a polite way to say :"i am not interested in your design, i dont want to analyse it, f**k off. "

I'd be in his position I'd would too ask to see some code or at least some data analysis supporting the design. You can't just propose stuff and expect the people reviewing it to do all the leg work. An implementation at least proves your design is conceptually sound. It's easy to forget certain aspects when you theorycraft, and having to implement at least the PoC certainly motivates you to keep it as simple as possible.

[solex]

Since the blocksize issue is controversial and may take some time to settle, we are better off implementing this elastic cap right now with a softfork (your phase II) and skip the hardfork part (phase I).

We could do that by choosing T=0.5MB (2T = 1MB = current maxblocksize).

True, if the circumstances were different. Presently the average (7-day, moving) block size (ABS) is about 400KB, 80% of T, and T will certainly be less than the ABS by the time an elastic cap can be implemented. ABS will never approach 2T because some miners will always churn out small or empty blocks (certainly while the reward > block fees), so the present ABS is for practical purposes equivalent to T now. The point which would normally be an alert for attention to be given to the prevailing limit.

The elastic cap with fee pool does need working out in detail, as the discussion between molecular and NewLiberty shows. Its mechanics, incentives, attack vectors and long-term implications need to be known and understood. This won't happen quickly. Phase I buys time for Phase II.

Increasing the 1MB is technically very simple, accepting just the usual risk inherent in larger blocks.

If this type of consensus can be hand-waved away, then we are going to have to go home and learn to love our respective fiats, because they will all be around for a lot longer.

latest poll results "should the blocksize be raised?".
http://www.poll-maker.com/results329839xee274Cb0-12#tab-2:

[Meni Rosenfeld]

A longer time period over which the reward is given doesn't help, as the larger nodes or entities will still get a larger ratio of the rolling over fees, by definition.

While writing a response to this, I realized the situation is actually much more nuanced than I thought.

Big miners don't have an advantage over small miners. Rather, the existence of big miners shifts the balance of the system as a whole, creating a situation where miners get more rewards, users pay less fees and/or get more of their transactions included, while nodes have to deal with larger blocks.

Here are example scenarios, with made up values for the penalty function. I assume for simplicity (not a necessary assumption) that there is endless demand for transactions paying 1mBTC fee,  that typical blocks are around 2K txs, and that there are no minted coins. The pool clears at 1% per block.

Scenario 1: The network has 100 1% miners.
Every 1% miner knows he's not going to claim much of any penalty he pays, so he includes a number of transactions that maximize fees-penalty for the block. This happens to be 2K txs, with a total fee of 2 BTC and penalty of 1 BTC.

The equilibrium for the pool size is 100 BTC.
Miners get 2 BTC per block (2 fees + 1 pool collection - 1 penalty).
There are 2K txs per block.

Scenario 2: The network has 1 90% miner, and 10 1% miners.
The 1% miners build blocks with 2K txs, fee 2 BTC, penalty 1 BTC, like before.
The 90% miner knows that if he includes more txs, he'll be able to reclaim most of the penalty, so the marginal effective penalty exceeds the marginal fee only with larger blocks - say, which have 4K txs, 4 BTC fees, 4 BTC penalty.

The average penalty per block is 3.7 BTC. The equilibrium pool size is 370 BTC.
There are on average 3.8K txs per block.
A 1% miner gets, per block he finds, (2 + 3.7 - 1) = 4.7 BTC - more than in scenario 1!
The 90% miner gets, per block he finds, (4 + 3.7 - 4) = 3.7 BTC - less than small miners get in this scenario, but more than miners get in scenario 1!

To restate what goes on - the big miners create supersized blocks because it benefits them, but it benefits the small miners even more.

Miners are happy. Users are happy. Nodes are not happy.

But note the following: Big miners make the mining rewards bigger, but you don't have to be a big miner to enjoy it (small miners actually are at an advantage). So if a miner becomes big, he encourages more people to start mining, raising the difficulty and countering the effect.

So more analysis is still in order, but overall, I don't think these dynamics encourage the formation of big miners.

[Meni Rosenfeld]

I'd be in his position I'd would too ask to see some code or at least some data analysis supporting the design. You can't just propose stuff and expect the people reviewing it to do all the leg work. An implementation at least proves your design is conceptually sound. It's easy to forget certain aspects when you theorycraft, and having to implement at least the PoC certainly motivates you to keep it as simple as possible.

Not sure to which extent this is criticism to me. But I believe everyone has a part to play in this world, and should be doing what he's best at. My comparative advantage is in coming up with ideas and discussing them; and in unrelated work (to those who don't know me, my day "job" is in promoting Bitcoin in Israel). It's not in coding and empirical analysis - I'll leave that to others. This methodology worked quite well at the time I helped mining pool operators with implementing DGM. Perhaps the discussion I've started will result in this or a similar idea being implemented and accepted. But if not, so be it.

I'll clarify that I think Gavin's request is perfectly legitimate. I didn't exactly expect him to be so dazzled by the idea that he'd drop everything he was doing and start working on it.

[goatpig]

I'd be in his position I'd would too ask to see some code or at least some data analysis supporting the design. You can't just propose stuff and expect the people reviewing it to do all the leg work. An implementation at least proves your design is conceptually sound. It's easy to forget certain aspects when you theorycraft, and having to implement at least the PoC certainly motivates you to keep it as simple as possible.

Not sure to which extent this is criticism to me. But I believe everyone has a part to play in this world, and should be doing what he's best at. My comparative advantage is in coming up with ideas and discussing them; and in unrelated work (to those who don't know me, my day "job" is in promoting Bitcoin in Israel). It's not in coding and empirical analysis - I'll leave that to others. This methodology worked quite well at the time I helped mining pool operators with implementing DGM. Perhaps the discussion I've started will result in this or a similar idea being implemented and accepted. But if not, so be it.

I'll clarify that I think Gavin's request is perfectly legitimate. I didn't exactly expect him to be so dazzled by the idea that he'd drop everything he was doing and start working on it.

It's not criticism directed towards anyone per say. In the course of my work with Armory, I get suggestions to implement this and that, but an idea that can be summarized in a single sentence can often demand 10k LoC. I'm much more inclined to look at a pull request than just some formulated concepts. As I said, having a PoC to support the idea has several advantages, one of which is to make the task of reviewers simpler, another which is to go through the most obvious optimizations right away. An idea without a PoC is not diminished, but an idea with a PoC is certainly improved. I felt like I should share that. It wasn't even an attempt to defend Gavin.

Obviously if you can find someone to work a PoC for your proposal, that would be fantastic.

You're an idea man, I'm a nuts and bolts guy, I can't help but look at this from my perspective. Your natural stance towards people with my skill set is "you don't sophisticate enough". My natural stance towards people with your skill set is "you complicate too much". This isn't about to change anytime soon, yet that doesn't make it a personal attack. Present a patient with some general syndrome to N different medical specialists, all in different fields, and they will come up with N different diagnosis. They're not all necessary wrong.

If you think there is some underlying ad hominem in my criticism of your proposal, that is not my intent. There are plenty of other sections in this forum which are ripe for this kind of rhetoric. I'm going to defend my point of view with every opportunities I get, I don't expect less from others. The intensity of the criticism may come across as unwarranted but that's only cause I'm genuinely interested in this discussion. That should vouch on its own for the importance I bear to theoretical research.

[molecular]

So more analysis is still in order, but overall, I don't think these dynamics encourage the formation of big miners.

This is encouraging... it sounded yesterday as if you had almost regretted making this thread and were about to pull your own support from the proposal because of this and now it looks like it might be less of a problem than initially thought.

I'm having some trouble following the logic of the objection. I dug it from upthread:

That said, a problem with any kind of rollover fee is that it assumes that moving fees to future blocks is also moving fees to different nodes.

Put differently; centralizing nodes is a way of avoiding the penalties you're trying to introduce with this protocol.

Put differently again; Paying fees over consecutive blocks gives a competitive advantage to larger mining entities when making larger blocks.

Put triply differently; A node that can reliably get another block within X blocks is less penalized than a node that cannot, where "X" is the number of blocks that the rollover fee is given.

So if the goal is to avoid centralization, then the protocol does the opposite of the intention. If the goal is to make Bitcoin fail-safe, I'm not convinced that Bitcoin isn't already. When blocks fill, we will see higher transaction fees, potentially lengthier times before a transaction is included in a block, and as a result more 3rd party transactions.

TLDR: How does a fee over "X" blocks not incentivize a centralization of nodes?

firstly, what we're talking about here is not, as DumbFruit generalizes, a "rollover fee". It's a disporportional penalty on mining large blocks. I'm not sure wether this changes his argument or its validity.

For thinking about this I'm using the following hypothetical mining landscape: 25%, 25%, 5 x 10%.

Now I think there are at least 2 interesting questions we can ask:

• Do the 2 25% miners (or 2 of the 10% miners) have a higher-than-in-current-system incentive to collude?
• Is Menis proposal making it easier for the 2 25% miners to try to drive out small (as in bandwidth) miners by mining disproportionally large blocks?

Both questions boil down to

• Does Menis proposal encourage centralization more than the current system?

Before I go on... am I asking the correct questions?

[molecular]

Obviously if you can find someone to work a PoC for your proposal, that would be fantastic.

If the proposal finds enough support, I'm sure we can crowd-fund a PoC.

[goatpig]

firstly, what we're talking about here is not, as DumbFruit generalizes, a "rollover fee". It's a disporportional penalty on mining large blocks. I'm not sure wether this changes his argument or its validity.

Unless I missed something huge, the proposal is not only to penalize large blocks, but to redistribute the penalties collected from these blocks back to other miners. In that sense there is a rollover of mining rewards (since penalized miners stand to earn their own penalties back), just not "fees rollover" per say.

Do the 2 25% miners (or 2 of the 10% miners) have a higher-than-in-current-system incentive to collude?

I don't think they would collude, rather I expect a large miner can deplete the mempool of high fee transactions (and only these) while staying under the soft cap (thus paying no penalties), and leave other miners to pick up the slack. This behavior is incentivized by the fact that the other pools, if they do pick up the slack (and try to deplete the mempool with less regard for fees) stand to pay penalties for the extra service rendered, and the large miner in return stands to gain from that.

This results in one of 2 scenarii:

1) Other miners decide to pick the slack from the large miner, they naturally loose in profitability as the large miner is vampirizing their rewards by collecting penatlies. As a result, a lot of users from the other pools will point their hardware to the large miner, which will get an ever increasing share of the network hash rate, which enables his scheme even further and so on.

2) As a reaction to the large miner's behavior, every other miner adopts his policies, which is to at least avoid the penalties, by always emitting blocks below the softcap, and this change never fixed what it was meant to.

Is Menis proposal making it easier for the 2 25% miners to try to drive out small (as in bandwidth) miners by mining disproportionally large blocks?

Other pools need to verify a block before they start to mine on top of it, so very large blocks, that propagate slower and take longer to validate could supposedly give a head start to the miner who emitted it, and be a disadvantage to smaller pools which stand at a risk of having their block orphaned the longer it takes for other miners to receive and validate their work.

I expect any centralized pool, however small they are, can afford the bandwidth and processing power to deal with much larger blocks than we have at the moment. This could hurt p2pool miners though.

Here are example scenarios, with made up values for the penalty function. I assume for simplicity (not a necessary assumption) that there is endless demand for transactions paying 1mBTC fee,  that typical blocks are around 2K txs, and that there are no minted coins. The pool clears at 1% per block.

Scenario 1: The network has 100 1% miners.
Every 1% miner knows he's not going to claim much of any penalty he pays, so he includes a number of transactions that maximize fees-penalty for the block. This happens to be 2K txs, with a total fee of 2 BTC and penalty of 1 BTC.

The equilibrium for the pool size is 100 BTC.
Miners get 2 BTC per block (2 fees + 1 pool collection - 1 penalty).
There are 2K txs per block.

Scenario 2: The network has 1 90% miner, and 10 1% miners.
The 1% miners build blocks with 2K txs, fee 2 BTC, penalty 1 BTC, like before.
The 90% miner knows that if he includes more txs, he'll be able to reclaim most of the penalty, so the marginal effective penalty exceeds the marginal fee only with larger blocks - say, which have 4K txs, 4 BTC fees, 4 BTC penalty.

The average penalty per block is 3.7 BTC. The equilibrium pool size is 370 BTC.
There are on average 3.8K txs per block.
A 1% miner gets, per block he finds, (2 + 3.7 - 1) = 4.7 BTC - more than in scenario 1!
The 90% miner gets, per block he finds, (4 + 3.7 - 4) = 3.7 BTC - less than small miners get in this scenario, but more than miners get in scenario 1!

Those examples do not stand. They hinge on the premise that there is endless demand for transactions paying 1mBTC fee. I understand the need to simplify these demonstrations but that defeats the underlying premise of this entire discussion. You example assumes there is no competition over fees, which is the premise of a "no block limit + hard fees" system. Your system sets both soft and hard caps to the block size, so there is no reason to believe people will sit at a 1mBTC fee when there is an endless demand for transactions

Model your demonstration with a fee structure using the Pareto principle i.e. 20% of the transactions pay for the 80% of the total fees available in the mempool (which is a lot closer to the current network's fee distribution than your examples), and the system falls apart. Anyone building blocks large enough to get penalized is just giving his rewards away to miners that prioritize big fee transactions and make a point of staying under the soft cap.

The issue with your proposal is not the penalty per say, it's the reward: there is a point where it is more profitable to let others get penalized. The existence of this point creates a threat that keeps all miners functioning below the soft cap. The threat is that they lose profitability in comparison to other pools, and those pools start siphoning away their hash rate as miners migrate.

If you were to take away the reward from the system a few things would be smoother:

1) No opportunities to game the system anymore. It all comes down to where the acceptable margin of fee vs penalty stands for the given mempool.
2) Very simple to implement.

The drawback is that since there is no reward, obviously the penalties are just destroyed. I'm not sure that's a drawback per say, for the following reasons:

1) It's trivial to blackhole bitcoins and it's been agreed that this is not damaging to the system. So this method isn't introducing some new DOS attacks on the system.
2) By destroying the penalty, the value of every other BTC just went up. As opposed to your system where you want to reward other miners from the penalties, this time everyone is getting rewarded, albeit in a much smaller magnitude. This means both miners, but everyone else holding coins is rewarded when a miner builds a block above the soft cap. Incidentally, that also means people running nodes (as long as they hold BTC, which is expectable).

This is perhaps the only proposition so far that has some sort of reward mechanism for node maintainers (granted it's tiny) which take equal part in the cost of node propagation and validation as miners.

[Meni Rosenfeld]

Here are example scenarios, with made up values for the penalty function. I assume for simplicity (not a necessary assumption) that there is endless demand for transactions paying 1mBTC fee,  that typical blocks are around 2K txs, and that there are no minted coins. The pool clears at 1% per block.

Scenario 1: The network has 100 1% miners.
Every 1% miner knows he's not going to claim much of any penalty he pays, so he includes a number of transactions that maximize fees-penalty for the block. This happens to be 2K txs, with a total fee of 2 BTC and penalty of 1 BTC.

The equilibrium for the pool size is 100 BTC.
Miners get 2 BTC per block (2 fees + 1 pool collection - 1 penalty).
There are 2K txs per block.

Scenario 2: The network has 1 90% miner, and 10 1% miners.
The 1% miners build blocks with 2K txs, fee 2 BTC, penalty 1 BTC, like before.
The 90% miner knows that if he includes more txs, he'll be able to reclaim most of the penalty, so the marginal effective penalty exceeds the marginal fee only with larger blocks - say, which have 4K txs, 4 BTC fees, 4 BTC penalty.

The average penalty per block is 3.7 BTC. The equilibrium pool size is 370 BTC.
There are on average 3.8K txs per block.
A 1% miner gets, per block he finds, (2 + 3.7 - 1) = 4.7 BTC - more than in scenario 1!
The 90% miner gets, per block he finds, (4 + 3.7 - 4) = 3.7 BTC - less than small miners get in this scenario, but more than miners get in scenario 1!

Those examples do not stand. They hinge on the premise that there is endless demand for transactions paying 1mBTC fee. I understand the need to simplify these demonstrations but that defeats the underlying premise of this entire discussion. You example assumes there is no competition over fees, which is the premise of a "no block limit + hard fees" system. Your system sets both soft and hard caps to the block size, so there is no reason to believe people will sit at a 1mBTC fee when there is an endless demand for transactions

Model your demonstration with a fee structure using the Pareto principle i.e. 20% of the transactions pay for the 80% of the total fees available in the mempool (which is a lot closer to the current network's fee distribution than your examples), and the system falls apart. Anyone building blocks large enough to get penalized is just giving his rewards away to miners that prioritize big fee transactions and make a point of staying under the soft cap.

The issue with your proposal is not the penalty per say, it's the reward: there is a point where it is more profitable to let others get penalized. The existence of this point creates a threat that keeps all miners functioning below the soft cap. The threat is that they lose profitability in comparison to other pools, and those pools start siphoning away their hash rate as miners migrate.

I'm sorry, but I disagree. The scenarios don't "hinge" on this assumption, I just had to assume something in order to give concrete numbers. The effects I discussed should remain intact whatever the true transaction demand curve is.

The assumption also does not defeat the premise of the discussion, either. There is some demand curve for txs which depends on the real-world usage of the system, and we're talking about matching a supply curve to it. It's completely legitimate (though of course grossly exaggerated) to assume the demand curve is - infinite demand for txs with fee <1mBTC, and 0 for higher fees. With this hypothetical demand curve, fees will remain at 1mBTC because no one is willing to pay a higher fee, and no tx with lower fee will be accepted. In other words, the supply curve must intersect demand at its vertical drop line.

I'll try to repeat the calculations with a different demand curve, to demonstrate my point. But this will take some time and Shabbat is in soon, so that will have to wait.

As for the analysis, I also disagree. You can't "let others get penalized", every miner chooses his own penalty. The best you can do is penalize yourself, knowing that you'll reclaim the penalty later - but in so doing, you also increase the rewards for others.

The drawback is that since there is no reward, obviously the penalties are just destroyed. I'm not sure that's a drawback per say, for the following reasons:

1) It's trivial to blackhole bitcoins and it's been agreed that this is not damaging to the system. So this method isn't introducing some new DOS attacks on the system.
2) By destroying the penalty, the value of every other BTC just went up. As opposed to your system where you want to reward other miners from the penalties, this time everyone is getting rewarded, albeit in a much smaller magnitude. This means both miners, but everyone else holding coins is rewarded when a miner builds a block above the soft cap. Incidentally, that also means people running nodes (as long as they hold BTC, which is expectable).

Destroying coins in small amounts is not very harmful. But if done continuously as part of the system, it has negative macroeconomic implications.

per say

Grammar Nazi regulations state that if you make the same error twice in a discussion, you must be corrected. We're already at 4. It's "per se".

[Meni Rosenfeld]

So more analysis is still in order, but overall, I don't think these dynamics encourage the formation of big miners.

This is encouraging... it sounded yesterday as if you had almost regretted making this thread and were about to pull your own support from the proposal because of this and now it looks like it might be less of a problem than initially thought.

At the moment I'm confident that the claimed centralization issue does not invalidate the method - whatever effect there might be, it's nullified with a correct parameter choice. I'm not as confident in my ability to convince others of this, but I can try.

Note, though, that superlinear mining rewards is a problem in general, I just don't see how this proposal contributes to it.

• Do the 2 25% miners (or 2 of the 10% miners) have a higher-than-in-current-system incentive to collude?
• Is Menis proposal making it easier for the 2 25% miners to try to drive out small (as in bandwidth) miners by mining disproportionally large blocks?

Both questions boil down to

• Does Menis proposal encourage centralization more than the current system?

Before I go on... am I asking the correct questions?

To be honest, I'm not sure. There are different elements at play here with complex interplay at different timescales. The usual question I'd ask is "do big miners have a superlinear advantage over small miners?". I claim above that the answer is no, but I'm now sure that answering this question alone is sufficient.

Obviously if you can find someone to work a PoC for your proposal, that would be fantastic.

That was one of the purposes of this thread.

If the proposal finds enough support, I'm sure we can crowd-fund a PoC.

FWIW, I'll be happy to contribute to it.