firstly, what we're talking about here is not, as DumbFruit generalizes, a "rollover fee". It's a disporportional penalty on mining large blocks. I'm not sure wether this changes his argument or its validity.
Unless I missed something huge, the proposal is not only to penalize large blocks, but to redistribute the penalties collected from these blocks back to other miners. In that sense there is a rollover of mining rewards (since penalized miners stand to earn their own penalties back), just not "fees rollover" per say.
Do the 2 25% miners (or 2 of the 10% miners) have a higher-than-in-current-system incentive to collude?
I don't think they would collude, rather I expect a large miner can deplete the mempool of high fee transactions (and only these) while staying under the soft cap (thus paying no penalties), and leave other miners to pick up the slack. This behavior is incentivized by the fact that the other pools, if they do pick up the slack (and try to deplete the mempool with less regard for fees) stand to pay penalties for the extra service rendered, and the large miner in return stands to gain from that.
This results in one of 2 scenarii:
1) Other miners decide to pick the slack from the large miner, they naturally loose in profitability as the large miner is vampirizing their rewards by collecting penatlies. As a result, a lot of users from the other pools will point their hardware to the large miner, which will get an ever increasing share of the network hash rate, which enables his scheme even further and so on.
2) As a reaction to the large miner's behavior, every other miner adopts his policies, which is to at least avoid the penalties, by always emitting blocks below the softcap, and this change never fixed what it was meant to.
Is Menis proposal making it easier for the 2 25% miners to try to drive out small (as in bandwidth) miners by mining disproportionally large blocks?
Other pools need to verify a block before they start to mine on top of it, so very large blocks, that propagate slower and take longer to validate could supposedly give a head start to the miner who emitted it, and be a disadvantage to smaller pools which stand at a risk of having their block orphaned the longer it takes for other miners to receive and validate their work.
I expect any centralized pool, however small they are, can afford the bandwidth and processing power to deal with much larger blocks than we have at the moment. This could hurt p2pool miners though.
Here are example scenarios, with made up values for the penalty function. I assume for simplicity (not a necessary assumption) that there is endless demand for transactions paying 1mBTC fee, that typical blocks are around 2K txs, and that there are no minted coins. The pool clears at 1% per block.
Scenario 1: The network has 100 1% miners.
Every 1% miner knows he's not going to claim much of any penalty he pays, so he includes a number of transactions that maximize fees-penalty for the block. This happens to be 2K txs, with a total fee of 2 BTC and penalty of 1 BTC.
The equilibrium for the pool size is 100 BTC.
Miners get 2 BTC per block (2 fees + 1 pool collection - 1 penalty).
There are 2K txs per block.
Scenario 2: The network has 1 90% miner, and 10 1% miners.
The 1% miners build blocks with 2K txs, fee 2 BTC, penalty 1 BTC, like before.
The 90% miner knows that if he includes more txs, he'll be able to reclaim most of the penalty, so the marginal effective penalty exceeds the marginal fee only with larger blocks - say, which have 4K txs, 4 BTC fees, 4 BTC penalty.
The average penalty per block is 3.7 BTC. The equilibrium pool size is 370 BTC.
There are on average 3.8K txs per block.
A 1% miner gets, per block he finds, (2 + 3.7 - 1) = 4.7 BTC - more than in scenario 1!
The 90% miner gets, per block he finds, (4 + 3.7 - 4) = 3.7 BTC - less than small miners get in this scenario, but more than miners get in scenario 1!
Those examples do not stand. They hinge on the premise that
there is endless demand for transactions paying 1mBTC fee. I understand the need to simplify these demonstrations but that defeats the underlying premise of this entire discussion. You example assumes there is no competition over fees, which is the premise of a "no block limit + hard fees" system. Your system sets both soft and hard caps to the block size, so there is no reason to believe people will sit at a 1mBTC fee when there is an
endless demand for transactionsModel your demonstration with a fee structure using the Pareto principle i.e. 20% of the transactions pay for the 80% of the total fees available in the mempool (which is a lot closer to the current network's fee distribution than your examples), and the system falls apart. Anyone building blocks large enough to get penalized is just giving his rewards away to miners that prioritize big fee transactions and make a point of staying under the soft cap.
The issue with your proposal is not the penalty per say, it's the reward: there is a point where it is more profitable to let others get penalized. The existence of this point creates a threat that keeps all miners functioning below the soft cap. The threat is that they lose profitability in comparison to other pools, and those pools start siphoning away their hash rate as miners migrate.
If you were to take away the reward from the system a few things would be smoother:
1) No opportunities to game the system anymore. It all comes down to where the acceptable margin of fee vs penalty stands for the given mempool.
2) Very simple to implement.
The drawback is that since there is no reward, obviously the penalties are just destroyed. I'm not sure that's a drawback per say, for the following reasons:
1) It's trivial to blackhole bitcoins and it's been agreed that this is not damaging to the system. So this method isn't introducing some new DOS attacks on the system.
2) By destroying the penalty, the value of every other BTC just went up. As opposed to your system where you want to reward other miners from the penalties, this time everyone is getting rewarded, albeit in a much smaller magnitude. This means both miners, but everyone else holding coins is rewarded when a miner builds a block above the soft cap. Incidentally, that also means people running nodes (as long as they hold BTC, which is expectable).
This is perhaps the only proposition so far that has some sort of reward mechanism for node maintainers (granted it's tiny) which take equal part in the cost of node propagation and validation as miners.