Bitcoin Forum

tl; dr: I propose replacing the hard cap on the data size of a block (1MB, 20MB, or whatever it is) with an elastic one, where resistance to larger blocks accumulates progressively. Miners will be required to pay a superlinear penalty for large blocks, to be paid into a rollover fee pool. This will greatly increase Bitcoin's robustness to a situation where the block cap is approached, and allow a healthy fee market.

Background
In these days there is heated debate about whether the block size limit of 1 MB should be relaxed. Many arguments are being discussed, but the most common one is the classic tradeoff:

Bigger blocks -> Harder to run a node -> Less nodes -> More centralization
Smaller blocks -> Less payments can be done directly as transactions on the blockchain -> More reliance on payment processing solutions involving 3rd parties -> More centralization

Most agree that there is some optimal value for this tradeoff, and the debate is about what it is, how to determine it, how to make decisions revolving it, etc. And most assume that if the blocks are too small, we'll get a "heads up" in the form of increasing tx fees.

However, Mike argues (https://medium.com/@octskyward/crash-landing-f5cc19908e32), and I tend to agree, that this is not how it will work. Rather, if the block limit is too small for the transaction demand, the Bitcoin network will crash. The gist of his argument, as I see it, is - market forces should find a fee equilibrium where transaction supply matches demand, whatever they are. However, market forces don't react fast enough to fluctuations, creating buildups that technically can't be handled by the software.

Mike argues also that since we don't know when we'll reach the limit - and once we do, we will have catastrophic failure without warning - we must hurry to raise the limit to remain safe. That part I have an issue with - if Bitcoin can't gracefully degrade in the face of rising transaction volume, we will have problems no matter what the current block size limit is. We should instead focus on fixing that problem.

In this post I will introduce a protocol modification that might eliminate this problem. I will describe the suggestion and analyze how it can help. With it in place, we no longer run the risk of a crash landing, only rising fees - giving us an indication that something should be changed.

And then, we can go back to arguing what the block size should be, given the tradeoff above.

Rollover fee pool
This suggestion requires, as a prerequisite, the use of a rollover fee pool. I first introduced the concept here (https://bitcointalk.org/index.php?topic=80387) - this post is worth a read, but it used the concept for a different purpose than we have here.

The original idea was that fees for a transaction will not be paid to the one miner of the block which includes it - rather, fees would be paid into a pool, from which the funds will gradually be paid to future miners. You could have, for example, that in each block, the miner is entitled to 1% of the current funds in the pool (in addition to any other block rewards).

In the current suggestion, we will use such a pool that is paid out over time - but it will not be the users who put money into the pool. Transaction fees will be paid to the miner who found the block as normal.

Edit: Saying again for clarity - In the current proposal, fees from transactions will be paid to the miner of the current block instantly and in full. The miners can't gain anything by accepting tx fees out of band. The one paying into the rollover pool is the miner himself, as explained below.

Elastic block cap
The heart of the suggestion is - instead of forbidding large blocks, penalize them. The miner of a large block must pay a penalty that depends on the block's size. The penalty will be deducted from the funds he collects in the generation transaction, and paid into the rollover pool, to be distributed among future miners. If the penalty exceeds the miner's income of tx fees + minted coins + his share of the current rollover pool, the block is invalid.

This requires choosing a function f that returns the penalty for a given block size. There is great flexibility and there's little that can go wrong if we choose a "wrong" function. The main requirements are that it is convex, and has significant curvature around the size we think blocks should be. My suggestion: Choose a target block size T. Then for any given block size x, set f(x) = Max(x-T,0)^2 / (T*(2T-x)). (graph (https://i.imgur.com/EZvlJq7.png))

This will mean that there are no penalties for blocks up to size T. As the block size increases, there is a penalty for each additional transaction - negligible at first, but eventually sharply rising. Blocks bigger than 2T are forbidden.

Analysis
I assume we do want scarcity in the blockchain - this prevents useless transactions that bloat the blockchain and make it harder to run nodes, and incentivizes users to fund the Bitcoin infrastructure. A block size limit creates scarcity - but only if there ever really are situations where we reach the limit. But as things stand, reaching the limit creates technical malfunctions.

Mike calls the problem "crash landing", but to me it's more like hitting a brick wall. Transaction demand runs towards the limit with nothing slowing it down, until it painfully slams into it. One minute there is spare room in the blocks, and no reason to charge tx fees - and the next, there's no room, and we run into problems.

If more transactions enter the network than can be put in blocks, the queue will grow ever larger and can take hours to clear, meaning that transactions will take hours to confirm. Miners can use fees to signal to the market to ease up on new transactions, but the market will be too slow to react. First, because the software isn't optimized to receive this signal; and second, because transaction demand is inelastic in short time scales. If, over sustained periods of time, transaction fees are high, I will stop looking for places to pay with Bitcoin, I will sign up to a payment facilitation service, etc. But short-term, if I have a transaction I'm set on sending right now (e.g. a restaurant tab), I'll be willing to pay very high fees for it if I must. So fees are not effective in controlling the deluge of transactions.

Enter an elastic cap. When tx demand exceeds the target block value, a backlog doesn't accumulate - miners simply include the extra in the block. They will start to require a bit of extra fees to compensate for the penalty, but can still clear the queue at the rate it is filling. If the incoming tx rate continues to increase, the marginal penalty miners have to pay per tx will increase, so fees will become higher - but since the process is more gradual, clients will be in a better position to understand what fee they need to pay for quick confirmation. The process will resemble climbing a hill rather than running into a brick wall. As we push the limits, the restoring force will grow stronger until we reach an equilibrium.

On longer time scales, the market will have an understanding of the typical fees, and make decisions accordingly. It can also analyze times of the day/week when fees are lower, and use those for any non-time-sensitive transactions.

With a hard cap, the queue of transactions can only clear at a specific rate. Below this rate there is no fee tension, and above it there is instability. With an elastic cap, a longer queue cause transaction fees to be higher, encouraging miners to include more in a block, increasing the speed at which the queue clears. This is a stable equilibrium that prevents the queue from growing unboundedly, while always maintaining some fee tension.

Incidentally, the current system is a special case of this suggestion, with f(x) = If (x<T, 0, Infinity). We assume that an invalid block is equivalent to an infinite penalty.

The way forward
I believe there is very little downside, if any, to implementing this method. It can prove useful even if the crash landing scenario isn't as grave as Mike suggests. And the primitives will be handy for other possible protocol improvements. I believe it is an essentially simple and fully fleshed-out idea. As such, I hope it can be accepted without much controversy.

It is however a major change which may require some significant coding. When discussing this idea with Gavin, he explained he'd be in a better position to evaluate it if given working, testable code. Writing code isn't my thing, though. If anyone is interested in implementing this, I'll be happy to provide support.

Related work
A similar system exists in CryptoNote, see e.g. section 6.2.3 of https://cryptonote.org/whitepaper.pdf.

Greg Maxwell proposed a similar system with variable mining effort instead of reward penalty - see e.g. http://sourceforge.net/p/bitcoin/mailman/message/34100485/ for discussion.

This proposal is not directly related to floating block limit methods, where the limit parameters are determined automatically based on recent history of transaction demand.

This is a very good, but as you said will require a lot of work, seems like a good portion of BTC's code would need to be rewritten. I'd say we're better off just letting BTC croak and eventually once the damage that is caused to public image has faded, transitioning to a better alt will be in the best interest of the whole cryptocurrency community.

The key here is how is T set. If T is fixed then 2T becomes the hard limit and the problem remains. If T is set based on an some average of previously mined blocks then this may address the problem

Would this address bandwidth issues that China could suffer from if block size was increased?

I don't think the changes are that extensive. Basically you just need to change the rules for generation transaction validity.

Umm... No?


We still need some way to determine the optimal block size, but we have much more leeway. The wrong choice will not cause catastrophic failure, rather gradually increasing fees which will indicate that a buff is needed. The flexibility will make it easier to reach community consensus about changing hardcoded parameters.

Reusing what I wrote to Gavin an a private exchange - I don't believe in having a block limit calculated automatically based on past blocks. Because it really doesn't put a limit at all. Suppose I wanted to spam the network. Now there is a limit of 1MB/block so I create 1MB/block of junk. If I keep this up the rule will update the size to 2MB/block, and then I spam with 2MB/block. Then 4MB, ad infinitum. The effects of increasing demand for legitimate transaction is similar. There's no real limit and no real market for fees.

Perhaps we can find a solution that uses an automatic rule for short-term fluctuations, and hardcoded parameters for long-term trends. If a good automatic cap rule can be found, it will be compatible with this method.


Only very indirectly - it can help Bitcoin function with a low cap, hence reducing the need to increase the cap.

Isn't it precisely what is implemented in Monero? (except you don't have a rollover pool, the penalty is simply deducted from the block reward for good).

I think (in contrast to global politics) we should follow the most intelligent people, and Meni is certainly smarter than most people here.

So, I support the idea. Just that 2T is a bit high, will that work? I mean, will it be possible to download  a blockchain with 2T blocks in it? A 2T blockchain consisting of smaller blocks is not an issue, but what happens if the download gets stuck DURING the 2T?

Is the Bitcoin client ready to download blocks partially and continue after a crash/shutdown/process kill?  ???

No idea what happens in Monero, but if so, more power to them.

In theory you could have the penalty coins vanish here as well, but that has several disadvantages.

In case it's unclear, T doesn't mean terabyte. It's just a parameter that can be configured based on our needs. At this point in time I suggest T = 3MB.

Oh Monero solution to block size limit

Block size limit is not important in the discussion.

The important is

+ Too many people don't understand block size and block size limit is
+ Political discussions: bitcoin for everybody or for elites only (lots of people try to ignore where bitcoin network effect comes from)
+ Group interest
+ Boys crying wolf: Viacoin, Blockstream devs

Finally a new opinion that has nothing to do with the 1 or 20MB blocks debate...

---

Just one question, Meni:
Why do you think that a target block size T (equal to 3MB) would be a good choice?

Thanks!

If T=3MB it's like a 6MB limit with pressure to keep blocks smaller than 3MB unless there are enough transactions paying fees so it's worth including them?

I think T should scale over time as bandwidth is growing. 42 transactions per second is still a low limit for a global payment network.

Just a comment on the following:


I don't think you can say that - that would be like saying, queues are never a problem as long as utilization is < 1 (which of course is required for stability). But long queues do in fact develop when utilization is < 1, due to variability in service / arrival times (re: bitcoin, the dominant source of variability is in inter-block times).

As long as there are queues, fee tension will be present, as mempool transactions are largely prioritised by feerate. Empirically, we are observing periods of fee tension (i.e. busy periods, when pools' max block size is reached) quite often these days.

Otherwise I like this perspective on the block size problem (even though I can't really comment on the proposed solution), in particular the observation that in the short term transaction demand is inelastic. (Upon further thought, however, proponents of increasing block size would say that the inelastic demand problem doesn't really apply if the block size cap is sufficiently higher than the average transaction rate.)

Yes. There is a quadratic penalty imposed for blocks above the median size (with a maximum size of 2 * median(size of last 400 blocks)), with a dynamic, elastic block sizing algorithm. Unlike Meni's suggestion, the reduction in block subsidy is not given to a pool but rather deferred to future miners because the subsidy algorithm is based around the number of coins.

See section 6.2.3 of the CryptoNote whitepaper: https://cryptonote.org/whitepaper.pdf

It was one of the most criticized components by the Bitcoin core developers, but so far testing on the mainnet and testnet has failed to evidence any failures.

There is a major shortcoming I can see in the rollover fee pool suggestion: miners are incentivized to accept fees out of band so they can obtain all the highest fees instantly, thus defeating the entire purpose of that feature.

By George you have done it old boy! This solves so many problems.


Fantastic, we can just take our best guess without pulling our hair out with worry of what disasters will insure if the guess is wrong. Perhaps 8mb + 20% per year elastic cap where those values can be adjusted with a soft fork.

What if the rollover fee could become the full node reward?

That is the main aspect of what monero/bytecoin does that I complained about-- that you can simply pay fees out of band and bypass it as the subsidy declines (even with the constant inflation the subsidy might be inconsequential compared to the value of the transactions if the system took off).  In Bitcoin this is not hypothetical since 2011 at least pools have accepted out of band payments and its not unusual for varrious businesses to have express handling deals with large pools; and this is absent any major reason to pull fees out of band.

My proposal (on bitcoin-development and previously on the forum) is effectively (and explicitly credited to) the monero/bytecoin behavior, but rather than transferring fees/subsidy it changes the cost of being successful at the work function.

I haven't had a chance yet to read and internalize the specifics of what Meni is suggesting (or rather, I've read it but the complete, _precise_ meaning isn't clear to me yet).  The main things to watch out for solutions of this class are (1) bypass vulnerability (where you pay fees, or the like, out of band to avoid the scheme)  and (2) scale invariance  (the scheme should work regardless of Bitcoin's value).  My proposal used effort adjustment (it's imprecise to call it difficulty adjustment, though I did, because it doesn't change the best chain rule; it just changes how hard it is for a miner to meet that rule).

I think this kind of proposal is a massive improvement on proposals without this kind of control; and while it does not address all of the issues around larger blocks-- e.g. they do not incentive align miners and non-mining users of the system-- it seems likely that proposals in this class would greatly improve a some of them of them; and as such is worth a lot more consideration.

Thanks for posting, Meni-- I'm looking forward to thinking more about what you've written.

I personally think a P2P network cannot rely on magic numbers, in this case 1MB nor 20MB blocks. The bar is either set too low and it creates an artificial choke point, or it is set too high which opens room for both DOS attacks and more centralization. As such, a solution that pins block size to fees is, in my point of view, the most sensible alternative to explore. Fees are the defacto index to determine transaction size and priority, so they are also the best candidate to determine valid block size.

However I have a couple divergence with Meni's proposal:

First, while I find the fee pool idea intriguing, and I certainly see benefits to it (like countering "selfish" mining), I don't believe it is a necessary device to pin block size limits to fees. Simply put, I think it's a large implementation effort, or if anything a much larger one than is needed to achieve the task at hand. It also requires modifying, or at least amending consensus rules, something the majority of the Core team has been trying to keep to a minimum. I believe there is wisdom in that position.

Second, I expect a fee pool alone will increase block verification cost. If it is tied to the block size validity as well, it would increase that cost even further. The people opposing block size growth base it on the rationale that an increase in resource to propagate and verify blocks effectively raises the barrier to entry of the network, resulting in more centralization. This point has merits and thus I think any solution to the issue needs to keep the impact on validation cost as low as possible.

Meni's growth function still depends on predetermined constants (i.e. magic numbers), but it is largely preferable to static limits. Meni wants to use it to define revenue penalties to miners for blocks larger than T.

I would scrap the fee pool and use the function the opposite way: the total sum of fees paid in the block defines the maximum block size. The seeding constant for this function could itself be inversely tied to the block difficulty target, which is an acceptable measure of coin value: i.e. the stronger the network, the higher the BTC value, and reciprocally the lower the nominal fee to block size balance point.

With an exponential function in the fashion of Meni's own, we can keep a healthy cap on the cost of larger blocks, which impedes spam by design, while allowing miners to include transactions with larger fees without outright kicking lower fee transactions out of their blocks.

As Meni's proposal, this isn't perfect, but I believe it comes at the advantage of lower implementation cost and disturbance to the current model, while keeping the mechanics behind block size elasticity straight forward. Whichever the community favors, I would personally support a solution that ties block size limit to fees over any of the current proposals.

It would not, in any meaningful way.

If we're making drastic changes it may be worthwhile to target balance of unspent outputs instead of / as well as block size, since UTXO size seems to be a more likely bottleneck than the network, unfortunate American core devs still connected to the internet using 1870s-style copper wires notwithstanding.

This part seems a bit off. At any given time, some people will have an urgent need for settlement, but many/most won't. So we get smooth scaling for quite a while from a purely economic perspective. Now once we reach a point in adoption where there are so many urgent transactions that they fill the blocks on their own, that kicks up the frustration to unacceptable levels, but even then some will be willing to outbid others and again it's a gradual increase in pain, not a deluge.

Insofar as the prices miners charge do rise properly and users have an easy way of getting their transactions in at some price, fees will limit transactions even in the short term. All you're really describing here is reaching a point that is pretty far along that smooth pain curve, after all the less important transactions have been priced out of the market.

Overall this is a great idea, though!

Well, the pool does have the ultimate effect of deferring rewards to future miners.

Interesting. I argue that regardless of other issues, the particular function they suggest is not optimal, and the cap it creates is too soft.

This is only (potentially) a problem in my 2012 rollover fee proposal. Here, tx fees don't go into the pool - fees are paid to the miner instantly. It is only the miner's block size penalty that goes in to the pool, and he must pay it if he wants to include all these transactions.

Of course, if you'd like to post this criticism on that thread (https://bitcointalk.org/index.php?topic=80387), I'll be happy to discuss it.


That's true in general; however, for the specific time scales and fluctuations in queue fillrate we have here, I'd say that "no fee tension" may be an exaggeration, but captures the essence.

Sure, if the block limit is high enough we can always clear transactions... But then there will be little incentive to give fees or conserve space on the blockchain. The post assumes that we agree that too low is bad, too high is bad, and we don't want to be walking a thin rope in between.


I don't think this is an issue here. Transaction fees are paid instantly in full to miners, so users have no reason to pay fees out of band. Miners are forced by protocol to pay the penalty if they want to include the transactions (and if they don't include the transactions, they're not doing anything they could be paid for). You could talk about miners paying penalty into the pool hoping to only get it back in future blocks, but with a pool that clears slowly enough, that's not a problem.

In the space of block sizes, you do need to occasionally update the parameter to match the current situation. I think it's best this way - currently only humans can reliably figure out if the fees are too high or node count is too low. But if a good automatic size adjustment rule can be found, it can be combined with this method.

In the space of Bitcoin value, transaction fees and hardware cost, the proposed function is multi-scaled and thus cares little about all of that. For any combination of the above, the function will find an equilibrium block size for which the penalty structure makes sense.

I'm still hoping Red Balloons, or something like it, will solve that problem.

Thanks, and likewise - I've only recently heard about your effort-penalty suggestion, I hope to be able to examine it and do a comparison.


Right, the proposal here only adds a constant amount of calculations per block, taking a few microseconds. It doesn't even grow with the number of transactions.

This is similar to the idea of eschewing a block limit and simply hardcoding a required fee per tx size. The main issue I have with this kind of ideas is that it doesn't give the market enough opportunity to make smart decisions, such as preferring to send txs when traffic is low, or to upgrade hardware to match demand for txs.

Another issue with this is miner spam - A miner can create huge blocks with his own fee-paying txs, which he can easily do since he collects the fees.

Using difficulty to determine the size/fee ratio is interesting. I wanted to say you have the problem that difficulty is affected not only by BTC rate, but also by hardware technology. But then I realized that the marginal resource costs of transactions also scales down with hardware. The two effects partially cancel out, so we can have a wide operational range without much need for tweaking parameters.

Obviously increasing the block size requires a hard fork, but the fee pool part could be accomplished purely with a soft fork. 

The coinbase of the transaction must pay <size penalty> BTC to OP_TRUE as its first output.  Even if there is no size penalty, the output needs to exist but pay zero.

The second transaction must be the fee pool transaction.

The fee pool transaction must have two inputs; the coinbase OP_TRUE output from 100 blocks previously and the OP_TRUE output from the fee pool transaction in the previous block. 

The transaction must have a single output that is 99% (or some other value) of the sum of the inputs paid to OP_TRUE.


By ignoring fees paid in the block, it protects against miners using alternative channels for fees.

are you suggesting we drop btc and pick up vtc?

This is the most attractive concept I have seen yet for dynamic scaling which places a penalty by increasing the required difficulty target for miners building >1MB blocks. Do you have any idea of how that penalty could be calculated? I assume it would scale according to the percentage size increase above MAX_BLOCK_SIZE. I believe this would work because miners would not be incentivised to build builder blocks there was a need to, because prematurely doing so would put them at a disadvantage. This would also help in building fee pressure which will become more and more important as subsidy decreases.

An elastic supply is very important, but I think it can be accomplished more simply, without a pool.

Allow blocks to be expanded beyond their "nominal" size with high fee transactions.  The higher the fee, the further it can appear in the block.  Formally, define a function fee = T(x), where x is the location in the block.  If a transaction's fee is >= T(x), it can be placed in the block at location x.  T(x) = 0 for all x < 8MB (say) and increases super-linearly from there.


Note that this proposal does NOT look at the fees in aggregate -- the max block size <= S(sum(fees)), where S is some super-linear function.  That does not work because a miner could create a dummy transaction that pays himself a very large fee, thereby increasing the block size to allow space for a lot of low fee transactions.

Meni may have added the idea of a pool to solve the above problem.  But I believe that it is more easily solved by not looking at fees in aggregate.


EDIT: the biggest problem with this class of proposal is sizing the fee.  Especially given bitcoin's volatility.  However, if the fee function chosen starts at 1 satoshi, a high bitcoin price will tighten the elasticity of supply (in practice) but not entirely remove it.  At the same time, we STILL need to grow the "nominal" block size: i.e. 8MB + 20% per year, or risk pricing out personal transactions as adoption increases.  However, this class of proposal allows the network to react in a classic supply/demand fashion.  This reduces the pain when supply is exceeded, meaning that a "last-minute" hard fork as proposed by many of Gavin's opponents would be a lot less damaging to the network (block size increases could trail adoption rather than precede it).

As far as I can see, and given that:


Then it would be possible to raise the block size limit to 6, 20, 40, ... MB, but introduce a soft cap and a penality mechanism for "large" blocks. The penality function (and thus soft cap) may be freely adjusted over time, as long as the resulting block size doesn't exceed the hard limit.


Very well put, I like it.

+1 to that.  I think a max thats determined as either:
T = 2.50*(average(last 8000 blocks))         #T is set the average transactions for the last ~2 months. Plenty of room for slow and steady growth, and too great a timespan to attack the blockchain with spam. keep in mind that transactions at night will probably be 1/5th the volume of those during business hours
or
T = (2.00*(average(last 8000 blocks))) + (0.50*(average(last 144 blocks)))     #This would allow short-term fluctuations that take a day or two to develop. Could be susceptible to a spam attack that last longer that 3+ days.

personally, i think a block limit thats set based on the average volume of the last 1-3 months would be fine. It would be flexible if the number of transactions increases very quickly, and could grow to 3-8x the maximum within a year if theres substancial volume. combined with your proposal above it could be extremely flexible. However...

I'm EXTREMELY cautious of altering how fees are created and distributed, as any changes made will directly impact miners and could lead to bribery and corruption of the bitcoin code to better pay the centralised mining companies. Any code changes that are implemented should not involve factors or values that will need to be adjusted down the road, or it will simply lead to a 'democracy' of core-qt 'improvement'

It's difficult to know exactly how the quantitative factors will play out exactly. The inelasticity is not total, but I believe it is significant, and contributes to the phenomenon. Even if things will not be as catastrophic as Mike describes, I believe they can get rather ugly, so any change that alleviates it is welcome.


Not familiar with it.


This could work, but:

1. I'm not convinced it's actually simpler. If I understand it correctly, it requires, among other things, sorting the transactions by fee. Verification also requires examining each individual transaction in a rather elaborate way.
2. I think it's much harder to analyze how it will play out economically; and my initial thought is that it will be less stable. In my suggestion, the fee will be more or less consistent over txs, for any given load level. Here, some txs will be accepted with 0 fee and some will require very high fees; it will be difficult for each transaction to decide where it wants to go, and they can oscillate wildly between states.


This is the reason I chose a hyperbolic function rather than a polynomial one. Being hyperbolic means a wide range of marginal costs is covered with a relatively small span of block sizes. So whatever the reasonable fee should be, the system will find a block size that matches it.

My email correspondence with Gavin so far:

That's not a real limitation. It could easily grow to more than 100Mb in a single week.

I think it should be more like:
max block size = 1.2 * average block size of last 1008 blocks

Using the median instead of the average would make the scheme less prone to the influence of just one or few blocks.
Again, we are describing precisely the scheme used in Monero, proposed in the cryptonote whitepaper (see Sec. 6.2.2): https://cryptonote.org/whitepaper.pdf

lol he just called you an ideas man, your efforts are futile, Bitcoin is the Eldorado and has no flaws no dev will ever adopt something made by other coins, it would imply Satoshi was slight equivocated in his first attempt at blockchain :D

Why would a mod delete this comment from this thread? It was the second comment and it was completely related to the topic.

A reply of yours, quoted below, was deleted by a Bitcoin Forum moderator. Posts are most frequently deleted because they are off-topic, though they can also be deleted for other reasons. In the future, please avoid posting things that need to be deleted.

Quote
Brilliant!

lol man I remembered you when I saw this thread on reddit: https://www.reddit.com/r/Bitcoin/comments/389pq6/elastic_block_cap_with_rollover_penalties_my/
:P

This is quite the idea... It has definitely some legs to run the long run.

What bothers me is really how to implement it... I'm also not knowledgeable in code, but the pool seems a pretty complex thing to setup. The funds would have to reside in an address. Who would hold such private key?

Gavin ofc, he owns Bitcoin now like his ideological fellow at darkcoin/dash.

Holy Cow I was unaware of this thread, thanks I have some reading to do at work now.

And the initial reason I commented on this post is because nobody else had and it would have been a shame to see it fall off the first page with no responses.

That's not really what he said. I am mostly an idea man though, and happy to be one.


Well, I think there is some policy about writing comments that don't add new content of their own and just support previous content, or something. I was happy to see your feedback. As I recall, you posted your comment quite early, before there was a real need for bumping, but I appreciate the intent.


No, the funds don't reside in an address. That's like saying that the 6.8 million bitcoins that haven't been mined yet reside in an address.

The funds just exist as a feature of the network, and the protocol defines how they are paid out to future miners (in the same way that the protocol dictates that each miner currently gets 25 BTC).

I don't believe the implementation is that complicated, but people more familiar with the codebase are in a better position to comment on that.

Rusty Russell did some modeling on that today: What Transactions Get Crowded Out If Blocks Fill? (http://rusty.ozlabs.org/?p=500)

I assume you are referring to the debate on "hard block size limit + organic fees" versus "no block size limit + hard fees", the third option (no block limit and organic fees) being a non solution. Obviously an "organic block size limit + organic fees" is the ideal solution, but I think the issue is non trivial, and I have no propositions to achieve it. I don't even know if its philosophically possible.

In this light, a "pseudo elastic block size limit + organic fees" is the better and most accessible solution at the moment, and I will argue that my proposal cannot be reduced to "no block size limit + hard fees", and that it actually falls under the same category as yours. Indeed, like your proposal, mine relies on an exponential function to establish the fee expended to block size ratio. Essentially the T-2T range remains, where any blocks below T needs no fees to be valid, and the total fee grows exponentially from T to 2T.

In this regard, my proposal uses the same soft-hard cap range mechanics as yours. As I said, ideally I'd prefer a fully scalable solution (without any artificial hard cap), but for now this kind of elastic soft-hard cap mechanic is better than what we got and simple enough to review and implement. The fact that my solution has caps implies there will be competition for fees as long as the seeding constants of the capping function are tuned correctly. On this front it behaves neither worse nor better than your idea.

Since I believe fees should be pegged on difficulty, fees wouldn't be hard coded either. Rather the baseline would progress inversely to network hashrate, while leaving room for competition over scarce block room.


I will again argue to the contrary. As a matter of fact, I believe your solution offers no room for such adaptive market behavior, while mine does. To take both your examples in order:


With T being the soft cap and 2T the hard cap, your solution proposes to penalize all miners creating blocks larger than T. This pegs blockchain space to fees, the same as my proposal: the more txs waiting in the mempool, the higher the fee you need to get included in the next block. Inversely, the less items in the mempool, the more likely you are to have your low/zero fee tx mined right away, which creates an incentive to emit transactions during low traffic periods.

While your approach supports emitting txs during low traffic by imposing extra costs on high traffic, mine simply doesn't do with the extra cost. That doesn't mean emitting transactions during low traffic is NOT cheaper. As a matter of fact, it is, but the difference between low and high traffic isn't as significant.

The true difference between my solution and yours is that while mine allows miners to exceed the T soft cap as long as there are enough fees to go by, yours penalizes all blocks outgrowing T, which effectively locks all blocks at size T.

Indeed a selfish miner would have no incentive to build blocks beyond T, and they will also benefit in not including 0/low fee transactions. Indeed, by leaving all 0/low fee transactions in the mempool and only creating small blocks, where the block size is defined min(total size of high fee txs, T), a large selfish miner can deplete the mempool from all high fee transactions, leaving the rest of the network to pick up the slack.

Other miners are left with the choice to fill blocks up to T, but not further. Due to the selfish miners action (who are pumping high fee, small size blocks), there are not enough fees to be redeemed from the mempool, and the penalties for including transaction past T would come out of the good willed miners' coinbase rewards. You may have "benevolent" miners who would rather empty the mempool than follow game theory, but they only stand to earn less money than everybody else. On the other hand, selfish miners still qualify to get a cut of the fee pool, to which they make a point not to contribute to, effectively siphoning revenue from good will and "benevolent" miners.

The true effect on the network is that no one will bother creating blocks larger than T, and we will still have a defacto hardcoded block size cap.

My proposal offers to allow miners to take in extra fees as long as they remain below the curve defined by the capping function. Selfish miners do not have an opportunity to vampirize good willed miners anymore so while the entire behavior (high fee, low size blocks) is not deterred, it is at least not encouraged.

Please keep in mind that this analysis of your system relies on my current understanding of it. I'm still not 100% clear how the "fee pool" functions. I'm assuming it is either only funded in penalties, and all fees are paid to miners directly, or that all fees are pooled and distributed equally per blocks. The later assumption seems pointless since it can be easily bypassed, so I'm using the former one as the basis to my critics of your proposal.


Again I will argue that my proposal supports hardware improvement while yours doesn't. In your case, T will act as a defacto hard cap on block size limit, so there is no incentive for miners to be able to handle more traffic and create blocks beyond that value. As long as miners won't output blocks larger than T, there is no reason for the rest of the network to upgrade either.

With my solution, as long as there are enough fees to go by, up until 2T blocksize (or whatever the hard cap ends up being), miners are motivated to include transactions paying fees beyond the soft cap, which justifies hardware improvement to handle the extra load, with the consequences this has on the rest of the network.


Both in the current state and the solution you propose, malevolent miners can disturb the network by mining either empty blocks, or blocks full of "useless transactions", sending their own coins back to themselves. In my solution malevolent miners also have the added opportunity to mine large blocks by paying fees to themselves. Let's analyze what counter there is to these 3 disturbance attacks.

1) In case of empty blocks, all good willed and selfish miners should simply ignore them. It increases their revenue and impeaches the attack.

2) In case of "useless transactions", either the transactions were never made public, in which case the blocks are easy to identify (full of txs that never hit the mempool) and can be ignored for the same reason as above, or the transactions have been published publicly, and the attacker is making a point of mining only these. At this point you can't really distinguish this miner from good willed ones with either solution.

3) With my solution, malevolent miners can pay themselves fees and enlarge blocks. However that only holds true if they are keeping the transactions private. In this case other miners can identify such blocks as malevolent and ignore them entirely (2T wide blocks full of large fee txs that never hit the mempool). If the attacking miner makes the transactions public, he can't expect to rake all fees, so the solution to this attack is no different from 1 & 2

4) With your solution, what is to stop a malevolent miner from maxing blocks with 0 fee transactions? Sure he would give up the coinbase reward in form of penalties, but now these are available for the taking to other miners. Good willed miners may ignore such blocks, but selfish miners probably won't.

With my solution, there is an incentive for both good willed and selfish miners to ignore blocks outright constructed to bloat the network. With yours, there is a built in cost to such disturbance, so selfish miners can choose to ignore the disturbance for the benefit of the reward. In my system, the only viable economic option is to ignore the blocks.


2 years ago I would have opposed pegging fees and block size to difficulty, because ASIC technology was catching up to current manufacturing processes and as such was growing much faster than every other hardware supporting the network. That would require too much manual adjustments of the pegging function to be acceptable. As time passes by, that criticism loses ground, and now is not a bad time to consider it.

I would be interested to see if you have room to factor difficulty in your current function.


I try to not be so quick with drawing such conclusions. I'm not savvy with the Core codebase, but my experience with blockchain analysis has taught me that the less complicated a design is, the more room for optimization it has. You can't argue that adding a verification mechanic will simplify code or reduce verification cost, although the magnitude of the impact is obviously relevant. I'm not in a position to evaluate that, but I would rather remain cautious.

The point still remains, you don't need a fee pool to establish a relationship between fee, block size, and possibly difficulty.

Don't get me wrong, I believe the idea has merits. What I don't believe is that these merits apply directly to the issue at hand. It can fix other issues, but other issues aren't threatening to split the network. I also don't think this idea is mature enough.

As Gavin says, without an implementation and some tests it is hard to see how the system will perform. If we are going to “theorycraft”, I will attempt to keep it as lean as possible.


It seems your implementation pays the fee pool in full to the next block. That defeats the pool purpose in part. The implementation becomes more complicated when you have to gradually distribute pool rewards to “good” miners, while you keep raking penalties from larger blocks.

Otherwise, a large block paying high penalties could be followed right away by another large block, which will offset its penalties with the fee pool reward. The idea here is to penalize miners going over and reward those staying under the soft cap. If you let the miners going over the cap get a cut of the rewards, they can offset their penalties and never care for the whole system.

As a result you need a rolling fee pool, not just a 1 block lifetime pool, and that complicates the implementation, because you need to keep track of the pool size across a range of blocks.

Hey no problem. And I always feel slightly smarter after having read one of your theses, so thank you for that. As no doubt others have told you in the past, you have quite a gift for thinking outside the box in a non-profiteering sort of way, a quality which the cryptocurrency community is in dire need of.

Cheers from Hawaii,

Nutildah the Hungry

I really like Meni's elastic block cap proposal.


Absolutely. It's worth clarifying that the actual mechanics of how the pool fee works, (particularly how it is calculated), does not have to be on the critical path to resolving the 1MB problem.
All that is necessary today for Meni's proposal is to agree that a pool fee, of some make-up, can usefully exist.

Then it is possible to look at a simple implementation plan which overcomes the urgency of dealing with the 1MB, does not raise the limit too much, and allows time for an elastic cap with rollover penalties to be fully worked out, modeled, developed, and tested. As mentioned before, the pool fee could incorporate a function of block delta utxo and sigops.

Phase I:
Hard Fork to increase the max block size to 2T, e.g. via block version 4.
2T might be in the region of 6 or 8MB which also scales at a fixed percentage each year, say 20%, or a fixed multiple (e.g. 4x) of the recent average 144 or 2016 blocks. However, the difference this time is that no miner will be able to mine a block larger than T without paying a pool fee, but this won't be possible because it requires a supermajority on version 5 blocks to vary the pool fee from zero.

Phase II:
Soft fork to implement the full elastic cap, effective by supermajority, e.g. via block version 5, when blocks between T and 2T can then be mined.

Advantages:
Decoupling dealing with the hard-limit, from the making of a graceful decay in network performance as the limit is approached.
No urgency on how to best to set the pool fee, lots of time for debate and modelling.
A yearly scaling percentage can be more approximate because it should be easier to schedule hard-fork revisions to this as and when changes in global computing technology dictate.

Meni

Thanks for this great proposal.  Please could you explain how the rollover fee pool helps out and why the penalty cant just be burnt? Which would be more simple.


Just so I can understand properly, your penalty is related to a formula that excludes the transaction fees, therefore paying fees out of band won't reduce the penalty.  Is that how the problem is solved?

I think this idea seems really good, but I need to think a bit more.

Nice proposal. It is exciting to hear some carefully thought, incentivize based design


I think the current design already incentivize smaller blocks: Smaller blocks get broadcasted much faster and become less possible to be orphaned. If you consider that there are 25 coins to compete for, you would like to broadcast your block as fast as possible once you find it

With bigger blocks cap, it becomes more favorable to mine smaller blocks. 10MB blocks have a very high risk of being orphaned by 1MB blocks, since the time needed to broadcasting them will be much longer than 1MB blocks, maybe several minutes longer

However, if everyone is aiming for the smallest block possible, then most of the transactions will not be included. So far we have not run into this problem because the broadcasting speed is still relatively fast at current block size. But if one day it happens, a natural result is the bigger blocks will ask for more fee due to higher risk of being orphaned, how mathematically it is formulated is difficult to say without some real world cases. I guess it will be very similar to what OP described, above certain threshold, the fee will increase exponentially due to the risk of being orphaned also increase exponentially

While searching for block propagation data, I found out an article from Gavin. His Invertible Bloom Look-up Tables proposal will incentivize all the miners to include similar set of transactions to speed up the block propagation time (miners do not need to broadcast transactions that other peers already have in their memory pool, only block header and some extra transactions), but that seems to be a major change further down the road

https://gist.github.com/gavinandresen/e20c3b5a1d4b97f79ac2

I like this proposal. It takes away much of the destructiveness of hitting the block size limit. Makes 'impact' softer and actually allows for a market to function: with higher aggregate fees (higher tx demand), economically usable blocksize can actually increase, which is not the case with the current 'hard' implementation: no amount of higher aggregate fees will increase block space. That doesn't facilitate a market.

It would be interesting to hear from some core devs. Sounds to me the proposal could be acceptable to Peter Wuille and maybe even Peter Todd, for example, since the solution still offers to deliver economic incentive to develop scaling solutions.

Maybe this really could be (worked into) something with broad consensus.

I understand we will still argue about T, but it will be much easier because the consequences of choosing it wrongly aren't as dire.

Why not consider % of transactions with a fee?

This way users have direct influence in block size by paying a transaction fee (above median).
With paying a transaction fee users have a vote in the 'free rider effect'/spam and if current levels are acceptable.

1. small block size -> only transactions with fee will be processed fast (others get parked in mem. pool)
2. big block size -> a lot of transactions have a fee and a lot transactions without a fee (spam?) are accepted
3. median block size -> a balance between transactions with and without a fee

Also added a bigger scale (due to near future (holiday) transaction peaks).

I used Gavin's proposal as starting point:

I think the above simplifies your equation slightly, and uses better timeframes (~200hrs and ~50hrs) to compute the averages.

my biggest concern is what would happen in a spam attack, which could easily last several days, to as much as a week, if a quick-changing algorithm can be played to drastically increase blocksize during that time.

lets look at your original equation: assume that blocks are 0.5MB for the last 144 blocks (thus the block limit will probably be set to something like 5-10MB), but then spam attacks start. you see every block filled with 5MB of spam, 95% of transactions below the median for fees. after 144 blocks of this (1 day), you get something like this:
which seems to resist growing any larger.

however, the median transaction fee drastically drops as a result, and in a continued attack soon you could see that ~10% of transactions pay more than the median fee.

   If this attack persisted for 3-4 days you could see the blocksize increase to >20MB very fast, and >60MB within a week (would likely require very well-funded spammers though)

longer timeframes are absolutely necessary, bare minimum should be 600 blocks for calculations.

Since the blocksize issue is controversial and may take some time to settle, we are better off implementing this elastic cap right now with a softfork (your phase II) and skip the hardfork part (phase I).

We could do that by choosing T=0.5MB (2T = 1MB = current maxblocksize).

I think it's interesting to switch the conversation over to a soft-failure rather than trying to find the appropriate answer to the block size for now.

That said, a problem with any kind of rollover fee is that it assumes that moving fees to future blocks is also moving fees to different nodes.

Put differently; centralizing nodes is a way of avoiding the penalties you're trying to introduce with this protocol.

Put differently again; Paying fees over consecutive blocks gives a competitive advantage to larger mining entities when making larger blocks.

Put triply differently; A node that can reliably get another block within X blocks is less penalized than a node that cannot, where "X" is the number of blocks that the rollover fee is given.

So if the goal is to avoid centralization, then the protocol does the opposite of the intention. If the goal is to make Bitcoin fail-safe, I'm not convinced that Bitcoin isn't already. When blocks fill, we will see higher transaction fees, potentially lengthier times before a transaction is included in a block, and as a result more 3rd party transactions.

Rereading Mike Hearns article¹, changing bitcoin to include highest fee transactions into the mempool should result in the behavior I described. An end user might see delays before their transaction is included in a block, but I wouldn't call that a "crash landing", considering that the sorts of transactions that would be done at these rates are not as concerned about the speed of confirmation.

TLDR: How does a fee over "X" blocks not incentivize a centralization of nodes?

¹https://medium.com/@octskyward/crash-landing-f5cc19908e32

And if we use:

This excludes zero fee transactions in the calculation of the median transaction fee.
In that case a large quantity of zero fee transactions (and spam) does not have influence on the median transaction fee.

perhaps a threshold? The problem is that spam is usually sent as a single transaction with 0.001btc fees and hundreds of recipients 0.0000001BTC that get. There should be protection in place that ensures sending thousands or millions of dust payments becomes prohibitively expensive, without limiting its use for colored coins or increasing fees to the regular user.

right now, the only sort of filter for this is the miners deciding whether to include or not include a transaction, and will include a spam transaction if theres a justifiable fee to do so.


IMO, the simplest way to specify the max size is as
That provides a max size thats 3x the size of the average block over the past 1000hrs (41days). If demand suddenly rises, the max size can double within a week.

say current block is ~0.5MB on average. The new size cap under this formula is 1.5MB (50% larger than it is now)

assume by the end of 2015 theres now 2MB/block on average, with business hours being around 5MB/block. assume by the end of 2016 theres now 12MB/block on average, with business hours being around 30MB/block. assume that in the following week bitcoin gets popular, and suddenly there are 30-40MB blocks almost 24/7. within 4 days, the blocksize will increase to

Apparently, neither does Gavin.
He said he didn't want to talk to you until there was working code that does it?
Such code has been working for years, but people forget where the experimentation is occurring, the alts.
A good place to start:
https://github.com/monero-project/bitmonero/blob/54fbf2afb3bc029823ed6c200e08bd21fe42ac10/tests/unit_tests/block_reward.cp
and
https://github.com/monero-project/bitmonero/blob/c41d14b2aa3fc883d45299add1cbb8ebbe6c9ed8/src/cryptonote_core/blockchain.cpp#L2230-L2244

Can you explain a bit about the mechanism wherein the miner pays into the rollover pool, and why that is different from the 'original proposal'?  It is not obvious why this dictinction makes a difference.  It seems to still incentivize out of chain payments to miners for transaction inclusion regardless of whether it is paid by the miner, or deducted from the miner's reward, both are dependent on the fees in the block (which aren't there in out of block payments schemes).

The difference is quantitative. In this version the rollover effects only blocks that exceed a threshold of size.

OK, so its similar to Monero.
But there is a difference between this proposal and what Monero does that appears that it might need to be addressed.

The rollover pool creates an incentive for the miner to not use the fee pooling, and instead contract directly with the TX creators.
If implemented as written, this could become a problem.  Large TX creators and large miners would have an incentive to cartel because of the way this rollover pool works.

Monero avoids this problem, but most of the rest of this proposal has been implemented and running for quite a long time.  Its not new, or novel, except in ways that it is not as good.
An examination of the prior art is warranted.

I think the penalty is not dependent on the fee:


So it still has to be paid regardless of wether or not the tx fee payment was on- or off-chain.

This is penalty then ONLY taken from the coinbase reward and not also TX fees?
How would this method survive the successive halvings (much less the eventual cessation)?

If it includes the TX fees (which likely it must), then to the extent that fees are off chain, they are not subject to the penalty.

When Gavin says "I need to see working code", he probably means code he can directly deploy to his test setup within the framework of bitcoin-core. I can relate to this demand and find it reasonable.

Sure, there are quite a few edits that would be needed.
However, it has also been deployed and shown to work over time without pernicious economic effects. 

I read that as "from reward, tx fees and rollover pool input".


I stumbled accross this problem, too and I remember it was adressed upthread, not very far from OP. I just searched, but couldn't find it and I also can't remember why (or wether) this wasn't a problem. I think I wasn't convinced or didn't think it through.

But thinking about that a bit, assuming 0 block reward is reached. A block with no transactions in it can always be mined because it doesn't carry any penalty at all. As soon as there's a penalty to be payed there need to be some transactions in the block and the fees they pay must exceed the penalty (otherwise the block is invalid). Is this a problem? I'm not sure, seems to me that depends on the particularities of the penalty function...


 

Thanks for that, it was my reading also.
Thus TX fees that are not in the block but paid out of band are not subject to penalty...

Another difference, is that in the Monero method, the penalty can not reduce the block reward to below zero and make the block invalid.
There are many similarities, but also some important differences with the time tested Monero method and the current proposal.

No, that wasn't my understanding (the bolded part doesn't follow from what's said above, does it?). Fees aren't subject to penalty at all. The penalty depends purely on size of block (if I understand correctly). Even if the tx fee was payed out of band, the tx will still contribute to the penalty through increased block size. Yes, it's deducted from the tx fees (can be other tx fees), but it depends (in amount) on the fact that the tx takes up space in the block.

In other words: it's not the fee that's penalized (neither in block nor out of band), but the space it takes up in the block.

Again: if I understand correctly.

EDIT: regarding the discussion about what happens with block reward going to 0: this means that you can't mine a block with only transactions that had their fees paid out of band, because they would take up space and thus effect a penalty, but there'd not be enough on-chain fees to pay for it.

I see your point in that the amount of the fees are not part of the calculation of the penalty.
However, why would anyone mine a block for which they will not be paid?  
The incentive would remain to move fees off chain, especially as the coinbase reward decreased.
Off-chain fees would not be subject to the seizure and redistribution of the rollover.
Better for the miner to guarantee positive revenue and get it on the side.

I still don't follow you.

Why does it matter wether the fee is payed on- or off-chain. The penalty is the same and the miner has to pay it either way.

By miner has to pay it, we mean "not paid to miner but to rollover pool instead"?  They miner is never paying anything, they are simply not given the reward, yes?

No, by "miner has to pay it", we mean he always has to pay it, even if he receives no on-chain fees.

I'll try to examine your proposal in more detail later.


Looks correct, but I would say there was no problem to begin with.


This is correct but as far as I know, the magnitude of this effect is very small, and not enough to keep block size in check.


This is correct, and I hadn't given enough thought to this problem prior to posting.

Now that I've given it more thought, I think it can be significantly alleviated by making collection from the pool span a longer period, on the time scale of years. Relative hashrate is likely to change over these period, so it may not be the best plan to publish excessively large blocks hoping to reclaim the fee (and publishing typical-sized blocks does not give big miners an advantage). Also, with a different function you can change the balance between the marginal and total penalty so that the actual penalty is small, nullifying the effect (it will require the cap to be a bit harder, but still more elastic than what we have now).

I agree that this calls for more analysis.


Not much to explain. The penalty is deducted from the amount of bitcoins he can credit himself in the generation transaction; The network keeps track of the total pool size, and allows miners to add a fraction of it to the bitcoins they credit themselves.
Instead of
total outputs = new coins + tx fees
You have
total outputs = new coins + tx fees + collection from pool - penalty

The penalty doesn't depend on the fees of txs in the block. It depends on the size of txs in the block. Out-of-band payments are irrelevant - the miner must include the transaction to perform the service he would be paid for. If he includes it he pays a penalty for it, and expects a fee for it - it doesn't matter to either him or the user if the payment is out-of-band or not (except, of course, the extra friction of out-of-band).

This is simply false, as explained above. It seems many people are confusing the current suggestion, with the suggestion I brought up back in 2012 and referenced here.

It's an additive deduction, not multiplicative.

You seem to be thinking the miner's reward is:
(1 - penalty) * (minted coins + tx fees + collection from pool)
Where it is really:
minted coins + tx fees + collection from pool - penalty

Having more fees in the txs included doesn't increase the penalty. There is no difference between adding 1 mBTC to the fee and paying 1 mBTC out-of-band.

The miner chooses how many transactions to include. The penalty starts at 0 and stays there for a while (or alternatively, has a derivative starting at 0). He chooses the set of txs so that his profit is maximal, not so that it is negative...

(PS I apologize for the multiple comments on the same issue, I'm trying to respond to all claims in the order I see them...)

I'll need to examine the Monero implementation in more detail to answer that intelligently. One difference is that Monero, apparently, already has a mechanism to defer rewards to future miners, so it has no need to introduce the extra concept of a rollover pool. Additionally, I suggest a different scaling function (hyperbolic rather than quadratic). It has a different set of tradeoffs (which I currently believe is better overall). I could suggest a different function based on the requirements - finding the best functions to manifest given intuitive concepts is what I'm good at.

I agree with the last part. I, too, am upset when people try to reinvent the wheel. But it's difficult to know whether some feature exists in some system somewhere. I've read several discussions of block size and didn't see a mention it. I started with privately chatting with Gavin and Mike, and they seemed unaware of it either. I don't think it was reasonable for me to do anything but post about it and see what people think, which I did. If I had known about the situation in Monero I may not have made this post.

I'm glad that I did, though. As we can see from the lively discussion here and on reddit (http://www.reddit.com/r/Bitcoin/comments/389pq6/elastic_block_cap_with_rollover_penalties_my/), there are obviously many people who are interested in this kind of solution but who, like me, were not previously aware of Monero's system. I think wider awareness and discussion of these issues is a good thing.

You're both correct, it's a matter of terminology.

The miner has to pay it. But he's not paying by spending any of his previous coins, he's paying by having the payment deducted from the reward he collects. If the reward is not sufficient (negative reward after deduction), the block is invalid.

So the miner will not include so many transactions that the penalty will make the block invalid.

See my previous comment and I hope everything is clearer now. I apologize for not intervening sooner, I had a busy day, maybe this whole exchange could have been avoided...

I agree. Also with IBLT or other block propagation bandwidth conservation techniques (some of which seem to be used by pool interconnects already) this effect can (and will) be nullified.

Thanks for clearing it up. I can see now where NewLiberty was probably coming from (either the monero mechanism or your earlier proposal referenced in OP, both of which I didn't look at). Your reward calculation formulas cleared things up.

A longer time period over which the reward is given doesn't help, as the larger nodes or entities will still get a larger ratio of the rolling over fees, by definition.
Actually, making the rollover fees only extend over a couple of blocks would more likely mitigate the problem, but if you roll over the fee for about 3 blocks or so, then it might be worth it for a miner to hold blocks and release 2 at a time, depending on fee-over-time. This, in turn, might exacerbate the selfish miner exploit¹. The natural monopoly condition that already exists in Bitcoin² seems to be exacerbated either way.

Getting around this would be tricky, if it's possible.

¹http://fc14.ifca.ai/papers/fc14_submission_82.pdf
²https://bitcointalk.org/index.php?topic=176684.msg9375042#msg9375042

Pointing to Monero as an examination of prior art is asking a bit much. Are you expecting us to dig through the Monero source code? How do they get around the problem?

This is not very helpful;


https://getmonero.org/knowledge-base/moneropedia/coinbase

It's a polite way to say :"i am not interested in your design, i dont want to analyse it, f**k off. "

I'd be in his position I'd would too ask to see some code or at least some data analysis supporting the design. You can't just propose stuff and expect the people reviewing it to do all the leg work. An implementation at least proves your design is conceptually sound. It's easy to forget certain aspects when you theorycraft, and having to implement at least the PoC certainly motivates you to keep it as simple as possible.

True, if the circumstances were different. Presently the average (7-day, moving) block size (ABS) is about 400KB, 80% of T, and T will certainly be less than the ABS by the time an elastic cap can be implemented. ABS will never approach 2T because some miners will always churn out small or empty blocks (certainly while the reward > block fees), so the present ABS is for practical purposes equivalent to T now. The point which would normally be an alert for attention to be given to the prevailing limit.

The elastic cap with fee pool does need working out in detail, as the discussion between molecular and NewLiberty shows. Its mechanics, incentives, attack vectors and long-term implications need to be known and understood. This won't happen quickly. Phase I buys time for Phase II.

Increasing the 1MB is technically very simple, accepting just the usual risk inherent in larger blocks.

If this type of consensus can be hand-waved away, then we are going to have to go home and learn to love our respective fiats, because they will all be around for a lot longer.

While writing a response to this, I realized the situation is actually much more nuanced than I thought.

Big miners don't have an advantage over small miners. Rather, the existence of big miners shifts the balance of the system as a whole, creating a situation where miners get more rewards, users pay less fees and/or get more of their transactions included, while nodes have to deal with larger blocks.

Here are example scenarios, with made up values for the penalty function. I assume for simplicity (not a necessary assumption) that there is endless demand for transactions paying 1mBTC fee,  that typical blocks are around 2K txs, and that there are no minted coins. The pool clears at 1% per block.

Scenario 1: The network has 100 1% miners.
Every 1% miner knows he's not going to claim much of any penalty he pays, so he includes a number of transactions that maximize fees-penalty for the block. This happens to be 2K txs, with a total fee of 2 BTC and penalty of 1 BTC.

The equilibrium for the pool size is 100 BTC.
Miners get 2 BTC per block (2 fees + 1 pool collection - 1 penalty).
There are 2K txs per block.

Scenario 2: The network has 1 90% miner, and 10 1% miners.
The 1% miners build blocks with 2K txs, fee 2 BTC, penalty 1 BTC, like before.
The 90% miner knows that if he includes more txs, he'll be able to reclaim most of the penalty, so the marginal effective penalty exceeds the marginal fee only with larger blocks - say, which have 4K txs, 4 BTC fees, 4 BTC penalty.

The average penalty per block is 3.7 BTC. The equilibrium pool size is 370 BTC.
There are on average 3.8K txs per block.
A 1% miner gets, per block he finds, (2 + 3.7 - 1) = 4.7 BTC - more than in scenario 1!
The 90% miner gets, per block he finds, (4 + 3.7 - 4) = 3.7 BTC - less than small miners get in this scenario, but more than miners get in scenario 1!

To restate what goes on - the big miners create supersized blocks because it benefits them, but it benefits the small miners even more.

Miners are happy. Users are happy. Nodes are not happy.

But note the following: Big miners make the mining rewards bigger, but you don't have to be a big miner to enjoy it (small miners actually are at an advantage). So if a miner becomes big, he encourages more people to start mining, raising the difficulty and countering the effect.

So more analysis is still in order, but overall, I don't think these dynamics encourage the formation of big miners.

Not sure to which extent this is criticism to me. But I believe everyone has a part to play in this world, and should be doing what he's best at. My comparative advantage is in coming up with ideas and discussing them; and in unrelated work (to those who don't know me, my day "job" is in promoting Bitcoin in Israel). It's not in coding and empirical analysis - I'll leave that to others. This methodology worked quite well at the time I helped mining pool operators with implementing DGM. Perhaps the discussion I've started will result in this or a similar idea being implemented and accepted. But if not, so be it.

I'll clarify that I think Gavin's request is perfectly legitimate. I didn't exactly expect him to be so dazzled by the idea that he'd drop everything he was doing and start working on it.

It's not criticism directed towards anyone per say. In the course of my work with Armory, I get suggestions to implement this and that, but an idea that can be summarized in a single sentence can often demand 10k LoC. I'm much more inclined to look at a pull request than just some formulated concepts. As I said, having a PoC to support the idea has several advantages, one of which is to make the task of reviewers simpler, another which is to go through the most obvious optimizations right away. An idea without a PoC is not diminished, but an idea with a PoC is certainly improved. I felt like I should share that. It wasn't even an attempt to defend Gavin.

Obviously if you can find someone to work a PoC for your proposal, that would be fantastic.

You're an idea man, I'm a nuts and bolts guy, I can't help but look at this from my perspective. Your natural stance towards people with my skill set is "you don't sophisticate enough". My natural stance towards people with your skill set is "you complicate too much". This isn't about to change anytime soon, yet that doesn't make it a personal attack. Present a patient with some general syndrome to N different medical specialists, all in different fields, and they will come up with N different diagnosis. They're not all necessary wrong.

If you think there is some underlying ad hominem in my criticism of your proposal, that is not my intent. There are plenty of other sections in this forum which are ripe for this kind of rhetoric. I'm going to defend my point of view with every opportunities I get, I don't expect less from others. The intensity of the criticism may come across as unwarranted but that's only cause I'm genuinely interested in this discussion. That should vouch on its own for the importance I bear to theoretical research.

This is encouraging... it sounded yesterday as if you had almost regretted making this thread and were about to pull your own support from the proposal because of this and now it looks like it might be less of a problem than initially thought.

I'm having some trouble following the logic of the objection. I dug it from upthread:


firstly, what we're talking about here is not, as DumbFruit generalizes, a "rollover fee". It's a disporportional penalty on mining large blocks. I'm not sure wether this changes his argument or its validity.

For thinking about this I'm using the following hypothetical mining landscape: 25%, 25%, 5 x 10%.

Now I think there are at least 2 interesting questions we can ask:

• Do the 2 25% miners (or 2 of the 10% miners) have a higher-than-in-current-system incentive to collude?
• Is Menis proposal making it easier for the 2 25% miners to try to drive out small (as in bandwidth) miners by mining disproportionally large blocks?

Both questions boil down to

• Does Menis proposal encourage centralization more than the current system?

Before I go on... am I asking the correct questions?

If the proposal finds enough support, I'm sure we can crowd-fund a PoC.

Unless I missed something huge, the proposal is not only to penalize large blocks, but to redistribute the penalties collected from these blocks back to other miners. In that sense there is a rollover of mining rewards (since penalized miners stand to earn their own penalties back), just not "fees rollover" per say.


I don't think they would collude, rather I expect a large miner can deplete the mempool of high fee transactions (and only these) while staying under the soft cap (thus paying no penalties), and leave other miners to pick up the slack. This behavior is incentivized by the fact that the other pools, if they do pick up the slack (and try to deplete the mempool with less regard for fees) stand to pay penalties for the extra service rendered, and the large miner in return stands to gain from that.

This results in one of 2 scenarii:

1) Other miners decide to pick the slack from the large miner, they naturally loose in profitability as the large miner is vampirizing their rewards by collecting penatlies. As a result, a lot of users from the other pools will point their hardware to the large miner, which will get an ever increasing share of the network hash rate, which enables his scheme even further and so on.

2) As a reaction to the large miner's behavior, every other miner adopts his policies, which is to at least avoid the penalties, by always emitting blocks below the softcap, and this change never fixed what it was meant to.


Other pools need to verify a block before they start to mine on top of it, so very large blocks, that propagate slower and take longer to validate could supposedly give a head start to the miner who emitted it, and be a disadvantage to smaller pools which stand at a risk of having their block orphaned the longer it takes for other miners to receive and validate their work.

I expect any centralized pool, however small they are, can afford the bandwidth and processing power to deal with much larger blocks than we have at the moment. This could hurt p2pool miners though.


Those examples do not stand. They hinge on the premise that there is endless demand for transactions paying 1mBTC fee. I understand the need to simplify these demonstrations but that defeats the underlying premise of this entire discussion. You example assumes there is no competition over fees, which is the premise of a "no block limit + hard fees" system. Your system sets both soft and hard caps to the block size, so there is no reason to believe people will sit at a 1mBTC fee when there is an endless demand for transactions

Model your demonstration with a fee structure using the Pareto principle i.e. 20% of the transactions pay for the 80% of the total fees available in the mempool (which is a lot closer to the current network's fee distribution than your examples), and the system falls apart. Anyone building blocks large enough to get penalized is just giving his rewards away to miners that prioritize big fee transactions and make a point of staying under the soft cap.

The issue with your proposal is not the penalty per say, it's the reward: there is a point where it is more profitable to let others get penalized. The existence of this point creates a threat that keeps all miners functioning below the soft cap. The threat is that they lose profitability in comparison to other pools, and those pools start siphoning away their hash rate as miners migrate.

If you were to take away the reward from the system a few things would be smoother:

1) No opportunities to game the system anymore. It all comes down to where the acceptable margin of fee vs penalty stands for the given mempool.
2) Very simple to implement.

The drawback is that since there is no reward, obviously the penalties are just destroyed. I'm not sure that's a drawback per say, for the following reasons:

1) It's trivial to blackhole bitcoins and it's been agreed that this is not damaging to the system. So this method isn't introducing some new DOS attacks on the system.
2) By destroying the penalty, the value of every other BTC just went up. As opposed to your system where you want to reward other miners from the penalties, this time everyone is getting rewarded, albeit in a much smaller magnitude. This means both miners, but everyone else holding coins is rewarded when a miner builds a block above the soft cap. Incidentally, that also means people running nodes (as long as they hold BTC, which is expectable).

This is perhaps the only proposition so far that has some sort of reward mechanism for node maintainers (granted it's tiny) which take equal part in the cost of node propagation and validation as miners.

I'm sorry, but I disagree. The scenarios don't "hinge" on this assumption, I just had to assume something in order to give concrete numbers. The effects I discussed should remain intact whatever the true transaction demand curve is.

The assumption also does not defeat the premise of the discussion, either. There is some demand curve for txs which depends on the real-world usage of the system, and we're talking about matching a supply curve to it. It's completely legitimate (though of course grossly exaggerated) to assume the demand curve is - infinite demand for txs with fee <1mBTC, and 0 for higher fees. With this hypothetical demand curve, fees will remain at 1mBTC because no one is willing to pay a higher fee, and no tx with lower fee will be accepted. In other words, the supply curve must intersect demand at its vertical drop line.

I'll try to repeat the calculations with a different demand curve, to demonstrate my point. But this will take some time and Shabbat is in soon, so that will have to wait.

As for the analysis, I also disagree. You can't "let others get penalized", every miner chooses his own penalty. The best you can do is penalize yourself, knowing that you'll reclaim the penalty later - but in so doing, you also increase the rewards for others.

Destroying coins in small amounts is not very harmful. But if done continuously as part of the system, it has negative macroeconomic implications.

Grammar Nazi regulations state that if you make the same error twice in a discussion, you must be corrected. We're already at 4. It's "per se".

At the moment I'm confident that the claimed centralization issue does not invalidate the method - whatever effect there might be, it's nullified with a correct parameter choice. I'm not as confident in my ability to convince others of this, but I can try.

Note, though, that superlinear mining rewards is a problem in general, I just don't see how this proposal contributes to it.

To be honest, I'm not sure. There are different elements at play here with complex interplay at different timescales. The usual question I'd ask is "do big miners have a superlinear advantage over small miners?". I claim above that the answer is no, but I'm now sure that answering this question alone is sufficient.

That was one of the purposes of this thread.

FWIW, I'll be happy to contribute to it.

Did you miss this link? -> https://github.com/monero-project/bitmonero/blob/c41d14b2aa3fc883d45299add1cbb8ebbe6c9ed8/src/cryptonote_core/blockchain.cpp#L2230-L2244

Let's assume the demand curve - the number of transactions demanded as a function of the fee, per 10 minutes - is d(p) = 27/(8000p^2). It's safe to have d(p)->infinity as p->0 because supply is bounded (if there was no bound on supply, we'd need a more realistic bound on demand to have meaningful results). The behavior below is the same for other reasonable demand curves, as long as demand diminishes superlinearly with p (sublinear decay is less reasonable economically, and results in very different dynamics).

We'll assume 4000 transactions go in a MB, and that T=1MB. So the penalty, as a function of the number n of transactions, is f(n) = max(n-4000,0)^2 / (4000*(8000-n)).

We'll also assume that transactions are in no particular rush - users will pay the minimal fee that gives them a good guarantee to have the tx accepted in reasonable time (where this time is long enough to include blocks from the different miner groups). So there is a specific fee p for which the tx demand clears with the average number of txs per block (the number of txs can change between blocks). It would have been more interesting to analyze what happens when probabilistic urgency premiums enter the scene, but that's not relevant to the issue of mining centralization.

Scenario 1: 100 1% miners.

Each miner reclaims 1% of the penalty. If the optimal strategy is to have n txs per block, resulting in a fee of p, then n=d(p) and the marginal penalty (derivative of f) at n, corrected for the reclaiming, must equal p (so that adding another transaction generates no net profit). In other words, 0.99f'(d(p)) = p. Solving this gives p = 0.7496 mBTC, n = 6007.

Penalty is 0.5053 BTC, so pool size is 50.53.
Miners get 4.5027 BTC per block (6007 * 0.0007496 from txs + 0.5053 collection - 0.5053 penalty).
6007 txs are included per block.

Scenario 2: One 90% miner and 10 1% miners.

The market clears with a tx fee of p, with the 90% miner including n0 txs per block and the 1% miners including n1 txs per block.
The average #txs/block must equal the demand, so 0.9n0 + 0.1n1 = d(p).
Every miner must have 0 marginal profit per additional transaction, correcting for reclaiming. So
0.1 f'(n0) = p
0.99 f'(n1) = p

Solving all of this results in:
n0 = 7251
n1 = 5943
p = 0.6885 mBTC (lower than in scenario 1)

Penalty paid by 1% miners: f(5943) = 0.4589 BTC
Penalty paid by 90% miner: f(7251) = 3.5294 BTC
Average penalty: 0.9*3.5294 + 0.1*0.4589 = 3.2223 BTC
Pool size: 322.23 BTC

Reward per block for 1% miner: 5943 * 0.0006885 + 3.2223 - 0.4589 = 6.8552 BTC (more than in scenario 1)
Reward per block for 90% miner: 7251 * 0.0006885 + 3.2223 - 3.5294 = 4.68521 BTC (less than 1% miners in this scenario; more than the miners in scenario 1).

Average number of txs per block: 0.9 * 7251 + 0.1 * 5943 = 7120, more than in scenario 1.

Miners are happy - big or small, they gain more rewards.
Users are happy - more of their transactions are included, at a lower fee.
Nodes are not happy - they have to deal with bigger blocks.
Exactly as with the previously discussed demand curve.

Over time, difficulty will go up, nullifying the extra mining reward; and whoever is in charge of placing the checks and balances, will make the function tighter (or hold on with making it looser), to keep the block sizes at the desired level.


There is another issue at play here - the ones who benefit the most from the big miner's supersized blocks, are the small miners. The big miner could threaten to stop creating supersized blocks if the small miners don't join and create supersized blocks themselves. Forming such a cartel is advantageous over not having supersized blocks at all - however, I think the big miner's bargaining position is weak, and small miners will prefer to call the bluff and mine small blocks, avoiding the penalty and enjoying the big miner's supersized blocks. This is classic tragedy of the commons, but in a sort of reverse way - usually, TotC is discussed in this context when the mining cartel wants to exclude txs, not include them.

I really like this idea! Keep up the great work Meni Rosenfeld  :)

I like it, too.

Thinking about the next steps I re-skimmed the OP (pretending to be someone just being introduced to the idea) and I think the introduction of (and reference to) the 'rollover fee pool' is very misleading. I know it is explained right after that it's really a 'rollover size penalty', but I fear it might lead people onto the wrong track and make it harder than necessary for them to grasp the idea. Maybe it'd be less confusing and easier to understand the concept if that part was removed?

I have a feeling this idea is a hard sell, mainly because it isn't what many might expect: it's neither...

• a way to dynamically set the block size cap nor
• a solution for scaling nor
• a rollover fee pool

It concerns itself with a different (although related) issue, namely the way the system behaves when approaching the transaction throughput limit.

I personally think this is a very important issue and my expectation of the current behavior and the ramifications thereof regarding user experience and media coverage is one of the reasons I'm for Gavins simple 20MB kicking-the-can-down-the-road proposal. With the rollover penalty in place I might be willing to wait longer and let some pressure build on developing scaling solutions. I'm not opposed to seeing how a fee market would develop, I'm also not opposed to seeing business opportunities for entities working on scaling solutions. I just don't want to hit a brick wall, as Meni so aptly put it... it would do much damage and can potentially set us back years, I fear.

So what are peoples ideas of how a roadmap could look like, what kind of funds we might need and how we could organize enough (monetary and political) support?

So far I like this proposal very much, too.


The elastic cap, penalty fee pool and the hard limit could be addressed seperately, although it probably makes not much sense to introduce this mechanism with the current block size limit.

In this context I'd like to add some visibility to TierNolan's post on the second page:


It seems viable to set a high hard limit, and start with a lower-than-max elastic cap, which could be increased further at some point in the future.

this

actually, just use twice the average blocksize of the last two weeks.

 And you don't really need any of this complicated system.

1. Floating block limits have their own set of problems, and may result in a scenario where there is no effective limit at all.

2. Even if you do place a floating block limit, it doesn't really relieve of the need for an elastic system. Whatever the block limit is, tx demand can approach it and then we have a crash landing scenario. We need a system that gracefully degrades when approaching the limit, whatever it is.

1) i beg to differ, so long as the timespan is sufficient that only a LONG lasting spam attack or other growth could cause massive block caps. Personally, i think as long as it averages over at least 1-2 weeks, thats sufficient to prevent any sort of rampant spamming.
2) if the cap is set at double the recent volumes, it should provide enough room for fuller blocks so long as we dont see 5x network growth within less than a 1-2 month timespan. even then, the cap would grow over time and lower prioriy transactions may just be pushed back a few blocks. fees take priority until everything balances out after a few days/weeks

(I suggest 2.5x the average of 40days, or 6000 blocks) OR ((2x the last 6000 blocks) + (0.5x the last 400 blocks)).  The second allows for slightly faster growth if there's sudden demand for room.

I think a better solution would be to require miners to do more work for larger block sizes. Instead of hashing just the header of a block, miners have to hash something more: perhaps something proportional to the block size. So if a header is 80 bytes, it takes up 80/1000000=8e-05 of the whole block. So for any block size x > 1 MB, require a miner to hash the first (8e-05)x of the block in order for it to be valid. This will make Bitcoin automatically scale to the power of computers in the future, as big blocks will only be plentiful if computers (ASICs) are fast enough that it is worth taking the extra transaction fees with bigger blocks. Any problems with this?

I'm not sure, if "how long can spam last" covers the whole picture. I'd like to ask "how fast can miners deploy new hardware/adjust to an increased cap?" in addition.

mining hardware has little to do with the issue, besides the fact that if large blocks are slow to download it could allow large miners to [unintentionally] start creating small forms while the slower miners create orphans. The mining process is virtually unchanged

Sorry, poor choice of words. I wasn't thinking about mining hardware, but deploying additional bandwidth/adjusting hosting plans/[...] and alike, to handle larger blocks.

That's the basic idea behind Greg's proposal. I've yet to examine it in detail; I think it was actually what I thought about first, before eschewing it in favor of a deductive penalty.

I think there are errors in your description of how to implement this. It's not about what you hash, it's about what your target hash is. Also, you need to carefully choose the function that maps block size to mining effort.

Can you link Gregs proposal? I haven't seen it.

Currently I have this link - http://sourceforge.net/p/bitcoin/mailman/message/34100485/. It's not the original proposal but some followup discussion. I still need to dig and trace it back to the source.

I'm getting slammed at work so I won't be able to come up with a properly crafted response to your new model this week.

I don't see how this differs from in Monero?

In Monero, addition of txs up to the median blocksize is free. As you surpass the median blocksize, a quadratic penalty is applied to the subsidy of the coinbase, but amounts obtained from tx fees are untouched. The subsidy of the coinbase is initially dependent of the number of coins in existence, and so takes into account the previous penalties to coinbases of any previously generated blocks (comparable to your "pool" method). Then, the miner is free to add transactions meeting some economic equilibrium that maximizes their overall income when taking into account the blocksize penalty to the coinbase subsidy.

So, it's like this:
(1 - penalty) * (minted coins) + tx fees
where penalty is dependent on the size of the block above the median size according to the formulas found in the CN whitepaper.

gmaxwell criticizes this as promoting out-of-band transactions, but the fact remains that to permanently and secure transfer money you must use the blockchain and have it included in a block somewhere. Thus, I never thought it was much of an issue.

That's not the part that differs from Monero.

In fact, that's the part that NewLiberty claimed was worse than Monero, and I claimed is the same. (See also context (https://bitcointalk.org/index.php?topic=1078521.msg11535235#msg11535235))

Do you have a reference for Greg's criticism? It doesn't make sense to me. I've explained above why out-of-band is not a problem in my suggestion (as you said - inclusion in the blockchain is what you need to secure the tx, so you have to pay the penalty anyway, and it doesn't matter if you get payment in or out of band), and since Monero is similar in this regard, it shouldn't be a problem for Monero either.

It's possible he means that miners will make an out-of-band commitment to include a tx at a later, less crowded block. But... That's a problem with any other system (e.g. the current hard cap in Bitcoin), and it's not a problem - we want txs to be taken out of big blocks and into smaller ones. I don't think the users will actually agree to such a contract, but if they do, it's perfectly fine.

One difference with the Monero model is that the amount of the fee is not a factor of the penalty, nor is size of the coinbase reward.  The penalty is not multiplied by the block reward.  Additive, not multiplicative.

I'd missed that detail at first pass because it seemed impossible for anyone to advocate such a function, when on its face it seems to fail the 'future-proof' test.  

Without knowing what the fees will be in the future, how can the penalty be the same for every transaction?  Fee amount tends to adjust with how much milk and bread a bitcoin can purchase.  The penalty may thus become overly burdensome (if bitcoin value increases) or meaningless (if it falls).
We can know the coinbase reward, but we should not always expect that to be 99% of the total as it is now.  This will matter later.

The advantage of the Monero multiplicative method is that it does not have to guess, it works at all fee levels, block rewards, valuation, etc.

As I've explained multiple times, this is exactly the reason for choosing a hyperbolic function. The marginal penalty is the derivative of f, f'. f' ranges from 0 to infinity when the block size ranges from T to 2T. Whatever the current Bitcoin price etc., there is some x for which f'(x) is equal to the typical fee. So the block sizes stretch and expand to accommodate the changing fees.

The same happens with the quadratic function in Monero, but much more slowly, giving us less control over the block sizes. Because of this, the quadratic function actually assumes more about the fees than the hyperbolic one.

The penalty itself (as opposed to its derivative) is less important in my proposal, because it more or less cancels out with the collection from penalties of past blocks. But still, we prefer to keep the penalty as low as possible, meaning that we are interested in the ratio f'/f. A hyperbolic function, because it grows faster, achieves a better ratio - and we can improve it further with parameter choice.

Furthermore, if the reward is (1 - penalty) * (minted coins) + tx fees, then in the future when the minted coins go down to 0, there is no penalty at all. That's the opposite of future-proof.

Ok, so my post was nuked again (My computers fault) so this will be short and sweet. I have been away, so I apologize for the delay;

Suppose there is a big miner with 33% of the hashpower, and there are 67 small miners with 1% of the hashpower. Because of the Law of Big Numbers and simplicity, I'll assume that they all mine blocks in a predictable order;
BigMiner, SmallMiner[1], SmallMiner[2], BigMiner, SmallMiner[3], SmallMiner[4], .. , BigMiner, SmallMiner[66], SmallMiner[67]

I'll also assume that the rollover fee will be distributed over the next 100 blocks.

BigMiner can assume that it can recover 33% of the lost reward on a future block. So if the BigMiner can mine a block that exceeds T while simultaneously getting a reward that is better than a block less than T, given that his reward is reward - penalty + .33*penalty, then it's at a net positive. Whereas a small miner could only do reward - penalty + 0.01*penalty. What is further problematic for the SmallMiner is that if they mine anything with a rollover fee, the BigMiner will get 33% of the reward whereas they will be lucky to get 1% (Not only is the reward less for himself, but any rolling over reward is more beneficial to his larger competitor(s)). In this way, the BigMiner has a competitive advantage over SmallMiners.

I tried to come up with specific numbers that would show the attack on your algorithm, but I wasn't sure what values you were using to get your graph¹, so I think it would end up being a straw man anyway. The bottom line is that penalties or rewards rolling over hurts the SmallMiner more than the BigMiner. SmallMiners being less able to recover over future blocks.

By "penalties or rewards" I mean that it doesn't make a difference whether you are rewarding the current larger block and rolling a bonus over to future blocks, or if you are penalizing a big block and then passing the difference to future blocks.

I didn't see a response to that problem, sorry if I missed it.

¹https://i.imgur.com/EZvlJq7.png

Edit: T was defined as the target block size.

Monero is permanently inflationary, so there will always be an impact of the penalty.

See a worked example at https://bitcointalk.org/index.php?topic=1078521.msg11557115#msg11557115. I use the penalty function described in the OP - f(x) = max(x-T,0)^2 / (T*(2T-x)). And an earlier one (https://bitcointalk.org/index.php?topic=1078521.msg11536606#msg11536606) with made up numbers but additional analysis.

You need to be thinking in terms of reward per block. Of course the 33% miner should get about x33 the rewards of a 1% miner, simply because he's bigger. What we don't want is that the big miner would have more than x33 the rewards (superlinear gain). What I've shown is that the big miner actually gets less than x33 the total rewards of a small miner, or in other words, that the reward per block is smaller for the big miner.

Every miner gets, per block, (tx fees - penalty) + collection. The rollover pool is the same for all miners so collection is a roughly constant value. But (tx fees - penalty) differs per block, and is lower for the big miner. The big miner purposefully builds supersized blocks, because increasing the pool works to his advantage - but it works to the advantage of the small miners, too.

Put another way, the small miners don't reclaim much of their own penalty, but they reclaim the penalty of the big miners.

If you still disagree, please either
1. Explain why we shouldn't be looking at reward per block, or
2. Explain what is wrong with the example (https://bitcointalk.org/index.php?topic=1078521.msg11557115#msg11557115) (the one with real numbers) that shows the reward per block is smaller for big miners.

Ok. I was primarily refuting the implied claim that this formula would make sense when applied to Bitcoin.

This is simultaneously the best feature of the proposal, and the one most likely to prevent it from becoming part of Bitcoin.

There was no such claim, implied or otherwise.
The coins are fundamentally different, but the similarity of this feature would have provided a good place to have started, if you had known of it.
Better late than never though.

Right here;

No miner is going to mine a block that costs him more than 2 BTC to mine. Since it's not economically viable to mine larger blocks, you're right that there isn't an economic advantage given to the larger mining entity, and what I wrote earlier wouldn't apply. What I wrote would only apply to penalties that don't reduce the reward below the target block size reward.

Why would a miner mine a block at a loss just to accept more transactions? Regardless, any market participant that engaged in this behavior would just get out-competed by another that didn't.

Why not have both an elastic block cap and floating block limits? A common argument against floating block limits is "big miners will create super-sized blocks full of crap to artifically inflate the block limit". This attack is conveniently addressed by your rollover penalties, as the penalty is a function of block size (not of transaction fees), so miners cannot game the system by including self-mined transactions.

Consider the following floating block limit function:
evaluated every N blocks.
With k = 1.0 and N = something large like 8064, we have an equilibrium situation consisting of the status quo, where everyone stays under the soft cap of T. If a large mining cartel wishes to inflate block sizes against the will of smaller miners, they must begin creating larger blocks and paying penalties towards the smaller miners, for each block. With sufficiently large N, this is not sustainable.

Let's say Bitcoin experiences sustained, long-term growth and the fee pool/demand increases. Now everyone, including smaller miners, is creating blocks larger than T. Everyone pays penalties, but in doing so, the "penalty" isn't really a penalty; everyone receives about as much from the rollover pool as they pay. After 8064 blocks, T increases to account for this genuine, long-term increase in demand.

There are lots of parameters here, and they can be adjusted to disincentivize a mining cartel from artificially inflating block sizes. For example, increasing N and making the limit function f(x) "harder" will both increase the cost to artificially inflate block size limits. Another possibility is setting k < 1.0 (e.g. k = 0.98), which means that maintaining the status quo has a cost. If miners unanimously decide to maintain the status quo, then no-one is actually penalised because everyone receives as much from the penalty pool as they pay. However, if smaller miners feel that the status quo is unreasonable (because of past bloat from a large mining cartel), they can choose to make smaller blocks and "bleed" penalties from the larger miners. However, I am concerned that setting k < 1.0 might implicitly set an absolute minimum transaction fee.

tl;dr version:
With floating block limits + rollover penalties:
mining cartel tries to artificially inflate blocks => they must subsidize smaller miners with penalties
Bitcoin experiences genuine, long-term growth => miners unanimously include more transactions => block sizes will increase

I don't understand what you wrote here. These are the results for the optimal strategy for each miner, which maximizes his profit. Should I show the derivation? It's just a bunch of unwieldy equations that my silicon overlord (https://en.wikipedia.org/wiki/Mathematica) took care of for me...

Each miner wants to maximize tx fees + collection - penalty. The small miner can't hope to affect collection because it depends on what the other miners do. But the big miner does meaningfully affect collection as he makes bigger blocks. As he adds transactions to his blocks, all of tx fees, collection and penalty increase. At some point, the increase in penalty outweighs the increase in tx fees + collection, so he stops there. Whether one or more of these summands is more than 2 BTC is irrelevant, it's their sum that counts.

In the part you quoted, no miner mines at a loss. The small miner has a profit of 6.8552 BTC per block, and the big miner has a profit of 4.68521 BTC per block. If the big miner built smaller blocks, his rewards per block would be smaller. If, for example, he included only 6000 txs/block, like the small miners, he would get only 4.50 BTC per block (instead of 4.68), just like the miners in scenario 1.

If the big miner takes a longer-term view, he may notice that creating supersized blocks will give him a profit at first, but then attract more miners, who will increase the difficulty, which will reduce his profits, to the point where he is not competitive with the small miners - so he can give up the whole thing and just build normal blocks (ignoring reclaiming his own penalty in his optimality calculations). And then we go back to the situation where big and small miners are completely equivalent, and get the same reward per block.

Sounds good. I tend to agree there is good synergy between the two modifications - the limit will float only after it is stretched and penalized, making sure there is an actual need for it. I agree the time scales need to be fairly large, to make it expensive to artificially inflate the limit. Anyway, this requires more research.

I don't understand how your algorithm will give the entity with higher hashpower more profit by mining blocks that give less BTC per block, that doesn't make any sense to me. Why wouldn't the bigger miner mine smaller blocks and get the higher BTC per block, like your example was showing?

On the other hand, if it really is more profitable to mine larger blocks, then why wouldn't the smaller hashpower nodes do it? If they do, then the same problems I mentioned before would apply (The penalty would give a competitive advantage to the higher hashpower nodes).


if this is true;


Why isn't the 90% miner simply including 5943 transactions like the 1% miner is, in order to get about 2 BTC more per block?


The problem is, regardless of what the collection is in the pool, the highest profit per block is always greater by simply mining at the target block size.

If the major nodes have driven up the BTC in the collection pool to provide greater profits on the whole, then the first big miner that stops contributing to the pool will essentially drain the pool by mining blocks at the target block size. In the end, trying to maximize profits by relying your competitors to "play fair" won't work.

This is actually analogous to cartelization; Big companies cooperating to achieve greater profits. However.. Cartels, absent the power of the State, always break down, because the first one that does can achieve greater profits at the expense of the cartel as a whole.

http://wiki.mises.org/wiki/Cartel

Start a block chain and try it out.

Because in so doing, the miner increases the penalty pool, and will reclaim it in his future blocks.

Because if he does, the penalty pool will be smaller, and he will get less BTC per block. If nobody creates supersized blocks, we go back to scenario 1.  The big miner, like all other miners, will get only about 4.50 BTC per block. When the big miner creates supersized blocks, he gets 4.68 BTC per block.

Because if a small miner creates a large block, the effect on the penalty pool is not as meaningful as the effect the big miner has.

All miners want to maximize their (tx fees - penalty) + collection. Collection is equal to the average penalty paid by all miners. A small miner plays only a small part on this average, so collection is constant as far as he is concerned, so he more or less wants to maximize just (tx fees - penalty), which happens at around 6000 txs. If he goes further, he will decrease his (tx fees - penalty) with barely any effect on collection.

A big miner has a big effect on the average. As he creates blocks larger than 6000, (tx fees - penalty) decreases but collection increases at a faster rate. Above 7250 txs, the gain in collection doesn't justify the drop in (tx fees - penalty).

Because if he starts creating 5943-tx blocks, the penalty pool will be smaller, and so will the amount he collects per block. See scenario 1 here (https://bitcointalk.org/index.php?topic=1078521.msg11557115#msg11557115) - if the big miner behaves as the small miners do, it's the same as if they're all small miners. He will get just 4.50 BTC per block, instead of 4.68. He can't have the cake and eat it too - not pay a penalty, but expect to reclaim it...


I'll say again - the default state is where everyone just maximizes their own (tx fees - penalty). A big miner can afford to build bigger blocks than that because he knows he can reclaim 90% of the penalty, so it doesn't bother him that much. In so doing, he increases his profit per block - but he increases the profit of the miners who don't do this even more.

I'm not making any assumptions about fair play, reciprocity, collaboration or any such thing. All miners are behaving selfishly and without regard to the other miners. However, I am assuming each miner expects to continue mining, at a similar hashrate portion to his current (or, alternatively, he bases his calculations on his future anticipated portion). The big miner will create supersized blocks simply because he know he'll be around to reclaim some of the penalty later. So he doesn't maximize (tx fees - penalty), he maximizes (tx fees - 0.1*penalty), which happens at larger blocks, with a steeper marginal penalty.

If there are several big miners, they should all create supersized blocks, and then every miner will enjoy the supersized blocks of the others. But the reason for each individual miner to create supersized blocks is not that he hopes others will do the same. It's simply because it's more profitable for him to do so, regardless of what the others do. This is not the prisoner's dilemma, where there is a conflict between selfish desires and the greater good.

Of course, it is possible to miners to try and band up in a cartel, creating blocks even larger than what is selfishly optimal for each of them individually. But such a structure will not be stable.

That's not the point. What prevents the competition from pulling from the pool without contributing to it? I understand that it's in the interest of a massive node or cartel of nodes to mine large blocks, but only if they aren't undermined by their competitors, which can pull from the pool without paying any penalty themselves.

The nodes can't have their cake and eat it too... But they can eat it. Ultimately, there would be no pool.

Yes, they should, and this would be a cartel; where each is cooperating with each-other for greater profits, and it would break up in the same fashion as other cartels; Any member of the cartel profits more by not contributing to the penalty pool.

I apologize for making this even more academic, however this is a complex proposal and there will always be games and ways of analyzing this which are analogous to the prisoner's dilemma.  For example a large miner (A) could consider that producing a smaller block leaves more transactions in the meme pool, such that another miner (B) would be incentivised to produce a larger block and pay a larger penalty and this would increase the rollover pool, which large miner A could collect.

Miners want to maximize their (tx fees - X * penalty) + collection +-  X * impact the decision has on other miners block sizes and the associated penalty's

X = projected share of the network hashing power


Nothing prevents miners leaving pools.  Is that not desirable?  This could potentially be a great characteristic of Meni's proposal.  Undermining a large cartel of miners in favor of smaller miners seems great.  (I don't know if Meni thought of this when he made the proposal, or it was kind of luck?)


I think this potentially sounds like a good idea.

That's exactly what the small miners are doing. In my scenario 2, the small miners do not contribute and are not expected to. They just maximize their own (tx fees - penalty) at around 6000 txs, and enjoy the collection from the penalties the big miner is paying. They eat the cake and let the big miner bake it. The big miner creates supersized blocks knowing full well that the small miners will not follow in their lead. It's in the interest of a massive node to mine large blocks regardless of what the competition are doing. The optimization criterion is different for small and big miners. The 90% miner knows that he'll be able to reclaim some of his own penalty (not the penalty of the other miners), so he effectively gets (tx fees - 0.1*penalty) per block, which is optimized at a higher size. If there are several big miners, each of them creates supersized blocks because it is in their selfish interest. If miner 2 pulls out, miner 1 will continue creating supersized blocks. But in fact, miner 2 will not pull out, because it would just reduce his own profits.

As I mentioned, the above is as long as we ignore difficulty changes. In practice, what will happen is that the supersized blocks will attract more miners, which will increase the difficulty and reduce the profits. A big miner which takes this into account will probably want to avoid it, and will not create supersized blocks. If everyone (including the big miners) just maximize (tx fees - penalty) without regard to reclaiming, then all miners get the same reward per block. This, again, proves my original point, that the method does not create an additional incentive to centralize.


Also, numbers speak louder than words. In scenario 2 here (https://bitcointalk.org/index.php?topic=1078521.msg11557115#msg11557115), I've claimed that the optimal strategy for the small miners is 5943 txs/block, and the optimal strategy for the big miner is 7251 txs/perblock. This is a Nash equilibrium - each player has no better option, assuming everyone else sticks to their strategies. Are you saying this is not so? Can you suggest a different strategy for the big miner, which will result in getting more than 4.68521 BTC / block (without the small miners changing theirs)? Can you suggest a different strategy for a small miner (just one, not all of them as a collective) which will result in getting more than 6.8552 BTC / block? Again, we're talking about time scales long enough to reclaim penalties, but short enough for difficulty targeting and new miners to not be a factor. Or perhaps, you'd like to investigate a scenario with a different distribution of miners?

No. They are not cooperating. Each miner is simply doing what is best for him, expecting others to be similarly selfish. This is shown in my calculations, and if you'd like, I can repeat the calculations for a different distribution of miners.

Again - a big miner (regardless of whether the other miners are big, small, whatever) creates supersized blocks (blocks which are bigger than what optimizes tx fees - penalty) simply because he expects to reclaim some of his own penalty. Not the penalty of others.

That's indeed a possibility. I've run the calculation for a big miner that tries to do this (in the 90%/1% scenario), and found that it's optimal for him to put 7004 txs/block, which then causes the small miners to put 5955 txs/block. This bumps the big miner's reward to 4.719 BTC/block (from 4.685). So the effect exists, but is fairly small; also, this strategy is not a Nash equilibrium, so it will probably not be very stable.

Just to make sure everything is clear, the penalty pool is not related to mining pools, despite the usage of the word "pool". I have not talked about mining pools in this discussion.

Scenario 3: There are two 50% miners, Miner 1 and Miner 2.

Each Miner has a choice between creating normal blocks, which maximize tx fees - penalty, and creating supersized blocks, which maximize tx fees - 0.5 * penalty. These strategies will be called N and S respectively.

If both choose N, then the optimal block size, for both, is 6000 txs. The penalty per block is 0.5 BTC, and the fee is 0.00075 BTC. Each miner will get per block:
6000 * 0.00075 - 0.5 + 0.5 = 4.5 BTC.

If Miner 1 chooses S and miner 2 chooses N, then for miner 1 the optimal block size is 6476, and for miner 2 it is 5986. The fee is 0.000735975. Penalty for miner 1 is 1.00567 BTC, and for miner 2 it is 0.489597. The average penalty is 0.747634
Miner 1 gets 6476 * 0.000735975 - 1.00567 + 0.747634 = 4.50814 BTC per block.
Miner 2 gets 5986 * 0.000735975 - 0.489597 + 0.747634 = 4.66358 BTC per block.

Likewise, if Miner 1 choose N and Miner 2 chooses S, then
Miner 1 gets 4.66358 BTC per block.
Miner 2 gets 4.50814 BTC per block.

If both miners choose S, then the optimal block size is 6464. The penalty per block is 0.988167, and the fee is 0.00072258. Each miner will get per block:
6464 * 0.00072258 - 0.988167 + 0.988167 = 4.67076 BTC per block.

Putting all this in a table:

	2	N	S
1
N		(4.500, 4.500)	(4.664, 4.508)
S		(4.508, 4.664)	(4.671, 4.671)

This is not the prisoner's dilemma table. The Nash equilibrium is the best possibility for both. When a player chooses S, he helps the collective but also helps himself selfishly. There is nothing anyone can gain by "defecting". Every miner chooses S because it benefits him, not because he has some sort of pact where the other miner does the same.

In other words, it's not like choosing S gives -1 BTC to you, 3 BTC to the other. Instead, choosing S gives +0.008 BTC to you, +0.664 BTC to the other. You're helping the other more than you're helping yourself, but you're still helping yourself.

The defector does gain by defecting; They gain market share. This isn't a prisoner's dilemma, it's just the best way for a mining entity to out-compete the other. It doesn't make a lick of difference that they can make more profit (Split between the two) by cooperating over a long period of time by not leveraging a competitive advantage over that same period.

Market share and out-competing are means, not an end. The goal of a miner is to maximize his profits. A big miner does this by creating large blocks. Why would he care that the other miner also gets more profit? Is he jealous or something? Please clarify what it is exactly that you're trying to say...

Sure, but the defector gets more profits by capturing market share. The defector gets more profit at 90% market share even if total profit split between the two is higher by cooperating at 50%.

It really is very similar to a cartel, except that the mechanism for higher profits is different. The way the cartel falls apart is the same; The defector gains profit for itself by capturing a larger market share.

Sorry my mistake.  I was thinking miners could leave mining pools in this scenario, to benefit from the remaining members of the pool producing larger blocks.

I'm sorry, and I hoped it wouldn't have to come to this, but I give up. This discussion is not going anywhere. You keep making the same statements and I keep making the same refutations. The difference is that I've backed my claims with calculations. You didn't point out any problem in my calculation (the assumptions or one of the steps) and didn't provide a calculation of your own. You just object to the final conclusion without any regard to the derivation that led to it.

It seems you're trying to force classical game theory cases into fitting the situation we have here. You say you agree that "this isn't a prisoner's dilemma" but all of your statements are taken from the world of prisoner's dilemma. But they just don't fit, the game here is different, and you refuse to address or listen to the logic that demonstrates this.

You also make generic statements without clarifying under which situations they apply, whereas I've tried to map out how the situation will play out in varying circumstances.

To conclude this line of discussion, I'll restate my thoughts on the matter:

1. If we assume that the hashrate distribution and difficulty are static, big miners will create supersized blocks because they know they'll reclaim some of the fee themselves. This is a Nash equilibrium and does not rely on any sort of collaboration or reciprocity. This will shift the balance we tried to achieve with the cap in the first place, allowing more txs into blocks than what would happen without this effect, and giving slightly more profit to big miners and much more profit to small miners.

2. In practice, the assumption above is not realistic; miners know that if they create supersized blocks, they will make mining more attractive, thus attracting more miners, thus increasing the difficulty, thus reducing their profits. So longer-term it is actually not beneficial for them to do this. So in all likelihood, big miners will not create supersized blocks, and all miners will get the same reward per block, without any superlinear advantage. Because of this, the original objection that the method in the OP encourages centralization of mining, has been refuted.

I already said my original argument wasn't true because it was only true in cases where larger blocks provide a higher reward per block to the miner. Your algorithm doesn't do this.

That's what I was saying here;

I agree that big miners will not create supersized blocks, but it isn't because it attracts more miners, it's because mining supersized blocks confers a competitive advantage to the other miners that don't. The net effect is I can't imagine a situation in which any miner would create any supersized blocks because doing so would immediately damage their position in the market.

Not sure about this, but maybe two things Meni says could be contested:

• "The goal of a miner is to maximize his profits": If miners were individual humans I would say this is false (let's hope it's not and they act rationally maximizing their profits). There are studies showing that humans prefer a small gain to a large one if it is comparatively larger than some implied gain of some other person. Say a choice between receiving 1 Bitcoin or receiving 2 Bitcoins is offered, but another person receives 0 Bitcoins in the first case and 10 in the second case. Then the individual will tend to choose to receive only 1 Bitcoin, just to receive "more than the other guy". That's not "maximizing profit", that's maximizing some kind of "relative perceived profit" and yes, it boils down to jealousy (or rather 'envy'). Not sure this applies here, but I think it's possible. In case such psychological effects do apply, there's more human peculiarities that might be of interest, like loss aversion (https://en.wikipedia.org/wiki/Loss_aversion), where avoiding losses is preferred to acquiring gains of the same amount.
• "The miner has higher profit choosing S over N": I think this might be dumbfruits point: there might be some other, indirect effects stemming from the increased profit given to the competition influencing the miners profit in the long run. Let's take for a example a rather efficient miner. Let's say he can cover his cost in both cases S and N and let's say his counterpart mines with higher energy cost. Choosing S might push his competition into the profit zone, while choosing N might force the competition to shut down operations. In such a scenario, choosing N would be the more profitable option. Or maybe less crass: by reducing the competitions profit overproportionally to it's own, a miner can maybe slow down their expansion or increase of efficiency and maybe deven drive them out of the game entirely.

;tldr: I'm not so sure if it isn't a prisoners dilemma after all. Sure, Menis math checks out, but maybe that model (looking only at direct profit) is inadequate for predicting miner behaviour because it possibly omits important side effects or (less likely imo) makes false assumptions about miner motivation / decision making premises.

Yes this could be another risk and your argument is probably correct, however the impact of this might be small and it actually has characteristics similar to normal mining economics.

Overall, at this point, in my view Meni’s proposal could be the best suggestion so far, in respect of partially alleviating some of the concerns associated with the block size limit issue.

The potential options we have are outlined below:

1. Increase the max blocksize to 20MB
Advocate: Gavin

2. Do Nothing
Advocate: ?

3. Introduce a variable difficulty based on blocksize
Advocate: Gregg

4. Limit the CoVar of the transaction fees in blocks
Advocate: Sergio

5. Elastic block cap with rollover penalties
Advocate: Meni

6. Dynmaic block size limit based on averages of historic blocks sizes or historic fees
Advocate: ?
 
7. A less direct solution like Lightning, Sidechains or off chain transactions
Advocate: ?

8. Allow users to choose their own limits, with the 1MB chain confirming transactions in larger chains
Advocate: Adam

9. A compromise between 1MB and 20MB
Advocate: ?

http://b85.imgup.net/Untitled0e4e.png

Unlike most of the other proposals, Meni's suggestion appears to have less of an impact on the network if it is not used and has far fewer security risks than the other proposals, in my view.  At the same time it can be combined with a variable limit based on historic medians, to produce a more complete solution to the problem.  I hope people scrutinize this proposal, and the others, in an objective way and that we eventually implement a solution in a thoughtful and patient manner, based on the best technical solution, with the smallest amount of valid attack vectors.

But you didn't explain why "competitive advantage" and "position in the market" matters. And I can't myself think of any reasonable explanation, other than something equivalent to what I said about increasing difficulty. (See also response to molecular below).


Of course, I am assuming straightforward profit maximization. Assuming otherwise would drastically alter the problem. But, in the scales involved, I think this assumption will be close enough to reality.

This is a special case of what I said about "attracting more miners". It doesn't matter if the miners he is attracting are new miners, or old miners who are now able to continue operating or grow their operation. Note that "competition" in mining only matters through the difficulty mechanism.

Well, it is a prisoner's dilemma, but in scenarios different from the ones I was discussing. Most of what I wrote here on the centralization issue refers to the "static hashrate distribution and difficulty" case, which is not very realistic but is the baseline scenario; without agreeing on it, there's little hope to agree on anything else. Which is why the discussion with DumbFruit was frustrating - he kept disagreeing, but he never clarified if he thought the results for this case are incorrect (with which I strongly disagree) or just ultimately irrelevant (with which I agree, and have agreed from the very start - see next-to-last paragraph here (https://bitcointalk.org/index.php?topic=1078521.msg11536606#msg11536606))

There are two cases where this becomes a prisoner's dilemma:

1. Miners aren't satisfied with just supersized blocks, they want to make hyper blocks - blocks even bigger than what optimizes (tx fees - penalty * (1-hashrate)). This is not a Nash equilibrium and only profitable if others follow their lead. I've mentioned this scenario to demarcate the discussion (See last paragraph here (https://bitcointalk.org/index.php?topic=1078521.msg11584523#msg11584523)) but it's simply not what I was talking about. I was talking about selfish supersized blocks.

2. Miners consider the long-term effects on difficulty and hashrate distribution. Then the Nash equilibrium is probably creating normal blocks. If they want to band up in cartels and create supersized blocks even in this scenario, sure, they can go ahead, but... I'm not sure it's profitable even with a strong cartel (since it's the small miners who benefit the most), and again, I was never trying to argue miners should create supersized blocks in this scenario.

In other words, in all cases I am advocating going with the Nash equilibrium. I was never even considering strategies that would make this into a prisoner's dilemma.

The table is nice (didn't really check the numbers though), but I think it fails to consider the ways in which different ideas can be combined. Some ideas work very well with others.

I'll comment on my own position:

• I haven't yet examined Greg's proposal in detail, but originally my main concern with it revolved around implementation difficulty, and the magnitude of change in the spirit of Bitcoin. If those are out of the way... Then it might turn out that mathematically and economically, it's actually equivalent or even superior. We do need to carefully consider how this proposal changes the difficulty.
• An elastic cap should be combined, IMO, with an increase in the block limit. T=3MB would work well for now. Without the soft cap, the block limit should go up to something like 4MB.
• I still find it hard to believe that Side chains are actually possible (recent news about first sidechain being rolled out notwithstanding). But assuming they are, they're a game changer. They would pretty much take the edge off the entire debate since everyone can just choose a chain to their liking. Assuming someone implements it, of course, but that should be easier when people can spend less of their time arguing and more developing.
• Micropayment-channel based payment networks are a critical component of long-term scalability, but they're not a replacement, and still need more time to be viable.

I decided to analyse the situation in the case of mining pools. As a start, I used Meni's parameters as described in https://bitcointalk.org/index.php?topic=1078521.msg11557115#msg11557115 (https://bitcointalk.org/index.php?topic=1078521.msg11557115#msg11557115).

Assume I have 1% of the total hash rate. I am currently in a 90% mining pool. There are also 10 1% miners. Using the results in Meni's post, my expected income per block is 0.04685 BTC.

Will I earn more by leaving the 90% mining pool and solo-mining?

If I solo-mine, the 90% mining pool is now an 89% mining pool and there are 11 1% miners. Solving for these parameters:
n0 = 7217 (slightly smaller)
n1 = 5945 (almost the same)
p = 0.6906 mBTC (slightly larger)
Penalty paid by 1% miners: f(5945) = 0.4602 BTC
Penalty paid by 89% miner: f(7217) = 3.3043 BTC
Average penalty: 0.89*3.3043 + 0.11*0.4602 = 2.9914 BTC
Reward per block for 1% miner: 5945 * 0.0006906 + 2.9914 - 0.4602 = 6.6368 BTC
Reward per block for 89% miner: 7217 * 0.0006906 + 2.9914 - 3.3043 = 4.6712 BTC

My expected income per block is now 0.06637 BTC, 42% higher than when I was in a mining pool. There is a very strong incentive to betray the large mining pool. I have not done any additional calculations, but I suspect it is also profitable for every individual miner in the larger mining pool to leave and solo-mine, or at least join a smaller pool.

Thus a sensible large mining pool operator should not mine supersized blocks. Meni, I realise you've come to this conclusion another way (higher income means higher difficulty), but this is yet another reason why rollover penalties discourage large mining pools from mining large blocks.

My expected income per block that I find is 6.637 BTC. It's true that my expected income per block found by the network at large is 0.06637, but I find it confusing to refer to it as "expected income per block".

Right. Mining pools, in this view, are essentially a way for small miners to band in a cartel to create blocks which are individually hyper-sized (but supersized for the pool as a whole), hoping for a greater profit. Hyper-sized blocks are not a Nash equilibrium (every miner hopes that the others band up, and enjoy their supersized blocks without contributing himself), and thus pools that try to form around this objective will quickly disintegrate. A big miner needs to be monolithic to have any hope of creating supersized blocks and not disintegrate.

Is this considered a security improvement?

No, it just strengthens the point that the proposal does not create new reasons to centralize. It does nothing about old reasons though.

If I understand correctly, in this proposal the miners get the ultimate say about what the block limit will be. That is ill-advised, as their incentives are not aligned with those of the users and the nodes. A voting mechanism makes it very easy for miners to collude.

I agree that it will be good to set a precedent that the Bitcoin technology can be changed, and that therefore we should start with a small change. But is seems Jeff's proposal is anything but, rather it's a very controversial change with significant economic implications.

Take that part out, and I support this. I'd prefer, though, either going straight for 4MB, or an accelerated timetable for 2MB.

Can we use this curve to set the minimum mining fees?  The "penalty" needs a  constant multiplier because the units cancel each other out and we don't get bitcoins units at the end of our calculation.



Suppose the constant multiplier is 1 mBTC and we use a free limit of 400kB and a hard limit at 1 MB.  Then when blocks get to 95% filled the transactions simply get rejected unless they are paying 15.125 mBTC per transaction.  Getting close to 750kB means we have to pay around 1 mBTC.

Small Blocks
Take the soft limit to be say 400 kB and the hard limit to be 1 MB with a fee multiplier of 1 mBTC.

size (MB)	Cost in mBTC
0.1	0
0.2	0
0.3	0
0.4	0
0.5	0.05
0.6	0.25
0.7	0.75
0.8	2
0.9	6.25
0.95	15.125

Big blocks.
Soft Limit of 3 MB, and a hard limit of 6 MB.  The fee multiplier of 1 mBTC.

size (MB)	Cost in mBTC
0	0
0.25	0
0.5	0
0.75	0
1	0
1.25	0
1.5	0
1.75	0
2	0
2.25	0
2.5	0
2.75	0
3	0
3.25	0.00757575757575758
3.5	0.0333333333333333
3.75	0.0833333333333333
4	0.166666666666667
4.25	0.297619047619048
4.5	0.5
4.75	0.816666666666667
5	1.33333333333333
5.25	2.25
5.5	4.16666666666667
5.75	10.0833333333333

I would say this would be what the miners would have to adopt for a change in a protocol, or they could adopt one of these fee tables without a change in the protocol.

Perhaps an SPV client could look at the size of the last ten blocks and judge what fee will probably get charged and quote that to the user.  Maybe 10 mBTC would be a better multiplier.


sdp

BIP 100 is the best available option
Jeff’s BIP 100 suggestion is not perfect and Meni’s comments on it our partially true.  Please see Meni’s comments here:
http://www.reddit.com/r/Bitcoin/comments/39qh66/i_would_like_more_discussion_about_this_objection/

Under BIP 100 miners will vote on the blocksize limit and it is true that miners interests are not perfectly aligned with users and therefore the limit may be sub optimal.  The limit will however be able to dynamically increase with demand, but in a conservative manner, which should alleviate many peoples concerns that blocks will get too big.  This proposal may result in blocks being too small, for the reasons Meni says, but this may be appropriate conservatism.  No solution is going to be perfect, but we need to have some kind of limit policy.  I think BIP 100 is the best suggestion so far, better than doing nothing, increasing the limit to 20MB or implementing an historic median based rule.

Median based rule versus BIP 100
The classic problem with the median based rule is that each individual miner may try to maximise short term profit by including more transactions in blocks.  Therefore the limit is increases.  As a whole the mining industry would prefer not to include that many transactions as they would rather reduce supply a bit and achieve higher fees.  This is analogous to the tragedy of the commons problem.  BIP 100 solves this, as miners collectively vote on the maximum blocksize, in order to maximise the USD value of fees.  It is in the miners long term interests to see higher transaction volumes, a higher USD exchange rate and for Bitcoin to succeed.  At the same time miners need to ensure the supply of space in blocks is sufficiency scarce to be able to maintain high enough fees.  Miners will make a balanced decision, partially based on price elasticity of demand, which is ultimately driven by the users.

More Economics
This proposal allows miners to control the supply side of the market for transactions and users to control demand.  This seems an appropriate balance to me.  Unlike what may occur in other industries when cartels try to restrict supply, this vote will not enable the miners to implement barriers to entry.  Any miner will always be free to join the industry at will, and vote for larger blocks, without a competitive disadvantage.

The economic value of mining transaction fee revenue is the core Bitcoin metric that should be maximised
Miners will vote to maximise the value of mining transaction fee revenue over the long to median term.  People have different visions for what Bitcoin should be and would like to prioritise different metrics.  Personally I think maximising mining transaction fee revenue is perhaps the most important long term metric measuring Bitcoin’s success.  This measure reflects Bitcoin’s long term sustainable security.  I challenge anyone to suggest an alternative single metric considered more indicative of success.

Improves Network security
BIP 100 is also a defence  against any future SHA256 two way pegged Sidechain.  Currently if one wants to use bitcoin is a zero trust environment, one needs to use the main Bitcoin Blockchain, this creates transaction demand that should eventually secure the network.  If in the future a Sidechain begins to become successful, users could switch to this and miners could also begin to switch over.  Eventually if the Sidechain has more revenue for miners, it could have a higher hashrate than the mainchain which could undermine the mainchain security.  A Sidechain is more likely to achieve this if it implements policies which maximise mining revenue.  I therefore propose that Bitcoin core implements policies to maximise mining revenue now, to secure it against this and other future attacks, which aim to incentivise miners more on another system.  Sidechains may be inevitable anyway, and I am not against competition for the mainchain, I just think its in Bitcoins interests for the mainchain to be the most secure.

Therefore, at this point, I give my support to BIP 100.

In theory you are correct, the formula needs a constant multiplier in the Bitcoin dimension. I've deliberately set the constant to 1 BTC and ignored its existence for several reasons:
1. Because of the speed at which the function grows, the constant doesn't really matter much, it more or less just shifts the typical block size by a small amount. That is, the function works at multiple scales of the BTC price / processing cost dimension.
2. Because it doesn't matter much, I wanted to keep the formula simpler, and not make it seem like there are many parameters which need to be chosen.
3. A value of 1 BTC gives reasonable results. (Perhaps a bit lower is better, since there's too much support on the low block sizes).

I wasn't really able to follow the rest of your calculations:
1. It seems you are giving values of the total penalty, but to figure out tx fees you need to look at its derivative.
2. It's not clear how many txs you assume fit in a MB; for values of, say, 3000 txs per MB, I get much lower values.