Why Peter Rs Fee Market Wont Work

[jonald_fyookball]

I am wondering if there is some possible configuration where the difficulty
of a solving a block diminishes as fees increase.  

We suspect this will sort of happen naturally as the fee/reward ratio increases.  But it won't be that difficulty will decrease as available fees increase, but that the hash rate will increase instead.  Since the expected time to solve a block is proportional to the difficulty/hashrate ratio, the effect of the hash rate increasing is similar to the difficulty decreasing.

The reason we suspect this will happen is that if there are very few fees available, then only the miners with the lowest electricity costs will mine.  As more and more fee-paying transactions pile up in mempool, then more and more miners will turn on their equipment, thereby increasing the network hash rate.  Block times will no longer follow an exponential distribution, but something more like a second-order gamma distribution instead.  

The really cool thing about this is that it means that bitcoin transactions will always remain essentially free if you are willing to wait a long time to get your transaction mined.  

An interesting theory indeed.

But then you have to ask, what security levels will be in place at the low point when "only the miners with the lowest electricity costs are mining"....and under what conditions will that be adequate or inadequate?

[Peter R]

I am wondering if there is some possible configuration where the difficulty
of a solving a block diminishes as fees increase.  

We suspect this will sort of happen naturally as the fee/reward ratio increases.  But it won't be that difficulty will decrease as available fees increase, but that the hash rate will increase instead.  Since the expected time to solve a block is proportional to the difficulty/hashrate ratio, the effect of the hash rate increasing is similar to the difficulty decreasing.

The reason we suspect this will happen is that if there are very few fees available, then only the miners with the lowest electricity costs will mine.  As more and more fee-paying transactions pile up in mempool, then more and more miners will turn on their equipment, thereby increasing the network hash rate.  Block times will no longer follow an exponential distribution, but something more like a second-order gamma distribution instead.  

The really cool thing about this is that it means that bitcoin transactions will always remain essentially free if you are willing to wait a long time to get your transaction mined.  

An interesting theory indeed.

But then you have to ask, what security levels will be in place at the low point when "only the miners with the lowest electricity costs are mining"....and under what conditions will that be adequate or inadequate?

I don't think it will really affect security.  A lot of people think that network security directly depends on hashing power, but it actually depends on more directly on network difficulty.  Network difficulty tracks the average hashing power over 2-week intervals.

In the theory described above, the average hashing power will still be high--only it will no longer be constant over the 10-min (avg) block intervals (it will start low and increase as fees build up in mempool).

[johnyj]

TL/DR: A transaction fee market exists without a block size limit assuming miners act rationally.
.
.
.
Miners being net econo-rational is probably an axiom for Bitcoin to work in the first place.

This is the biggest assumption of various schools of economics. However, economy is always political, economy policy is decided by the ruling class to manage the majority of poor people, so they are modeled towards average households that barely can live without the next pay check. For these people, econo-rational means short term profit seeking

However, the definition of econo-rational is different for different people, depends on their income, their time frame, and their risk tolerance level. If you move those theory to bankers and large capitalists, you will clearly see their behavior do not follow these models. As we know, the miners and pool owners are bankers and capitalists in bitcoin ecosystem, so they won't seek short term profit like average household, their concerns are much larger and longer term.

Yes, I completely agree.  Like I said up-thread, with this academic work regarding the transaction fee market, what we're doing is considering different lenses through which to view the problem, so that we can make incremental progress in our understanding.  We know that these models are imperfect, but we ask what happens if the assumptions hold, so that we can gain intuition about the more complicated real problem.  

More here: https://bitcointalk.org/index.php?topic=1274102.msg13678877#msg13678877

And YarkoL's post was great: https://bitcointalk.org/index.php?topic=1274102.msg13680077#msg13680077

After so many days of reading exhausted debate from each direction, I'm so tired    

I guess in the end people will just simply reach agreement by meetings, a solution that is not too far away from today's model because that is mostly like to reach consensus. Any large change is almost impossible at this stage

As long as block space is unlimited, the fee income will always be a neglectable part of the block reward because they only need to compensate for the orphaning risk, so the miner's need to limit the block size to increase their incentive, but by how much? I think a target of $0.1 per transaction is reasonable, it gives miners enough incentive (higher than orphan risk) while don't hurt average users, so that bitcoin payment is still quite competitive against main stream international payment solutions

And it is also easy for consumers, they know exactly it will cost $0.1 to do a transaction in bitcoin network, it is a constant. Certainty is good

[gmaxwell]

In regards to your comment about using pre-consensus techniques to completely eliminating block-size dependent orphaning risk, you were wrong about that too, as I demonstrated in the subchain paper.

Your work is not a non-existence proof, it's an analysis of a specific scheme. You made the same error in reasoning in your prior orphaning work, where you erroneously concluded that orphaning was proportional to the size of the block-- this later work implicitly admits the error by saying it is proportional not to size but to the rate of additions, but does not go far enough.

You proposed a needlessly deficient design, one which was inferior to what I'd already described to you, due to having the flaw of increasing orphaning risk to add new transactions.

This flaw can be simply (and compatibility with your design!) corrected by miners: in your scheme miners add transactions by adding them to a block attempt, thus taking orphaning risk.  Instead, miners could add transactions by first adding then to a separate hash tree, committed to by the block whos data is only shared with weak blocks, not full block solutions.  Only when a transaction is well and buried in this second tree, do miners then add it to their blocks.  Avoiding the orphaning risk resulting from doing so.

Not only can miners do this this mitigate the orphaning from the new addition, but you cannot prevent them from doing so.

[Peter R]

Your work is not a non-existence proof, it's an analysis of a specific scheme.

Agreed.  It's an analysis about a specific scheme.  The paper assumes that miners build subchains according to that scheme, similar to how they presently build the blockchain according to the underlying "Bitcoin scheme."  Miners would benefit by agreeing to that scheme because it would allow them to reduce network orphaning rates and more quickly verify transactions for users.  The scheme is also a Nash equilibrium, in that no "profitable deviation" from the agreed-upon behaviour exists that a single miner could exploit to earn a greater profit.¹

Taken to the academic limit of infinitely-fast weak block generation, the fee market still exists with the subchain scheme.  This was an interesting result that surprised me--I initially suspected the cost of block space might fall to zero (in only the strictly academic case).

The reason you don't understand subchains is the same reason you don't understand the Bitcoin consensus and mining process.  Bitcoin doesn't "work" just because of the rules encoded in the software--it works because there are people who choose which rules to enforce and what behaviour to encourage.  The miners and nodes are not automata; the will not agree to follow the rules that they've deemed bad for the health of the system, just like how they are presently tearing down the wall at 1 MB despite the feverish cries of Blockstream Core devs.

Anyways, if your fixated on schemes that completely eliminate block size dependent orphaning risks, it's easy to come up them.  I mentioned one early in the thread in response to Jonald (i.e., miners agree to mine a "practice" blockchain first such that the real blockchain is a verbatim copy [minus the blockheaders of course] but lagging by 1 hour).  Another scheme would be just to transmit the block headers and tell everyone else to "figure it out from the Merkle root!"  The challenge is to come up with one that may actually be followed.

¹This is why my analysis assumed miners built off the "highest fee" subchain rather than the "most work" subchain.  There would be a profitable deviation away from the most-work subchain, even though I suspect that is the behaviour that most miners would actually adhere to.

[Zangelbert Bingledack]

Bitcoin doesn't "work" just because of the rules encoded in the software--it works because there are people who choose which rules to enforce and what behaviour to encourage.

Key point, though it is hard to get people to see the intended interpretation of this.

[gmaxwell]

I'm not suggesting that miners engage in a wild scheme that acts against their own self-interests.

I'm pointing out that they merely need click a box and lose orphaning risk from adding new transactions. Miners who do this will be more profitable, even if some did not-- they will be out competed by more profitable miners who do, end up operating at profit levels that are uninteresting and exit the system.

It is _locally_ more profitable for them to do so-- the scheme you describe is NOT an equilibrium. Any miner can add a commitment for what they'd like to include and do so at no cost. Any miner can use any existing commitments to decide to include transactions earlier. No coordination is needed-- beyond the one time act of someone writing the software, no orphaning cost taken to move out of that state.

Miners making local decisions that are somewhat adverse to the health of the network but profitable for themselves is not a hypothetical-- it's an observed effect. Without it we never would have had huge mining pools, to give one example. (arguably: we'd have no GPU or asic miners, either!)

Expecting miners to continue indefinitely to take considerable orphaning costs against their own self-interests in order to uphold some abstract 'good of the network' that comes from the 'virtue' of orphaning is isomorphic to expecting them to just pay a $1 sin tax for every transaction they include without any enforcement mechanism. They simply won't.  

Even if you assume the existence of some conspiracy to punish people who behave locally rationally-- and ignore the potentially devastating effects this would have on the other properties of the system--, these schemes can be arbitrarily undetectable... anyone who didn't like it couldn't even tell it was happening.

[gmaxwell]

Anyways, if your fixated on schemes that completely eliminate block size dependent orphaning risks, it's easy to come up them.

If you're aware of this, why do you keep repeating as fact that it isn't possible? e.g.

In regards to your comment about using pre-consensus techniques to completely eliminating block-size dependent orphaning risk, you were wrong about that too, as I demonstrated in the subchain paper.

it's easy to come up them.I mentioned one early in the thread in response to Jonald (i.e., miners agree to mine a "practice" blockchain first such that the real blockchain is a verbatim copy [minus the blockheaders of course] but lagging by 1 hour).

By "easy to come up" you mean it's easy to copy them from the previously linked private email correspondence I had with you on this subject months ago? -- even including the time constant:

Miners volutarily participate in a fast consensus mechenism which
commits to transactions-- it could be a merged mined blockchain (like
the P2Pool chain, which commits to transactions) or something entirely
different.  No reward is given for participation in this scheme.  In
their Bitcoin blocks miners only include transactions which are at
least 6 blocks deep in that chain, moreover they include all
transactions which are in it, in a determinstic order.

I am cannot fathom what you are thinking.

[Peter R]

...

The reality Greg is that several years ago you convinced yourself that a strict protocol-enforced block size limit was required to prevent miners from destroying the system and that Bitcoin would only ever be a settlement layer. You then raised money and built a business around off-chain payment solutions to address a "market need" that you thought you had identified.  

But now you are beginning to realize that you were wrong.  Unfortunately, you've dug your heels in so deep that you just can't quite admit it--maybe not even to yourself.

At least now I can understand why you are so aggressive towards me. It was when people started talking about my work on scalability...how a transaction fee market exists without a block size limit, and later how subchains would permit massive on-chain scaling and improved 0-conf security...that forced you to re-evaluate what you thought you knew to be true.  And you lashed out.

You know deep inside that:

The block size limit will soon be raised.  

An application of weak blocks, perhaps even the subchain protocol, will be implemented.

Bitcoin will massively scale both on-chain and off-chain.  

And this war will come to an end.

[BlindMayorBitcorn]

^This has become a very popular opinion.

[brg444]

At least now I can understand why you are so aggressive towards me. It was when people started talking about my work on scalability...how a transaction fee market exists without a block size limit, and later how subchains would permit massive on-chain scaling and improved 0-conf security...that forced you to re-evaluate what you thought you knew to be true.  And you lashed out.

You know deep inside that:

The block size limit will soon be raised.  

An application of weak blocks, perhaps even the subchain protocol, will be implemented.

Bitcoin will massively scale both on-chain and off-chain.  

And this war will come to an end.

See someone, please. You need counseling.

[jonald_fyookball]

I'm not suggesting that miners engage in a wild scheme that acts against their own self-interests.

I'm pointing out that they merely need click a box and lose orphaning risk from adding new transactions. Miners who do this will be more profitable, even if some did not-- they will be out competed by more profitable miners who do, end up operating at profit levels that are uninteresting and exit the system.

It is _locally_ more profitable for them to do so-- the scheme you describe is NOT an equilibrium. Any miner can add a commitment for what they'd like to include and do so at no cost. Any miner can use any existing commitments to decide to include transactions earlier. No coordination is needed-- beyond the one time act of someone writing the software, no orphaning cost taken to move out of that state.

Miners making local decisions that are somewhat adverse to the health of the network but profitable for themselves is not a hypothetical-- it's an observed effect. Without it we never would have had huge mining pools, to give one example. (arguably: we'd have no GPU or asic miners, either!)

Expecting miners to continue indefinitely to take considerable orphaning costs against their own self-interests in order to uphold some abstract 'good of the network' that comes from the 'virtue' of orphaning is isomorphic to expecting them to just pay a $1 sin tax for every transaction they include without any enforcement mechanism. They simply won't.  

Even if you assume the existence of some conspiracy to punish people who behave locally rationally-- and ignore the potentially devastating effects this would have on the other properties of the system--, these schemes can be arbitrarily undetectable... anyone who didn't like it couldn't even tell it was happening.

Sounds like you are simply describing a tragedy of the commons.

You say that miners won't take on orphaning risks charitably,
which I assume is true, but the whole point of Peter's fee
market is that the fees would compensate for the risk.

Going to back to your argument earlier in the thread that
there might be negligible orphaning risks, then my next
question would be:

Even if there's no "orphaning risk" due to propagation because
of relays, etc... wouldn't miners still prefer to solve the
blocks with the biggest fees anyway?

[gmaxwell]

Sounds like you are simply describing a tragedy of the commons.

Indeed, you can look at it like that. And one that can be avoided in the standard way: by not allowing unrestricted exploitation of the commons.

Even if there's no "orphaning risk" due to propagation because
of relays, etc... wouldn't miners still prefer to solve the
blocks with the biggest fees anyway?

Indeed, and absent a limit, flexcap, or forced collusion (or something no one has described yet), that is the one that involves taking pretty much all transactions that pay any fee at all.

Peter_R's argument was that a limit isn't needed to achieve this effect because orphaning will accomplish it, but we see now that he finally agrees that volume related orphaning pressures can be more or less completely sidestepped by miners:

schemes that completely eliminate block size dependent orphaning risks, it's easy to come up them.

And then he goes on to describe one of the schemes I emailed him.

[YarkoL]

Sounds like you are simply describing a tragedy of the commons.

Indeed, you can look at it like that. And one that can be avoided in the standard way: by not allowing unrestricted exploitation of the commons.

And who erects the fences? Who wears the crown?

[jonald_fyookball]

Sounds like you are simply describing a tragedy of the commons.

Indeed, you can look at it like that. And one that can be avoided in the standard way: by not allowing unrestricted exploitation of the commons.

Even if there's no "orphaning risk" due to propagation because
of relays, etc... wouldn't miners still prefer to solve the
blocks with the biggest fees anyway?

Indeed, and absent a limit, flexcap, or forced collusion (or something no one has described yet), that is the one that involves taking pretty much all transactions that pay any fee at all.

Peter_R's argument was that a limit isn't needed to achieve this effect because orphaning will accomplish it, but we see now that he finally agrees that volume related orphaning pressures can be more or less completely sidestepped by miners:

schemes that completely eliminate block size dependent orphaning risks, it's easy to come up them.

And then he goes on to describe one of the schemes I emailed him.

Rather than cap the blocksize, I would rather raise the minimum non-zero fee, to say 1 penny.
Either pay a penny, or don't.

Consider that 200tps would give you $172,800 a day in security under such a scheme.

[gmaxwell]

And who erects the fences? Who wears the crown?

The same force that prevents you from spending my coins: the hardcoded rules that define the Bitcoin system, autonomously enforced by every user.

Rather than cap the blocksize, I would rather raise the minimum non-zero fee, to say 1 penny.
Either pay a penny, or don't.

Who decides what a bitcoin is worth?

Consider that 200tps would give you $172,800 a day in security under such a scheme.

What reason do you have to believe that 200 TPS is viable in a decentralized consensus system or will be in an interesting timeframe?

What happens if 200 TPS _is_ in fact viable, but at the limit of it... but the offered load at a 1ct fee is 40000 TPS? Do you expect the miner(s) to turn down an extra $34m/day in income to preserve decentralization?

Why do you believe $172k/day is adequate security?  That is only $1200/block.  If someone accepts a single confirm, then it would be pretty profitable (200% ev) to attempt a reorg attack against them if you have access to--say-- 10% of the hashpower and can make a vulnerable $12k transaction with them.

[jonald_fyookball]

And who erects the fences? Who wears the crown?

The same force that prevents you from spending my coins: the hardcoded rules that define the Bitcoin system, autonomously enforced by every user.

Rather than cap the blocksize, I would rather raise the minimum non-zero fee, to say 1 penny.
Either pay a penny, or don't.

Who decides what a bitcoin is worth?

I suggested a penny as a minimum non-zero fee.
The idea is to have a discrete, granular unit
(obviously higher than a satoshi) so that the
problem you described where transactions
with "any fees at all" would not be negligible.

This penny could be converted into satoshis
and later adjusted by miners via emergent
consensus.

Consider that 200tps would give you $172,800 a day in security under such a scheme.

What reason do you have to believe that 200 TPS is viable in a decentralized consensus system or will be in an interesting timeframe?

I'm not sure if you are asking whether its
technically viable, or adoption wise.

As far as the adoption, it will be decades
before subsidies become miniscule and therefore
there is enough time for orders of magnitude of growth.
If not, then Bitcoin is probably a failure.  As
Satoshi said "“I'm sure that in 20 years
there will either be very large transaction
volume or no volume.”

What happens if 200 TPS _is_ in fact viable, but at the limit of it... but the offered load at a 1ct fee is 40000 TPS? Do you expect the miner(s) to turn down an extra $34m/day in income to preserve decentralization?

Obviously, that is a different problem
than not having enough
fees.

If there is demand for 40,000 TPS, it
doesn't make sense to limit the network
to 200 tps.  I think I see what you're
saying as far as very large blocks could
create a less decentralized landscape, but
on the other hand, accomodating the greater
load through other layers would create its
own form of centralization.  

This is an interesting area to think about.

Why do you believe $172k/day is adequate security?  That is only $1200/block.  If someone accepts a single confirm, then it would be pretty profitable (200% ev) to attempt a reorg attack against them if you have access to--say-- 10% of the hashpower and can make a vulnerable $12k transaction with them.

I never said $172 was adequate necessarily;
it was just an example.  We could possibly
have higher fees than a penny or more transactions.
How much security do you feel is adequate?