[XMR] Monero Improvement Technical Discussion

[hyc]

I was doing some research on long term storage solutions for work and came across the notion that the factor that limits a SSD's life is the number of writes performed. Reading from a cell doesn't really kill the cell, its the need to write to a cell that will eventually kill it.

So, my question is whether LMDB currently (or can be modified to) only write to the database once for a given entry.

My gut tells me that - yeah - because the blockchain is just a database, then once your node is sync'd up, everything is written. However, the innerworkings of LMDB are unknown to me - so perhaps it has to rewrite certain locations?

Reading from a cell can wear it out too, it just takes longer to happen.

LMDB is a copy-on-write design, so it almost never overwrites existing pages. It is actually proven to be more SSD/flash-friendly than most other DB designs in existence.

Here's some relevant work on a flash-optimized database, compared against LevelDB - LevelDB is over 70x worse in terms of write amplification and data wearout. https://www.usenix.org/conference/atc15/technical-session/presentation/marmol

(LMDB is not compared in this work, but we have other tests that show LMDB's write amplification is far smaller than LevelDB's http://symas.com/mdb/#bench )

Relax. There is nothing going on in the database world that LMDB hasn't already solved.

I've been developing systems on SSDs for over a decade, I encountered and solved these wearout problems long before any other DB authors even knew they existed. LMDB was designed for solid state storage.

http://forums.storagereview.com/index.php/topic/22805-does-mechanical-storage-have-a-future/#entry230060
http://forums.storagereview.com/index.php/topic/24749-160-gb-flash-ssd-anounced/#entry240229

[TPTB_need_war]

Random writes can wear out the SSD much more rapidly is because it causes entire sectors to have to be cleared and rewritten (potentially even moved) even if only one bit is changed in the sector. Sequential writes are much healthier for the SSD.

[hyc]

Random writes can wear out the SSD much more rapidly is because it causes entire sectors to have to be cleared and rewritten (potentially even moved) even if only one bit is changed in the sector. Sequential writes are much healthier for the SSD.

Yes, but that's only part of the story. Flash sectors are 512 bytes each, LMDB writes in pages (4KB on common systems). So right off the top, the wear out factor is reduced by a factor of 8. Also LMDB is copy-on-write so rewriting of individual pages is a rare occurrence; rewriting of single random sectors is a non-occurrence. The one factor we can't control for is filesystem and partition alignment - if your disk layout doesn't line up with a multiple of 2MB, you're probably fragmenting all your accesses. This is the essential part of using SSDs - flash memory can be read and written in sectors, but erases can only be done in erase blocks that are commonly 2 or 4 MB in size. The insane thing is that for years, flash drive vendors shipped their devices with a default disk geometry of 255 heads, 63 sectors/track, which gives you cylinders of 16065 sectors per cylinder. Even though disk devices generally use LBA instead of CHS addressing, disk partitioning tools and filesystems still use cylinders. I always reformat to 256 heads, 32 sectors/track which gives 8192 sector cylinders: 4MB. A misaligned partition can cause major performance degradation and accelerated wearout.

[GingerAle]

xpost: https://forum.getmonero.org/4/academic-and-technical/2525/enhancing-the-auditability-of-monero-transactions

Over the past couple of weeks, I've seen at least two instances where its become apparent the auditability of Monero transactions is inadequate, and this is a common critique of Monero for those that dive past the limitations of the viewkey. It's incredibly ironic that auditability is an issue, but it does make sense. There needs to be a trustless way to share your financial information if you choose to do so.

To date, there are two mechanisms: the view key, which has the fault of only showing incoming transactions, and the databasing by simplewallet of transaction history, which has the fault of relying on trust. "I trust that you have logged all of the transactions associated with this account."

I don't have a solution, but I'm hoping that we can collectively start thinking about a solution. Like I've said before, I don't understand the cryptography enough to understand why its difficult to view outgoing transactions without the ability to sign them.

[TPTB_need_war]

I don't have a solution, but I'm hoping that we can collectively start thinking about a solution. Like I've said before, I don't understand the cryptography enough to understand why its difficult to view outgoing transactions without the ability to sign them.

Viewing the outgoing transactions would break the rings for others, not just own, by reducing the anonymity sets for example if the entity you provided the viewkey to was a very popular entity to provide viewkeys to.

Other than that, I think it would be technically possible to make a viewkey for outgoing transactions by having two private keys and then proving in zero knowledge that the two ring signatures were equivalent, without giving up your private key which has power to spend your outputs.

[smooth]

Over the past couple of weeks, I've seen at least two instances where its become apparent the auditability of Monero transactions is inadequate

It isn't entirely clear to me whether it is or not. Is auditability of cash or gold inadequate?

The auditability comes from surrounding processes and not from the asset itself.

Maybe it is a mistake to try to push audibility too far into the system itself. The current receive view keys might be a reasonable middle ground, because with one it can be proved that you received some funds. It is then up to you to show what you did with them, and if you can't you can be held responsible (for embezzlement, etc.)

[GingerAle]

Over the past couple of weeks, I've seen at least two instances where its become apparent the auditability of Monero transactions is inadequate

It isn't entirely clear to me whether it is or not. Is auditability of cash or gold inadequate?

The auditability comes from surrounding processes and not from the asset itself.

Maybe it is a mistake to try to push audibility too far into the system itself. The current receive view keys might be a reasonable middle ground, because with one it can be proved that you received some funds. It is then up to you to show what you did with them, and if you can't you can be held responsible (for embezzlement, etc.)

Valid points - separating the asset from the rest of the mechanisms. I would argue, then, as have others, that we need to come up with a better way to describe the actual auditibility capacities of the extant monero network. I was under the impression, until I learned better, that Monero was private / opaque by default, but then you could turn it into a bitcoin-style thing if you gave someone the ability to do so. I think the term "viewkey" has this baked into it "oh, now I can view everything, where before I couldn't, because cryptography".

I just don't like the bolded part because its more work, and will probably be the number 1 reason future monerians would use a centralized service - to keep them compliant by keeping all their "receipts" in order. I guess ultimately its a matter of keeping your wallet.bin safe and noncorrupted once this becomes a stable thing. But i've gone through at least 5 wallet.bins for every one of my accounts by now.

[smooth]

I guess ultimately its a matter of keeping your wallet.bin safe and noncorrupted once this becomes a stable thing. But i've gone through at least 5 wallet.bins for every one of my accounts by now.

Have you seen the corruption recently? I think the wallet saving was changed to not overwrite in place within the past several months, which should eliminate most off the corruption. In time it will be replaced with a database.

[GingerAle]

As I was dosing off last night I remembered a possible point.... this puts a level of trust back into the system, which seems anathema for a trustless currency system. Unless we come up with a way to decentralize the auditing mechanism?

i dunno ... we share hashes of our wallets ?

fresh morning thoughts. possibly useless.

[GingerAle]

Ye olde selection of outputs problem. We'll call this the Monero problem. Maybe it will go down in the books like the byzantine generals problem or whatever.

I have a list of things. Some of the things in this list are mine - I own them. I want to use some of my things in this list without an observer knowing which of the things I'm using are actually mine. Therefore, I select my thing and some others as decoys, so that the observer doesn't know which of the things is mine. However, the nature of the things is such that once they are used by the true owner, they can not be used again. Thus, older things in this list have a higher probability that they have already been used and are only being used as decoys, though there is no way to determine whether a thing has been used.

Thus, how do I select my set of decoys to have the highest probability of appearing to be unused?

In triangular distribution (I think this is what is currently used in Monero), as demonstrated by mWo12 here: http://pastebin.com/raw/4TzcF9b9 , seems to use the pattern of

1. recent output - highly likely not used: seen as the columnar pattern on the far right.
2. then as far as I can tell, completely random selection throughout the blockchain. Unknown what the probability of usage actually is.

Probability of Usage - this is probably something that we could define somehow. Out of band information can tell us that probability of usage increases with more users of the blockchain, but how this manifests in the blockchain is a different beast.

[john-connor]

How does Monero propose to resolve the fact that it's blockchain is growing faster than Moores Law and pruning is so limited? Are there any proposals for a solution to the new faulty database that can easily fail an SSD? So far I've failed two during initial download at the 50% mark and the disk writes were well into the 100's of gigabytes(what is this database doing?).

[TPTB_need_war]

How does Monero propose to resolve the fact that it's blockchain is growing faster than Moores Law and pruning is so limited?

Human populations don't grow faster than Moore's law. Duh.

Disk arrays scale to anything we can fathom.

The issue is that no block chain consensus can maintain decentralization of validation, not because of scaling problems but because of the fundamental economic reality that not every miner can have an equal share of the hashrate, thus verification costs are not shared equally. The creates an asymmetry where economies-of-scale will maximize profit and grow hashrate the fastest thus centralizing mining.

The solution requires some clever innovation on proof-of-work.

[hyc]

How does Monero propose to resolve the fact that it's blockchain is growing faster than Moores Law and pruning is so limited? Are there any proposals for a solution to the new faulty database that can easily fail an SSD? So far I've failed two during initial download at the 50% mark and the disk writes were well into the 100's of gigabytes(what is this database doing?).

To be blunt, you're full of s#it.

https://gist.github.com/hyc/33f3eec6bae83246209d

[generalizethis]

How does Monero propose to resolve the fact that it's blockchain is growing faster than Moores Law and pruning is so limited?

Human populations don't grow faster than Moore's law. Duh.

Disk arrays scale to anything we can fathom.

The issue is that no block chain consensus can maintain decentralization of validation, not because of scaling problems but because of the fundamental economic reality that not every miner can have an equal share of the hashrate, thus verification costs are not shared equally. The creates an asymmetry where economies-of-scale will maximize profit and grow hashrate the fastest thus centralizing mining.

The solution requires some clever innovation on proof-of-work.

Not necessarily, algorithms could be programmed to move your funds between coins if an attack threshold is passed--of course this requires trustless exchanges to fill in the gap left by the inability to decentralize mining, and also produces lemming effects if the coin's mining doesn't adjust responsively, though this assumes the attacker is just greedy and not actually trying to destroy the coin. In a full-on attack of crypto,  a war to end the battles,  the algorithm approach forces the attack to be multi-pronged, but would not prevent coins from attacking each other to gain market share, though my guess is miners would make algorithm adjustments and speed becomes an asset as much as hashing power--I'm rambling possabilities, my guess is that there some noob assumptions to flesh out, if not totally dismiss.

[ArticMine]

This brings us back to the Cryptonote adaptive blocksize limit combined with a tail emission found in Monero where:
1) The cost of mining a block is set by the block subsidy

Correct, meaning the amount of hashrate miners spend will be equal to the block subsidy[1] (where block subsidy will ultimately be Monero's perpetual tail reward which is necessarily a fixed # of coins), because (as I pointed out in our prior discussion) transaction fees will trend to costs, due to that the median block size M_N will trend upwards to match market demand and thus there is no pricing power on transaction fees.

[1] Note this means the tail reward security of Monero will be very weak and insufficient.

2) The total amount in fees per block has to rise to a number comparable to, but most of the time smaller, than the block subsidy.

You wrote that before in our prior discussion:

The reason the above two scenarios do not apply to a Cryptonote coin with a tail emission such a Monero becomes apparent when one considers the economics of the total block reward components of fees and base reward (new coin emission). If the total in fees per block significantly exceed the base reward then it becomes economically attractive for miners to burn coins to the penalty by mining larger blocks. The block size rises until the total fees per block fall below a level where it is uneconomic for the miners to pay the penalty by increasing the blocksize. This level is comparable to the base reward. It is at this point where the need for a tail emission becomes clear, since without the tail emission the total block reward (fee plus base reward) would go to zero.

And it still doesn't make any sense to me. The block size will trend upwards to match transaction demand, because the penalty is driven to 0 as the median block size increases as  miners can justify burning some of the transaction fees to the penalty. That drives the median block size upwards, which drives the penalty to 0 again. The median block size doesn't have any incentive to decrease again, thus transaction fees then fall to costs.

Sorry as I told you before, Monero does not solve the Tragedy of the Commons in Satoshi's design. It does adaptively increase the block size while preventing spam surges.

I doubt John Conner's design has achieved any better, because as I explained at our prior discussion, there is no decentralized solution to that Tragedy of the Commons in the current proof-of-work designs. I have a solution, but it is a very radical change to the proof-of-work design that relies on unprofitable mining by payers.

As far as I can see, Monero has not solved the Tragedy of the Commons in Satoshi's design. I reiterated my rebuttal to ArticMine:

https://bitcointalk.org/index.php?topic=1441959.msg14599446#msg14599446

Clarification:

Security of a coin will be very tied to its transaction rate × average transaction size, i.e. velocity adoption and wealth of the velocity. The problem I have with the fixed size tail reward as compared to the design I am contemplating is that tail reward only captures those metrics indirectly through exchange price appreciation. I am not sure if the two models are equally powerful. I will need to think more deeply about it. My design also has an orthogonal tail reward.

Edit: some aspects of Monero's tail reward and block size adjustment algorithm are analogous to aspects of my design. There are some other things I didn't mention. I will need to really take the time to distil this into a carefully written white paper. So I would caution readers not to form any concrete conclusions (either for or against any design mentioned here) from these vague discussions.

BTW, I would suggest that Tragedy of the Commons is an ineffective analogy for explaining whatever it is you are trying to explain because obviously-intelligent people such as ArticMine don't understand it. It may be that you are entirely correct, but if you want to communicate effectively you need a differently-worded explanation.

Agreed at the appropriate time. I deem it necessary to be vague since I am months (or moar!) away from implementing my design.

The fundamental issue here is that transaction fees should not be seen as a way to secure a Cryptonote coin such as Monero. The security of the coin is based on the base reward. Now that does not mean tha transaction fees will tend to zero and stay there. In fact one would expect the total transaction fees per block to reach for the most part an equilibrium at some fraction of the block reward. To understand this we must understand that while transaction fees are needed to overcome the penalty in order increase the blocksize, there is no rebate on penalty when the blocksize falls. This means that just normal fluctuation in transaction demand will require a significant fraction of the block reward in transaction fees. The median blocksize adjustment time is less than a day in Monero. In practice one would expect total transaction fees per block to temporarily approach or even slightly exceed the block reward if there is a sharp rise in transaction demand, and conversely it is possible for transaction fees to temporarily fall close to zero if there is a sharp drop in transaction demand. Since security is only as strong as its weakest point for this reason alone one cannot count on transaction fees to secure a Cryptonote coin. The implications for Monero is that the tail emission alone becomes the source for POW security. A further important conclusion is that a coin that uses a Cryptonote adaptive blocksize or something similar and does not have a tail emission or equivalent (for example demurrage) big enough to secure the coin will become insecure and fail.

In the case of Monero one must keep in mind that over time one would expect the tail emission will actually reach an equilibrium with lost coins for a given purchasing power of Monero. The assumption here is that lost coins are proportional to the the total emission for a given purchasing power and transaction velocity. One advantage that Monero has here is that monitoring Bytecoin could provide a significant early indication of the risk of too low a tail emission with a two year or more lead time. Paradoxically this early warning system works because of the Bytecoin two year premine / ninjamine. Also the Bitcoin inflation rate will also fall below that of Monero so it could also provide an advance indication of risk. As has been indicated above this risk is of course largely mitigated with exchange rate appreciation, which would normally correlate very strongly with the use of the currency.

[GingerAle]

Thought of this because I came across another bitcoin block propagation network and hope that monero can be designed to avoid any type of centralizing development.

I'm jumping ahead possibly many years, but suppose we encounter scaling issues regarding transaction propagation. Presumably, there are logical means for light block propagation that has been described elsewhere. But what if we encounter the fact that transactions do not propagate quickly enough? Monero transactions are larger than your average cryptocurrency, and its unknown whether a ring size of 3 (mixin of 3 in the old parlance) will stay the standard. I.e., what if a ring size of 10 becomes necessary? Or 100?

So my main question is whether the current block hash is made from all of the information in the transactions (inputs, outputs, etc), or if it is made from the hash of the transaction.

If its made from the hash of the transaction, then we could implement a hash-first transaction propagation. So you could imagine the network cache would be split into more layers. There would be a hash pool (just the hashes), then the transaction pool (hashes + rest of transaction data) and then however the rest of the cache is divied up (i.e., if lightblocks are implemented there's another caching layer).

So, first the transaction hash races through the relay network with its measly size of whatever it is. 64 bytes? Every node that gets it can now start mining with that transaction in their block. Presumably, while they are mining on this transaction, the rest of the data will catch up. Hell, it could even be possible to solve a block and broadcast a block before the rest of the transaction data even arrives.

If its made from the entire transaction, then we'd first need to modify block architecture to just be a hash of the hashes.

Hrm... I see it now. The main problem is validation. But then again, if the transaction data associated with a given hash turns out to be crap, then the transaction data isn't stored in the blockchain. So at worst you have empty hash placeholders in the blockchain (which could be pruned), so you sacrifice blockchain bloat (and potentially grease the ability to spam the network) for transaction speed propagation.

Unless there was some way to determine if a hash is valid. But then we're defeating the whole purpose of unidirectional mathematics (or whatever the term is called).

[aminorex]

Post-quantum stuff:

https://eprint.iacr.org/2015/1092.pdf
https://eprint.iacr.org/2016/659.pdf

[GingerAle]

Okay. First things first - I deleted a lot of posts during the TPTB_need_war threadsplosion that is best summarized here -
https://bitcointalk.org/index.php?topic=1139756.msg13634887#msg13634887

there are remnants of the discussion still in there, because some contain some good tidbits and frankly I just got lazy. Deleting posts one by one is a bitch.

After some pruning of this thread I want to start things up with the auto adjusting transaction fees.

@arcticmine, I remember that you were going to lead this thought train. I'm curious if you've had any insights into things?

During the last dev meeting, you mentioned that the minimum block inclusion fee (what we commonly think of as the transaction fee) would be based on a calculation that uses the block size as the independent variable. However, in your original musings on the topic, you suggested that the difficulty would be used

https://bitcointalk.org/index.php?topic=1139756.msg12199259#msg12199259

and edited for the final post:

There should be a factor for block subsidy.

Yes. In order for the difficulty to be a surrogate for the value of XMR one needs to factor in the actual block reward, So the formula would look as follows:

F = F₀*D₀*R_A/(D_A*R₀)

Where

F₀ is a constant comparable to the current per kb fee
R₀ is a constant comparable to the current block reward
R_A is the average block reward of the blocks N-2K to N-K
D₀ is a constant comparable to the current difficulty
D_A is the average difficulty of the blocks N-2K to N-K
N is the last block number
K is a constant for example 1000  

One could replace R_A by the average of base block reward namely by netting out fees and penalties; however then it would not track value as closely particularly if fees were to dominate the actual block reward in the future.

The question of the responsiveness of the difficulty to a sudden increase in price is valid to a degree; however it is tempered by the fact that the increase would be temporary, the increase would hit XMR holders at a time when they are experiencing a significant increase in the value of their XMR,  and in the worst case scenario any drop in transaction volume would likely temper the sudden price movement.  Attacks based on manipulating the hash rate I do not see as an issue here simply because any impact on fees is extremely minor compared to the risk of a 51% type attack.

My main concern here is to not create a situation similar to what is currently happening in Bitcoin with the blocksize debate. It is fairly easy to get the necessary consensus to fork an 18 month old coin, it is quite another matter to try to undo the fork 5 years later; particularly when there is no immediate threat. This is the critical lesson we must learn form Bitcoin.    

The discussion initiated by that post was good, and I recommend re-reading.

We could also use the data from the recent price spike and the subsequent difficulty adjustment to see how the fee would have played out. I'm using the queenly we here and should be smacked for doing so. Maybe I'll play around with excel in a bit.

[ArticMine]

Thanks for reviving this thread and the old discussions from last summer. My thoughts on this have evolved considerably since then largely due to gaining a further understanding of the adaptive blocksize limit in Monero and how it impacts fees. In the following post I discussed the response of the adaptive blocksize limit to a spam attack and this is critical in understanding what determines fees in Monero ove the long term.

ArticMine PMed me after I wrote that flaming post, and said he would reply after studying my posts. He has not yet replied. Does that mean I am correct and there is no solution for Monero. I think so.

It is fundamental. Afaics, you'd have to completely rewrite Moaneuro.

Rewrite Monero, is not necessary at all but some documentation on how the Cryptonote adaptive blocksize limits actually work is needed, especially given the formula in section 6.2.3 of the Cryptonote Whitepaper is wrong. https://cryptonote.org/whitepaper.pdf. My response will come in time.

I will start by examining the Cryptonote Penalty Function for oversize blocks. This is critical to understand any form of spam attack against a Cryptonote coin. From the Cryptonote whitepaper I cited above the penalty function is:

Penalty = BaseReward (BlkSize / M_N - 1)²

The new reward is:

NewReward = BaseReward - Penalty

Where M_N is the median of the blocksize over the last N blocks
BlkSize is the size of the current block
BaseReward is the reward as per the emission curve or where applicable the tail emission
NewReward is the actual reward paid to the miner
The Maximum allowed blocksize, BlkSize, is 2M_N
The penalty is only applied when BlkSize > (1 + B_min) M_N Where 0 < B_min < 1 In the Cryptonote whitepaper B_min = 0.1.
 
The error in the Cryptonote Whitepaper was to set NewReward = Penalty

For simplicity I will define:
BlkSize = (1+B) M_N
BaseReward = R_base
Penalty (for a given B) = P_B
NewReward (for a given B) = R_B

The penalty for a given B becomes:
P_B = R_baseB²
While the new reward for a given B becomes:
R_B = R_base(1 - B²)
The first derivative of P_B with respect to B is
dP_B / dB = 2R_baseB

In order to attack the coin by bloating the blocksize the attacker needs to cause at least over 50% of the miners to mine oversize blocks and for an expedient attack close to 100% or the miners to mine oversize blocks. This attack must be a maintained over a sustained period of time and more importantly must be maintained in order to keep the oversized blocks, since once the attack stops the blocks will fall back to their normal size.  There are essentially two options here:

1) A 51% attack. I am not going to pursue this for obvious reasons.

2) Induce the existing miners to mine oversize blocks. This is actually the more interesting case; however after cost analysis it becomes effectively a rental version of 1 above. Since the rate of change (first derivative) of P_B is proportional to B the most effective option for the attacker is to run the attack with B = 1. The cost of the attack has as a lower bound R_base but would be higher, and proportional to, R_base  because miners will demand a substantial premium over the base reward to mine the spam blocks due to the increased risk of orphan blocks as the blocksize increases and competition from legitimate users whose cost per KB for transaction fees needed to compete with the attacker will fall as the blocksize increases. The impact on the coin is to stop new coins from being created while the attack is going on. These coins are replaced by the attacker having to buy coins on the open market in order to continue the attack. The impact of this is to further increase the costs to the attacker.

It at this point where we see the critical importance of a tail emission since if R_base = 0 this attack has zero cost and the tragedy of the commons actually occurs. This is the critical difference between those Cryptonote coins that have a tail emission, and have solved the problem, such as Monero and those that do not, and will in a matter of time become vulnerable, such as Bytecoin.

There are two very distinct scenarios I see for setting fees in Monero.

The first scenario is when the  median of the blocksize over the last N blocks, M_N < effective median of the blocksize over the last N blocks, M₀ = 60000 bytes. This is the current  scenario and not the scenario I discuss above, since M_N is replaced above by M₀ eliminating the penalty for a blocksize increase. The current M_N is less than 300 bytes.

The second scenario is where M_N > M₀ and the penalty applies. In this scenario per KB fees are proportional to the base reward divided by the median of the blocksize over the last N blocks, R_base/M_N. The objective here is to stimulate an efficient market between blocksize scaling and fees. One way to achieve this is to set 5 fee levels corresponding to transaction confirmation priorities. minimal, low,  normal, high, very high. The per KB fees are calculated using the known values of the block reward, and, the median of the block size. One places  minimal and low below the penalty boundary,  normal at the penalty boundary and high and very high above the penalty boundary.

The prior discussion in this thread can in reality only be possibly applied to the first scenario for the minimal fee and the low fee. The rest of the fees would have to be set as above but using M₀ rather than M_N.

Edit: One important consideration is that there is a 200x increase in transaction volume and possible corresponding increase in price before we trigger the second scenario, so we will have to scale down the fees with no trigger of the penalty. It is here where my original idea as edited and modified above may be a possibility to set the low minimal and low fees.

[GingerAle]

( snip )
There are two very distinct scenarios I see for setting fees in Monero.

The first scenario is when the  median of the blocksize over the last N blocks, M_N < effective median of the blocksize over the last N blocks, M₀ = 60000 bytes. This is the current  scenario and not the scenario I discuss above, since M_N is replaced above by M₀ eliminating the penalty for a blocksize increase. The current M_N is less than 300 bytes.

The second scenario is where M_N > M₀ and the penalty applies. In this scenario per KB fees are proportional to the base reward divided by the median of the blocksize over the last N blocks, R_base/M_N. The objective here is to stimulate an efficient market between blocksize scaling and fees. One way to achieve this is to set 5 fee levels corresponding to transaction confirmation priorities. minimal, low,  normal, high, very high. The per KB fees are calculated using the known values of the block reward, and, the median of the block size. One places  minimal and low below the penalty boundary,  normal at the penalty boundary and high and very high above the penalty boundary.

The prior discussion in this thread can in reality only be possibly applied to the first scenario for the minimal fee and the low fee. The rest of the fees would have to be set as above but using M₀ rather than M_N.

Edit: One important consideration is that there is a 200x increase in transaction volume and possible corresponding increase in price before we trigger the second scenario, so we will have to scale down the fees with no trigger of the penalty. It is here where my original idea as edited and modified above may be a possibility to set the low minimal and low fees.

I'm confused, though I think I'm putting it together. I see you're threading the needle of making the fee adaptive while giving the fee the ability to serve its intended purpose of preventing blocksize expansion. I think you were on to something before with using the difficulty as an on-chain surrogate of external value. I think the need for that factor will exist at any stage of the chain's life - during initial distribution curve and during the tail emission. Your second scenario above seems focused on the tail emission portion of the coins existence, which doesn't happen for another ... however many years.