Can't we avoid reorgs once and for all?

[NotATether]

The analogy with the dice is likely flawed, and I find it more complicated than needed. Mining works simply enough, over-simplifying might have opposite result.

Energy is only wasted if two or more miners are searching for the hash in the same search space.

E.g miner A and B are both calculating the SHA256d in the region xxxxxxxxxxxxx[10000000-20000000]xxxxxxx... where the x's are constant digits and might be different from each other, within one hash. But each of A and B'a hashes, this sequence of digits is always going to be same, effectively making A and B adjust only the digits in brackets.

When you have A and B searching in such a small range like that, there will of course be overlapping in the search space fairly often, which is why most mining software and pools try to give miners unique and sufficiently large ranges to work on.

[BlackHatCoiner]

We do not consider those hashes to be wasted, so why would a hash that ends up being reorged out be considered wasted? It's no more or less wasteful than all of the hashes that weren't valid.

It's wasted in the sense that in a parallel system, wherein you had had 20 minutes interval and the exact same hash rate, you'd have lower chances to mark valid blocks as invalid.

Reorgs aren't bad because they waste energy. They're bad because they mean that low confirmation numbers cannot be relied upon.

Unreliable low confirmation numbers means less security. Less security with the same hash rate, when that very hash rate only contributes security-wise, is waste.

They all went to securing the network, regardless of the method by which they were unsuccessful.

Building on top of a valid block, regardless of the invalid hashes, is the work. Building on top of an invalid block, while part of the process, is and should be considered waste. Just as executing a 51% attack without the necessary hash rate is discouraging, because the attacker is likely to spend energy to beat the air-- ergo to waste energy.

[o_e_l_e_o]

Building on top of a valid block, regardless of the invalid hashes, is the work. Building on top of an invalid block, while part of the process, is and should be considered waste.

This would be true only if the block being built upon is invalid. That would indeed be wasted work, as it is impossible for that work to find the next block and therefore it isn't contributing to the security of the network. But the stale block from such a chain split is not invalid. It is perfectly valid, and could indeed have been the accepted block if a different miner had found the next block. All the work built on top of it is still contributing to the security of the network.

It might be wasted in the sense that it was later decided that this hash power was mining on top of a stale block, but it is not wasted in the sense that at the time it wasn't contributing to the security of the network.

[BlackHatCoiner]

But the stale block from such a chain split is not invalid. It is perfectly valid, and could indeed have been the accepted block if a different miner had found the next block.

It's the old chain invalid. The block might be valid, but another miner successfully questioned that chain. Transactions in that block are not confirmed anymore; at least not until someone else confirms them. And energy spent on a chain that is not the longest one difficulty-wise, is wasted energy.

All the work built on top of it is still contributing to the security of the network.

All the work put in the invalid chain is wasted work, because the same work could have been used to provide security. Energy spent on such chain doesn't help anywhere.

It might be wasted in the sense that it was later decided that this hash power was mining on top of a stale block, but it is not wasted in the sense that at the time it wasn't contributing to the security of the network.

No, but we acknowledge that a small percentage of blocks are stale. Therefore, energy spent on that small percentage, is wasted. We might not know which blocks are stale, and thus we don't know at the time we mine them that we're wasting energy, but that's what we do.

And since block interval is what determines that percentage, adjusting it would adjust the waste as well.

[garlonicon]

And since block interval is what determines that percentage, adjusting it would adjust the waste as well.

But it is already adjusted on other networks. If you want to make it shorter, then note that in pools, miners submit their shares every 30 seconds or something like that. Also, they submit "invalid" blocks, in that sense the Proof of Work does not meet the network difficulty, but they are rewarded with some fractions of the coinbase.

Another thing is making the block time longer, for example one hour. For compression and Initial Block Download, I also thought about two weeks per "package". And for sidechains, it is proposed to be three months per on-chain update.

So, if you think that any work is "wasted", then note that in pools it is not. If miners are rewarded for "invalid" blocks (because of not meeting the difficulty), then I am pretty sure they are also rewarded for stale blocks.

[o_e_l_e_o]

And energy spent on a chain that is not the longest one difficulty-wise, is wasted energy.

This is true if it is external hash power coming from elsewhere and attempting to 51% attack the main chain, for example. But (and I could well be wrong) I don't think it is true if it is hash power which is already mining the main chain that temporarily breaks off to attempt to mine a fork before rejoining the main chain.

In my previous example where the hash rate splits evenly in two, then it doesn't matter that both halves are attempting to build on top of a different block. The total hash rate hasn't changed, and so the next block will still arrive in 10 minutes on average. If half of the network's work was truly wasted, then the next block would take 20 minutes to arrive. But because we haven't pre-determined which fork will win, then the work of both forks is contributing to the security of the network.

All the work put in the invalid chain is wasted work, because the same work could have been used to provide security.

But it did provide security. If the work on the stale chain had found a successful hash first, then it would be the main chain. Just as if the work on any of the failed candidate blocks had found a successful hash first, then that candidate block would be on the main chain. If we look at a pool like BTC.com for example - they have an estimated 3% of the hashrate, but haven't found a block in almost 100 blocks. Does that mean all their work on their now invalid candidate blocks was wasted?

I think the confusion here is stemming from our frame of reference. If you look at the chain in retrospect, then all the work that didn't find a block can be called "wasted". But looking forward, all the work which attempts to find a block, regardless of whether or not that block is accepted, is contributing to the difficulty of finding a block and therefore the security of the network.

[BlackHatCoiner]

So, if you think that any work is "wasted", then note that in pools it is not. If miners are rewarded for "invalid" blocks (because of not meeting the difficulty), then I am pretty sure they are also rewarded for stale blocks.

That doesn't mean it isn't waste. The administrator of a pool decides to do various things to have the business running. An attractive policy is to pay miners for blocks, whether these are stale or not. But, stale blocks don't create money. The money come from business' budget. Stale blocks consist an expense for the pool that doesn't contribute in any way. That's the definition of waste.

But looking forward, all the work which attempts to find a block, regardless of whether or not that block is accepted, is contributing to the difficulty of finding a block and therefore the security of the network.

It isn't. Difficulty determines the security of one chain. Stale blocks create a situation where we have two chains, therefore two potential difficulties. The security of the network is essentially split in half for a moment, and the next miner decides which part will be sacrificed.

I thought it is apparent enough that block interval comes with an opportunity cost. That cost defines the security that would have been provided if there was a greater block interval. But in the end, it's a tradeoff. Lower security for faster confirmations. Maybe it is technically wrong to call stale blocks waste, because you actually gain something-- be it time or security accordingly.

[tromp]

But looking forward, all the work which attempts to find a block, regardless of whether or not that block is accepted, is contributing to the difficulty of finding a block and therefore the security of the network.

I think everyone would agree with that, if we assume that the miner is honest.
I.e. they follow the longest chain rule in attempting to find a block that builds on the block with the most cumulative difficulty.

The remaining questions are:

1) If a new block has been found but not propagated to a specific miner yet, is that miner's attempt wasted?

2) If there are multiple new blocks found nearly simultaneously and both are being mined upon, are the attempts on top of the losing block wasted?

2) should be answered in the negative, since even with two tips at height h (and thus an undetermine height h winner), all the hashpower is still working on finding a block at height h+1,  and the expected time to find one is no longer than if there is a unique tip at height h.

But I would answer 1) in the positive...

[garlonicon]

That doesn't mean it isn't waste.

That exactly means it isn't waste, if you define "waste" as "not being paid for producing hashes". You have one lucky miner that submits a share, where the whole block is valid, and 6.25 BTC plus fees are collected. And you have a lot of miners, where each submitted shares below the target, stale, or those which were skipped, that way or another. But how rewards are truly splitted? This lucky miner is not getting 6.25 BTC plus fees. This miner just provided some reward, which is splitted between all miners, and this lucky miner could for example get only 0.01 BTC, because 6.24 BTC plus fees were splitted between other miners, despite none of them mined any accepted block.

The administrator of a pool decides to do various things to have the business running. An attractive policy is to pay miners for blocks, whether these are stale or not.

The whole reason why pools are needed at all, is that miners cannot collect 0.01 BTC on-chain for mining valid 7 BTC coinbase (6.25 BTC base + 0.75 BTC in fees), at 700 times lower difficulty. They have to use pools, because it is not something that is supported directly by the mining protocol.

So, if you think that another method of splitting rewards is better, then you can simply introduce that by forming your own pool. You can collect hashes, calculate chainwork for them, count the total chainwork in your pool, and then split rewards, based on that. There are many methods of rewarding miners. If you think that anything is "wasted", and your "chainwork-based method" is better, then call it Pay-Per-Chainwork (PPC? PPCW?), and form a pool.

As long as you have two (or more) blocks at the same height, and all of them are valid, you can locally pick another block, and work on it. The only thing you cannot change is forcing other nodes to think that your block is the only valid block, and make it a consensus rule. But when it comes to picking the block to work on, you can locally use any method you want, and stay in the same network. You will have the same chances of picking it wrong as other nodes, as long as you apply it only for a single block, and keep the actual chainwork rules for two or more blocks.

[o_e_l_e_o]

That doesn't mean it isn't waste.

Waste from the point of view of a mining pool is very different to waste from the point of view of the network. I'm sure mining pools probably do see stale blocks as waste. Every hash which does not earn a mining pool money is waste, regardless if it is because of a stale block or just the 99.999...% of hashes which are unsuccessful. But those hashes are not wasted from the network's point of view.

And as garlonicon pointed out, this is different again when considering miners instead of mining pools, since miners earn money for unsuccessful hashes too.

Stale blocks create a situation where we have two chains, therefore two potential difficulties.

The difficulty between both chains is the same, not different. And the combined hash rate across both chains for that difficulty will still mean the next block is found in 10 minutes (give or take the usual caveats).

tromp has put this very well I think. Despite the split at height H, all the hash rate on both sides of the split is working on the block at height H+1, just as it would be if there was no split.

But I would answer 1) in the positive...

In the situation you give in point 1), then yes, if a miner is attempting to mine on a chain which is not the main chain (as they would if they were not aware of the latest block), then that work is wasted. But in a chain split as being discussed here, we don't know which chain is the main chain yet, and so the work of both chains contributes to the security of the network.

[BlackHatCoiner]

That exactly means it isn't waste

Some people got paid, while others didn't, while both paid for a cost. The former are miners. The latter is the pool administrator. For the former it wasn't waste, but it was for the latter.

The difficulty between both chains is the same, not different.

It's the same value, but not the same in the sense that the chains are different. Anyway, that's just a play on words. We both understand the same.

2) If there are multiple new blocks found nearly simultaneously and both are being mined upon, are the attempts on top of the losing block wasted?

2) should be answered in the negative, since even with two tips at height h (and thus an undetermine height h winner), all the hashpower is still working on finding a block at height h+1,  and the expected time to find one is no longer than if there is a unique tip at height h.

We are still working at finding a block at H+1, but there has likely been more energy used than needed.

[o_e_l_e_o]

We are still working at finding a block at H+1, but there has likely been more energy used than needed.

On average, I don't think there has.

The difficulty is always the same (for this difficulty period). The difficulty does not change based on the previous block's hash. It does not matter if every miner is building on top of the same block (as usually happens), but it also does not matter if literally every miner in the world was trying to build on top of a block unique to them. The previous block hash makes absolutely no difference to the difficulty. It will (on average) require the exact same number of hashes and the exact same amount of energy to find the next block, regardless of the presence of a chain split.

Now when the next block is found, you could say that all the energy spent mining on top of the now stale block was not needed. But you could equally say that all the energy spent mining by the pools which did not find the next block was not needed.