Mitigating Miner Penalty for Large Blocks by Reducing Propagation Penalty by 600

[foolish_austrian]

Below is a proposal for reducing the penalty miners pay for larger block size by [I would argue] at least 6000%...

When miners produce simultaneous blocks at the end of the block chain, each peer individually chooses which block to trust. (In the absence of other considerations… peers usually trust the first block they see.)

This leads to the condition where miners are incentivized to produce the smallest block size (minimum number of transactions) to minimize propagation of their blocks.

Is it possible to mitigate this effects by establishing the rule that, when the block chain forks and there are 2 blocks at the same block height, miners always choose the block with the lowest hash (both blocks would obviously have hash below the target).

In the event that 2 blocks are solved nearly simultaneously, the outcome of which block is accepted by the majority of other miners will be deterministic based on their headers, not on network propagation. From the perspective of the miner who just found a solution, they needn’t worry about the size of their block or how quickly it propagates through the network. They simply hope it’s header is less than any other possible competing block.

Could this lead to miners continuing to mine at a given block height, even if a solution has been broadcast? No, because the moment they become aware of a solution, their effective target has just been raised. If miners collectively agree to accept only the lowest hash, there is no incentive to attempt to compete below the highest known block height.

In the event that 2 blocks (a1 and a2) are solved at height n and n+1 respectively, before block b1 fully propagates, it raises the potential conflict if b1 would have orphaned a1. This would be the case if the hash of b1<a1. However, if a2 was found before b1 fully propagates through the network, then b1 will be orphaned due to propagation delays [which I’m trying to mitigate]. However, the chances of b1 being orphaned due to a2 are at least 60 times less likely than being orphaned due to a propagation race with a1, as calculated below.

To calculated the expected reduction in block orphanage due to network propagation, let’s assume 20s propagation time with a block solution following an exponential distribution with a mean time of 300sec (5 min) [note: this is towards a worst case scenario].

Integration the exponential PDF from 290sec to 310sec results in an orphan probability of 2.45%, this would correspond to a propagation time of 20sec while a competitors propagation time is 0sec, clearly an unacceptable disadvantage for miners.

However, let’s calculate the probability of the above scenario, where b1 and a1 are found simultaneously, but a2 is found while both are still propagating through the network. For sake of simplicity, let’s assume that a1 has negligible propagation time. With the whole network working on a2, the probability of a2 being solved in the first 20sec is 6.45%. Therefore the probability of both of these events occurring simultaneously is 0.158%.

To further refine this, let’s assume that block a1 actually propagates through the network at a linear speed, so only part of the network is working on a solution to extend a1 [the linear propagation rate is probably a bad assumption, I’m just schmooing lambda of the exponential distribution, but this would overestimate the probability of a2 being found]. Using linear propagation to half the network, then being overtaken by b1, the probability of a2 being solved before b1 fully propagates is 1.65%, which results in an orphan rate for b1 of less than 0.05%.

This would mitigate the effect of block size on orphanage by 6000%

(I’m a physicist, not a computer scientist, I hope this makes sense, and furthermore I hope this is a viable solution to the tx/sec limit being imposed by miners!)…

[gmaxwell]

Is it possible to mitigate this effects by establishing the rule that, when the block chain forks and there are 2 blocks at the same block height, miners always choose the block with the lowest hash (both blocks would obviously have hash below the target).

This is an oft repeated suggestion going back to at least 2011.  It opens a trivial attack where when you do get a very low hash you refrain from announcing it, comfortable in the fact that you're guaranteed to win a race should one arise. In doing so, your competition is all wasting their time mining along a path that is likely to lose. The effect is stronger the larger you are, creating an increase in expectation for larger miners that doesn't otherwise exist.

[foolish_austrian]

Is it possible to mitigate this effects by establishing the rule that, when the block chain forks and there are 2 blocks at the same block height, miners always choose the block with the lowest hash (both blocks would obviously have hash below the target).

This is an oft repeated suggestion going back to at least 2011.  It opens a trivial attack where when you do get a very low hash you refrain from announcing it, comfortable in the fact that you're guaranteed to win a race should one arise. In doing so, your competition is all wasting their time mining along a path that is likely to lose. The effect is stronger the larger you are, creating an increase in expectation for larger miners that doesn't otherwise exist.

Forgive me if this has already been discussed, but I believe [humbly] that you are appealing to the Monte Carlo fallacy (i.e. that probabilities are cumulative).

You are correct that a miner could withhold a block solution, but it would be of no advantage. I make this claim based on the assertion that mining is not deterministic, even if a miner set out with a given nonce increment pattern. I'm asserting that mining is only probabilistic, because until a solution is found, there is no guarantee that the solution exists, only a probability of it existing.

I think I could make a convincing argument for why the above scenario would be of no advantage, but this hinges on my understanding that mining is not deterministic. The basic argument goes that even if a selfish miner hashes x number of hashes secretly, he is still no closer to a solution than any other miner, therefore there is no advantage to the selfish miner if he excludes other miners for a period of time.

If this has been discussed, I'm be interested to read up on previous discussions. Is there consensus that mining is deterministic or probabilistic (i.e. do probabilities mature or are they independent)?

Again, I think I could put together a good argument, but I don't want to re-hash someone else's work!

[foolish_austrian]

By the way, this is the mind bending implications of statistics... I had to work through this myself to convince myself of the security implications of block frequencies (i.e. Litecoin vs. Bitcoin)...

I convinced myself quite strongly that mining is never deterministic, which has this tensions between 'gut feeling' and reality. At first, it made sense to me that once a miner begins mining with a set nonce pattern, the number of hashes to a solution becomes deterministic... however, after further reflection this is not the case.

A miner can hash x hashes, but is absolutely no hashes closer to a solution....

This is why a miner withholding a block gains no advantage.

[shorena]

Is it possible to mitigate this effects by establishing the rule that, when the block chain forks and there are 2 blocks at the same block height, miners always choose the block with the lowest hash (both blocks would obviously have hash below the target).

This is an oft repeated suggestion going back to at least 2011.  It opens a trivial attack where when you do get a very low hash you refrain from announcing it, comfortable in the fact that you're guaranteed to win a race should one arise. In doing so, your competition is all wasting their time mining along a path that is likely to lose. The effect is stronger the larger you are, creating an increase in expectation for larger miners that doesn't otherwise exist.

Forgive me if this has already been discussed, but I believe [humbly] that you are appealing to the Monte Carlo fallacy (i.e. that probabilities are cumulative).

a very low hash as in "a solution with a hash value below traget"

You are correct that a miner could withhold a block solution, but it would be of no advantage.

Ofc, it would an advantage. The following block must reference the block the attacking miner is withholding. Thus the attacking miner has more time to find the next block, while everyone else is still working on an block that has allready been found, but not yet broadcasted.

I make this claim based on the assertion that mining is not deterministic, even if a miner set out with a given nonce increment pattern. I'm asserting that mining is only probabilistic, because until a solution is found, there is no guarantee that the solution exists, only a probability of it existing.

That is true, but if I allready tried 10% of all possible solution my chance to find the solution is higher than yours (who just started).
Also there is allways a solution just not allways for any given set of transactions. But since miners can choose the transactions as well as the nounce, there is allways a possible solution.

I think I could make a convincing argument for why the above scenario would be of no advantage, but this hinges on my understanding that mining is not deterministic. The basic argument goes that even if a selfish miner hashes x number of hashes secretly, he is still no closer to a solution than any other miner, therefore there is no advantage to the selfish miner if he excludes other miners for a period of time.

Here is the missunderstanding I pointed out earlier. "Low hash" means "found a solution" in this case. Just a random hash that is no solution is not helping.

By the way, this is the mind bending implications of statistics... I had to work through this myself to convince myself of the security implications of block frequencies (i.e. Litecoin vs. Bitcoin)...

I convinced myself quite strongly that mining is never deterministic, which has this tensions between 'gut feeling' and reality. At first, it made sense to me that once a miner begins mining with a set nonce pattern, the number of hashes to a solution becomes deterministic... however, after further reflection this is not the case.

A miner can hash x hashes, but is absolutely no hashes closer to a solution....

This is correct.

This is why a miner withholding a block gains no advantage.

This, however, is not. Withholding a block that you know will be accepted by the network because it has a very high priority (lower hash value) gives you the advantage to work on the next block before everyone else.

[gmaxwell]

Forgive me if this has already been discussed, but I believe [humbly] that you are appealing to the Monte Carlo fallacy (i.e. that probabilities are cumulative).

You are correct that a miner could withhold a block solution, but it would be of no advantage. I make this claim based on the assertion that mining is not deterministic, even if a miner set out with a given nonce increment pattern. I'm asserting that mining is only probabilistic, because until a solution is found, there is no guarantee that the solution exists, only a probability of it existing.

I'm not, but I've dealt with (literally) hundreds of people who were, so I can forgive you for assuming. I've found that explaining that mining is "progress free", like throwing dice helps people understand the quickest when they suffer that particular confusion.

For simplicity, Imagine you, and I are mining in perfect competition with equal hashrate, and there is another 20% held by others.

I find a block with a very low value— very likely to win under your scheme.  Instead of announcing it I keep it for myself and I mine the successor.  If you find a block I will release mine immediately and win.

Sometimes I find a second block before anyone else does. Sometimes I'll even find a third.  I can begin announcing my older blocks only when there is a non-trivial risk that the network would over take me with the next block it finds.  In this way, even though I do not have a majority hashpower I can exclude large chunks of other miners blocks from the network by having an information advantage over the chain I'm keeping private.

Mining is progress-free, but the blockchain as a whole is potentially not.  Preferring the first seen block creates a strong incentive to announce as fast as possible and helps the whole system behave in a progress free manner.

[foolish_austrian]

Ok, so I think I understand. It lowers the barrier to entry for a block withholding attack...

My point was that there is no such thing is 'working' on a block, it is either solved or not solved. In the case where the greedy miner is forced to reveal the block before finding a second block secretly, there is no benefit. It would only have been realized as a benefit if he finds 2 consecutive blocks before the network forces him to reveal the first block.

3 questions:

1) Has anyone calculated the window for execution of such an attack? Either empirically, numerically, or both? My suspicion is that it's actually quite a bit smaller than most people would anticipate. (but I could be wrong!)

2) If the effect is small and doesn't present appreciable bias towards larger pools, would allowing block withholding be something that bitcoin could just be agnostic too?... as long as it doesn't effect the overall network, it's like allowing miners a little gambling with their blocks [again assuming the overall affect on the network is small and doesn't provide advantage to large pools]. My instinct is that it would have almost negligible effects on block frequency, and that if all miners played the same game, it would have almost no bias.

3) Is this something that's simply politically unworkable [assuming that the final decision is really up to the miners]? Even if the real benefit of withholding a block was smaller than anticipated, is this something miners would simply be too resistant to?

I'd be willing to find analytically/numerical solutions to how a potential attack could be executed, and quantify its potential effect on the overall network, but if this idea has already been put to rest...