Reducing block time variability

[Hal]

Bitcoin aims to produce blocks every ten minutes. But the actual time between blocks is quite variable, being governed by the Poisson distribution. I listen to block formation, and this variability is apparent. It's not unusual for blocks to be formed just a few seconds apart, while sometimes an hour goes by without one.

This variability is intrinsic to the hash solutions used by Bitcoin, and will not change even as the network grows. It may be a problem particularly for transactions that want a single confirmation. An application where a ten minute delay is acceptable might be in trouble if the delay becomes an hour.

A way to reduce the variability while maintaining the average solution time is to split the hash problem up into several sub-problems. Instead of solving a hash puzzle with difficulty 16,000, solve 4 puzzles with difficulty 4,000. The total difficulty and average time to find a solution is the same, but the variability is much less. Splitting into even more sub-problems will further reduce variability.

A downside to reducing variability is that we would have more block collisions, where more than one node solved the same block at about the same time. This would reduce the efficiency of the network, as nodes work on blocks doomed to be superseded, and blocks get reshuffled in and out of the longest chain. Also, it would give more advantage to the fastest node; in the extreme, if we eliminated all variability, it would win every time.

So we wouldn't want to go too far with this. But splitting into a modest number of sub-problems could substantially reduce the odds of hour+ inter-block intervals, hopefully without causing too many problems.

[1714944760]

1714944760

[1714944760]

1714944760

[1714944760]

1714944760

[FreeMoney]

It will change!

Right now when a block is solved there is another 50BTC reward waiting to be claimed immediately; this is a little silly because there are no transactions to be processed yet.

In the far future this will not be the case. There will be no per block reward. Fees will be the entire incentive. So then when a block is found, everyone stops hashing. They wait until the cost of hashing is less than the expected value. This will be different for different operations, but as fees pile up more and more generation will turn on.

The longer a block goes without being solved the more power is devoted to it. This has several effects. It reduces variance and also means that during periods of high use blocks will come more frequently. Both are positive effects.

These effects will emerge gradually as fees become more important and the block reward less so. Probably it will be very pronounced in 10-18 years. In 18 years the per block reward will be only 1.5625, fees will probably be larger. I would guess that in some places the cost of electricity would make it not worth mining for less 2BTC, but very worth it if the fees pushed the value of a block over 3BTC.

[davout]

It will change!

Right now when a block is solved there is another 50BTC reward waiting to be claimed immediately; this is a little silly because there are no transactions to be processed yet.

In the far future this will not be the case. There will be no per block reward. Fees will be the entire incentive. So then when a block is found, everyone stops hashing. They wait until the cost of hashing is less than the expected value. This will be different for different operations, but as fees pile up more and more generation will turn on.

The longer a block goes without being solved the more power is devoted to it. This has several effects. It reduces variance and also means that during periods of high use blocks will come more frequently. Both are positive effects.

These effects will emerge gradually as fees become more important and the block reward less so. Probably it will be very pronounced in 10-18 years. In 18 years the per block reward will be only 1.5625, fees will probably be larger. I would guess that in some places the cost of electricity would make it not worth mining for less 2BTC, but very worth it if the fees pushed the value of a block over 3BTC.

This just blew my mind.

[ribuck]

Right now when a block is solved there is another 50BTC reward waiting to be claimed immediately; this is a little silly because there are no transactions to be processed yet.

There is still some value solving a block with no transactions, because it increases the security against double-spending of transactions in previous blocks.

I think your analysis of future mining is spot-on. When transaction fees become the dominant source of income for generators, variance in block times will be greatly reduced due to economic forces.

[FreeMoney]

Right now when a block is solved there is another 50BTC reward waiting to be claimed immediately; this is a little silly because there are no transactions to be processed yet.

There is still some value solving a block with no transactions, because it increases the security against double-spending of transactions in previous blocks.

Yep, that's why I didn't say 'pointless', 'silly' was even too strong.

[ptd]

It will change!

Right now when a block is solved there is another 50BTC reward waiting to be claimed immediately; this is a little silly because there are no transactions to be processed yet.

In the far future this will not be the case. There will be no per block reward. Fees will be the entire incentive. So then when a block is found, everyone stops hashing. They wait until the cost of hashing is less than the expected value. This will be different for different operations, but as fees pile up more and more generation will turn on.

The longer a block goes without being solved the more power is devoted to it. This has several effects. It reduces variance and also means that during periods of high use blocks will come more frequently. Both are positive effects.

These effects will emerge gradually as fees become more important and the block reward less so. Probably it will be very pronounced in 10-18 years. In 18 years the per block reward will be only 1.5625, fees will probably be larger. I would guess that in some places the cost of electricity would make it not worth mining for less 2BTC, but very worth it if the fees pushed the value of a block over 3BTC.

Well spotted! This shows that humans in their continual attempts to make more money, can't avoid being helpful.

[Hal]

Interesting analysis! Don't forget that in several years, if Bitcoin succeeds, there will be orders of magnitude more transactions. And some of these will be able to pay substantial fees. So there will be a strategy to start generating right away, beat the miners who are waiting, and collect these high value tx fees.

Also, currently miners don't update the block they're working on with new transactions as they come in. I think that makes the hash search more efficient. So by the time a block is generated, there's already a substantial pool of new transactions to start working on. This too may change in the future though.

[davout]

Interesting analysis! Don't forget that in several years, if Bitcoin succeeds, there will be orders of magnitude more transactions. And some of these will be able to pay substantial fees. So there will be a strategy to start generating right away, beat the miners who are waiting, and collect these high value tx fees.

Doesn't work like this, you have to know the transactions you're processing *before* hashing.
You can't find a winning hash, wait for a tx fee and then release, including th tx in our block will invalidate the merkle root and therefore the whole block.

Also, currently miners don't update the block they're working on with new transactions as they come in. I think that makes the hash search more efficient. So by the time a block is generated, there's already a substantial pool of new transactions to start working on. This too may change in the future though.

Are you sure ?
Adding a transaction is extremely cheap.

[tcatm]

Also, currently miners don't update the block they're working on with new transactions as they come in. I think that makes the hash search more efficient. So by the time a block is generated, there's already a substantial pool of new transactions to start working on. This too may change in the future though.

The miner in bitcoind itself and getwork based miners update the TX regularly. After all, the whole point of mining is to confirm transactions.

[Hal]

Ah, okay,  I didn't know that miners worked that way,  thanks.   

[sandos]

Reducing variability likely increases the chances of several miners finding solutions at the same time!

Which might be bad.