Why does the testnet difficulty permanently drops to around 1 periodically?

[theartlav]

I've noticed a surge of testnet blocks over the last few days, and apparently the difficulty permanently got reset to 1 around block 000000000002949f844e92645df73ce9c093e5aac0d962a0fa13eb076eec835c

I know it should go to 1 temporarily if there are no blocks mined for 20 minutes, but i wasn't aware of it being able to go to 1 permanently like that.

Further looking back reveals that this happens periodically. I.e. the previous one was at block 000000000007682725067b780feccae8143ae2a2b771639630f4d5b44b8548c1.

Is this a specific rule on the testnet - difficulty getting reset to 1 every now and then?

Or is it some sort of an effect from too many 20-minutes-late 1.0 blocks? The latter shouldn't be possible, since difficulty can't change by more than a factor of 4 at a time.

[1715004922]

1715004922

[1715004922]

1715004922

[1715004922]

1715004922

[1715004922]

1715004922

[1715004922]

1715004922

[1715004922]

1715004922

[theartlav]

Actually, it's quite periodic. Made a graph of testnet difficulty, below.
So, what rule am i missing here?

https://i.imgur.com/UDePepN.jpg

[cr1776]

I've noticed a surge of testnet blocks over the last few days, and apparently the difficulty permanently got reset to 1 around block 000000000002949f844e92645df73ce9c093e5aac0d962a0fa13eb076eec835c

I know it should go to 1 temporarily if there are no blocks mined for 20 minutes, but i wasn't aware of it being able to go to 1 permanently like that.

Further looking back reveals that this happens periodically. I.e. the previous one was at block 000000000007682725067b780feccae8143ae2a2b771639630f4d5b44b8548c1.

Is this a specific rule on the testnet - difficulty getting reset to 1 every now and then?

Or is it some sort of an effect from too many 20-minutes-late 1.0 blocks? The latter shouldn't be possible, since difficulty can't change by more than a factor of 4 at a time.

It is probably this, as you alluded to above:
"Minimum difficulty of 1.0 on testnet is equal to difficulty of 0.5 on mainnet. This means that the mainnet-equivalent of any testnet difficulty is half the testnet difficulty. In addition, if no block has been found in 20 minutes, the difficulty automatically resets back to the minimum for a single block, after which it returns to its previous value."

So it drops, then returns.

[theartlav]

So it drops, then returns.

No, that's not it.

The important part is "resets back to the minimum for a single block, after which it returns to its previous value".
There are plenty of such jumps - going between millions then one then millions over one block.
According to the code, the rule is that if the time of new block is over 20 minutes past the time of the previous block, then the difficulty can be set to one for this block and only this block.

What we are seeing on the graph is difficulty being permanently dropped, not just for one block, but for the whole next period, and then goes up slowly in steps of 2016. That is a completely different rule.

[achow101]

So it drops, then returns.

No, that's not it.

The important part is "resets back to the minimum for a single block, after which it returns to its previous value".
There are plenty of such jumps - going between millions then one then millions over one block.
According to the code, the rule is that if the time of new block is over 20 minutes past the time of the previous block, then the difficulty can be set to one for this block and only this block.

What we are seeing on the graph is difficulty being permanently dropped, not just for one block, but for the whole next period, and then goes up slowly in steps of 2016. That is a completely different rule.

The difficulty will reset to 1 if the time since the last block is more than 20 minutes. There is no stipulation that after a difficulty reset block that the next block must be the normal difficulty; if the next block is more than 20 minutes after the current block, then it can also have a difficulty of 1.

For blocks that are found within 20 minutes of each other, the block's difficulty will be the same as the difficulty of the last block in the difficulty interval whose difficulty was not 1 OR the difficulty of the first block in the difficulty interval. This behavior is defined here: https://github.com/bitcoin/bitcoin/blob/master/src/pow.cpp#L32.

The way that the difficulty retarget works is that, at the beginning of the new difficulty interval, the difficulty of the first block in that interval takes the difficulty of the last block in that interval and multiplies that by the time it took to mine the 2016 blocks and then divides it by the target time. The result is then clamped to be at least 1. Since this is based upon the difficulty of the last block in the previous interval, if that block is difficulty 1, then the next interval will also have a difficulty of one.

So what we are seeing here is that the last block in the interval is found 20 minutes after the block before it so it has a difficulty of one. Because the next block adjust the difficulty and it only looks at the block before it (which is difficulty 1), the difficulty of the next interval is 1. So the next 2016 blocks are mined at difficulty 1, and the difficulty then slowly adjusts up again.

Edit:

I was slightly wrong. Fixed that.

[cr1776]

So it drops, then returns.

No, that's not it.

The important part is "resets back to the minimum for a single block, after which it returns to its previous value".
There are plenty of such jumps - going between millions then one then millions over one block.
According to the code, the rule is that if the time of new block is over 20 minutes past the time of the previous block, then the difficulty can be set to one for this block and only this block.

What we are seeing on the graph is difficulty being permanently dropped, not just for one block, but for the whole next period, and then goes up slowly in steps of 2016. That is a completely different rule.

Ah yes, now that I look at the graph on a larger screen, I can see it better.  ;-). Perhaps they are time-warping it to reset it with two 1 difficulty blocks in a row and then it it stepping back up every difficulty change?  I can't tell for sure from that graph.  

[nathan_24]

The difficulty will reset to 1 if the time since the last block is more than 20 minutes. There is no stipulation that after a difficulty reset block that the next block must be the normal difficulty; if the next block is more than 20 minutes after the current block, then it can also have a difficulty of 1.

For blocks that are found within 20 minutes of each other, the block's difficulty will be the same as the difficulty of the last block in the difficulty interval whose difficulty was not 1 OR the difficulty of the first block in the difficulty interval. This behavior is defined here: https://github.com/bitcoin/bitcoin/blob/master/src/pow.cpp#L32. Because the difficulty look back is clamped to be the first block in the interval, if that block was found more than 20 minutes after the block before it, its difficulty will be 1, and so will all subsequent blocks in the difficulty period. After 2016 blocks, the difficulty adjustment will kick in and bring the difficulty back up.

So the behavior we are observing is that the first block of a difficulty interval (block height % 2016 == 0) is found 20 minutes or more after the previous block, so its difficulty is 1. Then blocks that are mined after it use the difficulty of 1 because that is the difficulty of the first block in the interval.

As far as I know, the difficulty of a block multiple of 2016 doesn't change if it takes more than 20 minutes to process. Such blocks are the only ones not affected by that rule in the testnet.

The difficulty of such blocks is calculated depending on the difficulty of the block immediately prior to them. If that block took more than 20 minutes, then this block could have the difficulty set to 1 as well, and so would the blocks mined after this one.

I said "could" because, if the previous 2,016 blocks took less than 2 weeks to process, then the difficulty could be slightly greater than 1.

[achow101]

As far as I know, the difficulty of a block multiple of 2016 doesn't change if it takes more than 20 minutes to process. Such blocks are the only ones not affected by that rule in the testnet.

The difficulty of such blocks is calculated depending on the difficulty of the block immediately prior to them. If that block took more than 20 minutes, then this block could have the difficulty set to 1 as well, and so would the blocks mined after this one.

I said "could" because, if the previous 2,016 blocks took less than 2 weeks to process, then the difficulty could be slightly greater than 1.

It's actually because the block at the difficulty adjustment only looks at the difficulty of the block before it since we assume (for mainnet) that all blocks in an interval are all the same difficulty. However on testnet the difficulty can be one, so if the last block in the previous interval has a difficulty of 1, then the next interval will do that too. I have edited my post to reflect this.

[theartlav]

It's actually because the block at the difficulty adjustment only looks at the difficulty of the block before it since we assume (for mainnet) that all blocks in an interval are all the same difficulty.

Huh, that's a neat example of unintended side effects.

Thanks all for clarifications.