How does the difficulty adjust so precisely?

[TheBitcoineers]

How is the mining difficulty able to adjust so finely? I would think that if only the target is changing (the number of zeros at the beginning of the block header hash) the difficulty would only increase by powers of two. So how can it adjust so precisely? Can the target be a block hash with a value less than 00000a instead of just 00000? Can something other than zeroes be specified in the target? Thanks.

[BrewMaster]

target is not number of zeros, it is simply a maximum 256 bit number that the header hashes have to be lower than it. but since it is padded to make it 32 bytes we end up seeing the zeros.

it changes based on the block timestamps. when 2016 blocks are found the time difference between the last block and the block that is 2015 blocks deep is calculated and the new target is computed by multiplying it by this time and dividing it by 2 weeks as seconds.

[BlackHatCoiner]

As @BrewMaster wrote, target is a 256-bit number.

it changes based on the block timestamps. when 2016 blocks are found the time difference between the last block and the block that is 2015 blocks deep is calculated and the new target is computed by multiplying it by this time and dividing it by 2 weeks as seconds.

And to answer that with maths, this is how it's calculated:

Code:

difficulty = expected time / actual time

Expected time is 2016 * 10 minutes = 20160 minutes. If we assume that the 2016 blocks after the difficulty adjustment are mined every 9 minutes then the new difficulty will be:

Code:

difficulty = 20160 / 18144 = 1.11

You get the idea.

A nice link

[odolvlobo]

target is not number of zeros, it is simply a maximum 256 bit number that the header hashes have to be lower than it. but since it is padded to make it 32 bytes we end up seeing the zeros.

Yes. The white paper talks about counting zero bits, but Bitcoin was never implemented that way.

[BrewMaster]

target is not number of zeros, it is simply a maximum 256 bit number that the header hashes have to be lower than it. but since it is padded to make it 32 bytes we end up seeing the zeros.

Yes. The white paper talks about counting zero bits, but Bitcoin was never implemented that way.

usually when people talk about "counting zeros" they are referring to the number of zeros in the hexadecimal hash not in the binary. counting zero bits is very similar to comparing two integers.
for example 0000_0111 is smaller than 0000_1111 because it has 5 zero bits while the second one has 4 while both of them have 1 zero in their hex encoding (07 and 0f).

[odolvlobo]

target is not number of zeros, it is simply a maximum 256 bit number that the header hashes have to be lower than it. but since it is padded to make it 32 bytes we end up seeing the zeros.

Yes. The white paper talks about counting zero bits, but Bitcoin was never implemented that way.

usually when people talk about "counting zeros" they are referring to the number of zeros in the hexadecimal hash not in the binary. counting zero bits is very similar to comparing two integers.
for example 0000_0111 is smaller than 0000_1111 because it has 5 zero bits while the second one has 4 while both of them have 1 zero in their hex encoding (07 and 0f).

0000_1110 is smaller than 0000_1111, even though they have the same number of leading 0s. As the OP pointed out, counting leading 0s only detects changes by a factor of 2. Simply comparing the values arithmetically has a much practical level of precision as long as the numbers remain large.

[thecodebear]

So what is the actual difficulty adjustment then. In specific terms. Cuz the way it's always been explained anytime I've read about it is like the OP is saying, you add one more leading zero as a requirement to the hash to get an increased difficulty, which would mean there would be a very limited number of difficulty levels, which is obviously not the case.

So is the adjustment, I guess basically what amounts to a max value the hash can be, more granular than just adding another zero? Like instead of an increased difficulty changing it needing to start with "0000" to "00000" it goes from "0000" to "00005" or whatever on an increased difficulty adjustment? I feel like this wasn't actually answered yet in this discussion. And how does the calculated difficulty (expected time / actual time) play into this...I'm assuming that difficulty adjustment factor is just multiplied to the previous max allowable hash value to get an increase or decrease in difficulty. The way this is always described - "add a zero to increase difficulty" - clearly isn't the way it actual works so if someone could clear this up that'd be great.

[ranochigo]

So is the adjustment, I guess basically what amounts to a max value the hash can be, more granular than just adding another zero? Like instead of an increased difficulty changing it needing to start with "0000" to "00000" it goes from "0000" to "00005" or whatever on an increased difficulty adjustment? I feel like this wasn't actually answered yet in this discussion. And how does the calculated difficulty (expected time / actual time) play into this...I'm assuming that difficulty adjustment factor is just multiplied to the previous max allowable hash value to get an increase or decrease in difficulty. The way this is always described - "add a zero to increase difficulty" - clearly isn't the way it actual works so if someone could clear this up that'd be great.

Difficulty is just a representation of the target. Each block header has a target and it is represent in bits. The current target is 170da8a1 or 0000000000000000000da8a10000000000000000000000000000000000000000 in hexadecimal. For a block to be valid, the hash of the block header must be lower than or equals to the current target. For the current difficulty, you'll just use the maximum target/current target which will result in 00000000FFFF0000000000000000000000000000000000000000000000000000 / 0000000000000000000da8a10000000000000000000000000000000000000000 , equals to approximately 20607418304385.63 which is the current difficulty.

Now, since difficulty is a function of target, you can do it in reverse to obtain the next difficulty. When calculating the next difficulty, you'll be taking the expected time/actual time of the last 2015 blocks. Let's take for example the previous difficulty adjustment which was on 9 Jan 2021. The average block time was about 9 minutes and 2 seconds. The expected difficulty increment would be 600/542 which is roughly 10.7% increment. If you were to convert the new target from the new difficulty (18,599,593,048,299 x ~1.107), you would be using 00000000FFFF0000000000000000000000000000000000000000000000000000 / 20607418304385.63 which would give you a target of 0000000000000000000da8a10000000000000000000000000000000000000000.

You can actually just calculate the target without a conversion to the difficulty but I wanted to show how they're linked. Given that the network attempts to keep the block interval at an average of 10 minutes, the difficulty adjustment is based on the deviation from the expected time and uses it to calculate the difficulty increase, or the decrease in the target. They have an inverse relationship.

[pooya87]

you'll be taking the expected time/actual time of the last 2015 blocks.

Are you sure it is 2015? I've seen this elsewhere too but the code is saying something else.
https://github.com/bitcoin/bitcoin/blob/c7ad94428ab6f54661d7a5441e1fdd0ebf034903/src/pow.cpp#L13-L47

Code:

    if ((pindexLast->nHeight+1) % params.DifficultyAdjustmentInterval() != 0)
    {
        // if nHeight is something like 2015 this branch won't execute
        return....
    }

    // Assuming nHeight is 2015, nHeightFirst is going to be 0
    int nHeightFirst = pindexLast->nHeight - (params.DifficultyAdjustmentInterval()-1);
    // I believe GetAncestor returns the block at given height (can't find the code for it)
    const CBlockIndex* pindexFirst = pindexLast->GetAncestor(nHeightFirst);

    // That means new target is calculated for blocks 0 to 2015 => 2016 blocks
    return CalculateNextWorkRequired(pindexLast, pindexFirst->GetBlockTime(), params);

I added the comments on the code above, am I wrong that we are computing it from 2016 blocks not 2015?

[ranochigo]

Are you sure it is 2015? I've seen this elsewhere too but the code is saying something else.
-snip-

The test code is here[1], I'm not very well versed in cpp but from my limited understanding, the underlying logic is to calculate the time taken to generate the previous 2015 blocks (and by extension the time interval within the 2015 blocks) because the time taken to generate the #0 block is not accounted for or else the code would overlap into the block prior to the #0 block to also account for the time taken for it. Please do correct me if I'm wrong, but that's what I understood when I skimmed through it previously. It's the off-by-one error, after a brief search and I can't find any remedy done for it.


[1] https://github.com/bitcoin/bitcoin/blob/4a540683ec40393d6369da1a9e02e45614db936d/src/test/pow_tests.cpp#L14

[pooya87]

the underlying logic is to calculate the time taken to generate the previous 2015 blocks (and by extension the time interval within the 2015 blocks)

That's what everyone is saying but then I look at the code and it makes no sense.

https://github.com/bitcoin/bitcoin/blob/4a540683ec40393d6369da1a9e02e45614db936d/src/test/pow_tests.cpp#L14

It is using blocks 30240 to 32255 inclusive, which are 2016 blocks (32255-30240+1=2016).

[ranochigo]

It is using blocks 30240 to 32255 inclusive, which are 2016 blocks (32255-30240+1=2016).

Why do you include 30240 though? The time taken for that block can't be considered because it only takes into account the timestamp that that block has but not the block prior to that to give the code a representation of the time to generate that specific block, and thus wouldn't actually include the time taken for that block right? Am I missing something?

If we're going by the logic to calculate time intervals of 2016 blocks and if we want to consider block 30240, wouldn't we include 30239 as well?

[pooya87]

It is using blocks 30240 to 32255 inclusive, which are 2016 blocks (32255-30240+1=2016).

Why do you include 30240 though? The time taken for that block can't be considered because it only takes into account the timestamp that that block has but not the block prior to that to give the code a representation of the time to generate that specific block, and thus wouldn't actually include the time taken for that block right? Am I missing something?

If we're going by the logic to calculate time intervals of 2016 blocks and if we want to consider block 30240, wouldn't we include 30239 as well?

You are absolutely right. I was focusing too much on block time, instead of what the time represents. If we wanted to compute the "time it takes to mine 2016 blocks" we have to take the 30239 block time instead of 30240.

[odolvlobo]

The fact that the difficulty is based on the time for 2015 blocks rather than 2016 has been a long-standing bug.  The result is that the expected block time is actually 600.3 seconds.

[DannyHamilton]

You are absolutely right. I was focusing too much on block time, instead of what the time represents. If we wanted to compute the "time it takes to mine 2016 blocks" we have to take the 30239 block time instead of 30240.

Don't feel too bad about overlooking that.  Satoshi did too. 

[acquafredda]

The fact that the difficulty is based on the time for 2015 blocks rather than 2016 has been a long-standing bug.  The result is that the expected block time is actually 600.3 seconds.

We never stop learning in the bitcoin world: I remember I answered this question many times and I was always very convinced about my explanations to the fact that the bitcoin difficulty adjustment happened every 2016 blocks! Up to now...
Is there a specific reason why that bug never got sorted out? Is there something problematic about it? I guess if nothing happened yet, we can live happy going forward.

[BrewMaster]

I was always very convinced about my explanations to the fact that the bitcoin difficulty adjustment happened every 2016 blocks! Up to now...

difficulty DOES adjust every 2016 blocks. the thing is that we only use the time that took to mine the past 2015 blocks instead of the 2016.

Is there a specific reason why that bug never got sorted out? Is there something problematic about it?

possibly because it needs a hard fork and the solving it doesn't really seem to be that big a deal. time of 2015 blocks is also good if slightly inaccurate.

[acquafredda]

I see.
If things are working nobody would ever want to even think about a hard fork only to solve such a small inaccuracy. It would be like to change a clock only because it loses a few milliseconds every two weeks which would be obviously pointless. Thanks for the hint.