Understanding Nonce

[promise444c5]

My post isn't about  teaching rather  me trying to understand how the "Nonce" works

Well, in the previous weeks I was getting my hands on understanding how minning nodes  works so I went a little bit deeper in minning.

There I learned about nonce beign added to header of every block along with timestamps and other related data for the transactions block which is completely hashed for minners to validate.

I also learned that nonce is in a form of 32-bit and its always generate randomly
Now here's where I got confused, Miners have to guess this number to find a compatibility hash for the header.....how do they get to change the noce in the header since its hashed along with the block-header-dat thought they have different  hash . I'm just completely confused here because the whole guessing of a thing seem like magic to me or does the hardwares get to do it all??

All corrections to my statements are welcome

[Frankolala]

The nounce is a number that is used by miners to help create different hash result, so that they can get the hash of their block header that will be below the target by luck. If you have not gotten the number below the target, you can keep adjusting the nonce field in the block header by increasing it.

You can read this article. https://learnmeabitcoin.com/technical/nonce

[odolvlobo]

Now here's where I got confused, Miners have to guess this number to find a compatibility hash for the header.....

Not just the nonce. Although that is how it is typically described, other data in the header is also manipulated in order to change the hash.

how do they get to change the noce in the header since its hashed along with the block-header-dat thought they have different  hash

The value of the nonce is completely up to the miner and does not depend on anything else. So, if one value doesn't work, then another is tried. Typically, the nonce starts a 0 and it is incremented until a qualifying hash is found or all the possible nonces have been tried. If all the nonces have been tried, then something else in the header is changed and the nonce starts back at 0. This process keeps going until a block header that has a qualifying hash is found.

[promise444c5]

 Typically, the nonce starts a 0 and it is incremented until a qualifying hash is found or all the possible nonces have been tried. If all the nonces have been tried, then something else in the header is changed and the nonce starts back at 0. This process keeps going until a block header that has a qualifying hash is found.

Alright, from your statement you said if all nonces have been tried .. does this means they have a specific range value or a max value??

[nc50lc]

Alright, from your statement you said if all nonces have been tried .. does this means they have a specific range value or a max value??

Yes, as you said in the OP:

I also learned that nonce is in a form of 32-bit and its always generate randomly

32-bit has a maximum value of 2^32 or 4,294,967,296, so the range is from 0~4,294,967,295.
It has a relatively low max value compared to the average ASIC's hashrate but miners rely on "extra nonce" (in the coinbase transaction) or any other data in the block header so that they can start the nonce from 0 again.

[PowerGlove]

32-bit has a maximum value of 2^32 or 4,294,967,296, so the range is from 0~4,294,967,296.

You mean 2^32-1, right? (As in, the range for uint32_t goes from 0 to 4294967295.)



(That's a riff by Leon Bambrick on the famous Phil Karlton quote, I think.)

Seriously off-topic: Leon Bambrick's blog is really worth spending time on. There's some excellent stuff there. You Feel The Blood Drain From Your Face gave me flashbacks (I think almost every programmer that has done time as the resident "problem solver" at a small/medium company has an anecdote or two similar to that one from their early days).

[moonie]

Nonce = Number-only-used-once.

By incrementing this value during hashing, miners search the transaction space for viable blocks which satisfy the difficulty threshold.

[promise444c5]

32-bit has a maximum value of 2^32 or 4,294,967,296, so the range is from 0~4,294,967,296.
It has a relatively low max value compared to the average ASIC's hashrate but miners rely on "extra nonce" (in the coinbase transaction) or any other data in the block header so that they can start the nonce from 0 again.

Oops didn't thought it that way ,thanks for the clarity.
However, can you clear me a little bit more on "coinbase transaction" I'm still a little bit confused. Sorry for getting too inquisitive over this topic 

[nc50lc]

Oops didn't thought it that way ,thanks for the clarity.
However, can you clear me a little bit more on "coinbase transaction" I'm still a little bit confused. Sorry for getting too inquisitive over this topic  

That's a general term for the first transaction in the block where the coinbase (or the block reward) is included as input.
Not to be confused with the company with the same name which is usually written with upper-case "C".

You can use any blockexplorer and look for blocks, like block height 800000: blockstream.info/block/00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054
The first transaction is the so called coinbase transaction.

[ranochigo]

Now here's where I got confused, Miners have to guess this number to find a compatibility hash for the header.....how do they get to change the noce in the header since its hashed along with the block-header-dat thought they have different  hash . I'm just completely confused here because the whole guessing of a thing seem like magic to me or does the hardwares get to do it all??

The nonce is a variable that can be used for the hash to be different; anything that changes the input hash (block header) would change the resultant block hash. Since using the exact same input gets the exact same hash, we would have to ensure that the nonce is different by incrementing it with every iteration. If not, then using the same random nonce and the same block headers could result in recomputing the same hash. Not all of the implementations will add 1 to the nonce, and certain clients may choose to skip nonces but the result should be that the nonce are distinct if the rest of the block headers are the same.

An interesting note, there is a good reason why nonce is the preferred variable to change. SHA256 is divided into chunks of 64 bytes; [nVersion, PrevHash, 28 bytes of merkle root] + [4 bytes of merkleroot, timestamp, difficulty, nonce and padding].  Changing the nonce would be more efficient than changing anything else since you would only have to compute the second chunk. Hence, miners repeatedly hash the latter part of the chunk to find the nonce which yields a hash that has the appropriate target.

[promise444c5]

Oops didn't thought it that way ,thanks for the clarity.
However, can you clear me a little bit more on "coinbase transaction" I'm still a little bit confused. Sorry for getting too inquisitive over this topic 

That's a general term for the first transaction in the block where the coinbase (or the block reward) is included as input.
Not to be confused with the company with the same name which is usually written with upper-case "C".

You can use any blockexplorer and look for blocks, like block height 800000: blockstream.info/block/00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054
The first transaction is the so called coinbase transaction.

Understood!

The nonce is a variable that can be used for the hash to be different; anything that changes the input hash (block header) would change the resultant block hash. Since using the exact same input gets the exact same hash, we would have to ensure that the nonce is different by incrementing it with every iteration. If not, then using the same random nonce and the same block headers could result in recomputing the same hash. Not all of the implementations will add 1 to the nonce, and certain clients may choose to skip nonces but the result should be that the nonce are distinct if the rest of the block headers are the same.

Thanks for this  I'm quite learning alot from it, it took me a while to understand as I had to go through it over and over
 I have another question: if not all client use the +1 system then how will they get a distinct nonce or could there be a possibility of an input hash not having a nonce

[JiiBs]

The value of the nonce is completely up to the miner and does not depend on anything else. So, if one value doesn't work, then another is tried. Typically, the nonce starts a 0 and it is incremented until a qualifying hash is found or all the possible nonces have been tried. If all the nonces have been tried, then something else in the header is changed and the nonce starts back at 0. This process keeps going until a block header that has a qualifying hash is found.

Just a little clarification here.

After having some runnings with the white paper,
I get the point that the nonce have got to start with a zero but, I thought it was a number that gets manipulated by means of an increase until you get an unused one that haves it’s start at zero to be used for a block header.

[ranochigo]

I have another question: if not all client use the +1 system then how will they get a distinct nonce or could there be a possibility of an input hash not having a nonce

The purpose of a nonce is a variable to vary the block header such that each hash is different. Hence, it wouldn't matter if it is unique or not, so long as the miner isn't hashing the same thing over and over again. Take this oversimplified example, for example 100001 is the entire block header and the nonce is 0000. Not incrementing the nonce would be equivalent to hashing 100001 over and over again, which yields the exact same hash and isn't helpful in helping us to find a valid block. If we were to increment the nonce, we could hash 100011, which yields a completely different block hash and we could change that until we find a valid block.

The nonce can be zero, but it cannot be non-existent.

I get the point that the nonce have got to start with a zero but, I thought it was a number that gets manipulated by means of an increase until you get an unused one that haves it’s start at zero to be used for a block header.

Nonce doesn't have to be unused across everything. It just has to be unused for any given block header, if you were to reuse a nonce, you would be hashing something that has been hashed by you before. It would be a big waste of resources.

All miners should have a unique blockheader, whether it is by varying the extranonce, transaction order, or anything else that would make the block header different.

[promise444c5]

I have another question: if not all client use the +1 system then how will they get a distinct nonce or could there be a possibility of an input hash not having a nonce

The purpose of a nonce is a variable to vary the block header such that each hash is different. Hence, it wouldn't matter if it is unique or not, so long as the miner isn't hashing the same thing over and over again. Take this oversimplified example, for example 100001 is the entire block header and the nonce is 0000. Not incrementing the nonce would be equivalent to hashing 100001 over and over again, which yields the exact same hash and isn't helpful in helping us to find a valid block. If we were to increment the nonce, we could hash 100011, which yields a completely different block hash and we could change that until we find a valid block.

The nonce can be zero, but it cannot be non-existent.

Hummm, the purpose of the rehashing is to get a target a hash that is equal or less to the real input hash  so with that example aren't we going to get anything above the hash (if true )

[ranochigo]

Hummm, the purpose of the rehashing is to get a target a hash that is equal or less to the real input hash  so with that example aren't we going to get anything above the hash (if true )

That's not how hashing works. You might've misunderstood something there. For starters, there is something known as the Avalanche Effect in Cryptography, for which the slightest change in the input results in an entirely different hash. You can verify it by hashing 100001 and 100011, they are completely different hashes and the latter hash is actually smaller as compared to the former when converted to hex.

The probability of the hash meeting the target is equal with every single nonce in the range.

[promise444c5]

Hummm, the purpose of the rehashing is to get a target a hash that is equal or less to the real input hash  so with that example aren't we going to get anything above the hash (if true )

That's not how hashing works. You might've misunderstood something there. For starters, there is something known as the Avalanche Effect in Cryptography, for which the slightest change in the input results in an entirely different hash. You can verify it by hashing 100001 and 100011, they are completely different hashes and the latter hash is actually smaller as compared to the former when converted to hex.

The probability of the hash meeting the target is equal with every single nonce in the range.

I think I need to read deeper into hash to understand how it works then...
 

[mikeywith]

Hummm, the purpose of the rehashing is to get a target a hash that is equal or less to the real input hash  so with that example aren't we going to get anything above the hash (if true )

real input hash is irrelevant here, the "hashed" version of the input is what matters to solve a block, let me try to simplify the subject as much as I can, when you start hashing a new block you would take the previous blockhash and let that be "1" and the current target is a number that starts with one zero "0" target and difficulty are basically the same thing represented in an inversed way to explain the same thing, but let's just say it's a number that starts with a single zero.

Now go to a SHA256 hash tool like https://emn178.github.io/online-tools/sha256.html and input the previous block hash which is "1"



No matter how many times you try to hash that blockheader, you are always going to get

Code:

6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b

But what you need is a hash that starts with 0, so now you add a nonce to it in order to produce a hash that has a single leading zero, so let's try to add 1 and hash 11 (first 1 is the prev blockhash, second one is our nonce)

result = 4fc82b26aecb47d2868c4efbe3581732a3e7cbcc6c2efb32062c08170a05eeb8   = no block
let's try 111

f6e0a1e2ac41945a9aa7ff8a8aaa0cebc12a3bcc981a929ad5cf810a090e11ae

keep trying randomly, I got mine at 156 so nonce 56 got me a valid block.

Code:

0fecf9247f3ddc84db8a804fa3065c013baf6b7c2458c2ba2bf56c2e1d42ddd4

Now let's assume difficulty has increased, which means the hash needs to start with 00 not just 0

so you take my first block hash which is

Code:

0fecf9247f3ddc84db8a804fa3065c013baf6b7c2458c2ba2bf56c2e1d42ddd4

and add a nonce to it, hash them together, to get 00xxxxxxxxx, I was actually very lucky with this random nonce "3010023340584300011"

hash > 0fecf9247f3ddc84db8a804fa3065c013baf6b7c2458c2ba2bf56c2e1d42ddd4430100233405843 00011 = 00c0a543e028d4e08d8b6e81cf3a5e1f43b57b20723d1c6625e09e788a69e3ce

which is a valid block.


Obviously, since Nonce is pretty small (~ 4 billion value range) it's exhausted in no time by all ASIC miners, the average miner today can do 100TH/s, which is 100 trillion double sha hashes per second, so changing Nonce alone isn't practical and there needs to be something else to reset it.

The good thing about hashing is that any little change you make to the input would generate a completely different output, miners would use Extra Nonce and nTime, as ranochigo explained above if you have

[nVersion, PrevHash, 28 bytes of merkle root] + [4 bytes of merkleroot, timestamp, difficulty, nonce and padding]

In your input, anything you change there would mean a new set of possibilities, of course, there are things that miners today can't change (given how mining pools operate) but then mining pools would give a different block template to every single miner and the miner would never run out or rehash the same inputs.

To give you another "real example" let's just hash the text I quoted above and start with 1111 once

[nVersion, PrevHash, 28 bytes of merkle root] + [4 bytes of merkleroot, timestamp, difficulty, 1111 and padding]

let's assume the max nonce I can use is 9999, so I did try up to 9999 but couldn't find a hash that has two leading zeros, what can I do now? I will change a tiny bit of the timestamp chunk and make it timestamp1, even without changing the 1111 once I still get a new different hash which is

Code:

b80e254e1edba8fbc10ab93ceab7eec458ea949b947ce8506b0f543418c29add

Think of Nonce in colors, you paint a picture and somebody is waiting to see a certain final mixed color, your base color can be changed but the colors you add are pretty limited(green and yellow), so you created your base color by mixing blue+red, you added green, it didn't work, you added yellow -- didn't work, you added 0.3 yellow and 0.7 blue it didn't work, you run out of mixing options and the desired color is not there yet, you go back to your base color and add some orange color, now that 0.3 - 0.7 mix would result in a new different final color, hope that makes sense.

Obviously, the actual mining process and the blocktemplete are a lot more complicated than my explanation above, but essentially, it is how it works.