A few basic, or not so basic questions

[newb789]

1. How does a pool know miner's hash rate? I know miner sends easier successful hashes (shares?), but that's not a direct hash rate.

2. What inputs are used for hashing? I've read about Nonce, timestamp, extra Nonce.

3. What is the algorithm for input variations?

4. Is there a random element in how input variations are determined so that two identical miners do not follow the same input sequence? Because if there isn't then they would arrive at exactly the same sequence of hashes, and therefore a miner with better Internet connection (less latency) wins.

5. I am guessing that there is randomness. In this case, is there a chance that the same combination of input parameters may be repeated, thus wasting valuable hash rate? Or is the number of combinations of input variations is so great that there is little chance of that happening?

6. Why hash rate keeps fluctuating? E.g. for my LV07, it averages 1.0TH, but fluctuates between 0.85TH - 1.2TH. Every 3-4 min or so it reaches low point, and in the next 3-4 min it reaches the high point. And on, and on.

7. My LV07 frequency is set to 575 MHz. It generates around 1.0 TH. This means that during every tick the ASIC calculates (1.0 Th / 575 MHz) = 1739 hashes. How is that possible? Each hash is a complex calculation, and doing 1739 of them in a single tick!?! A regular CPU may execute a simple operation in a single tick - like moving data from register to memory, or perform an addition, etc.

[NotFuzzyWarm]

Each hash is a complex calculation, and doing 1739 of them in a single tick!?! A regular CPU may execute a simple operation in a single tick - like moving data from register to memory, or perform an addition, etc.

A. It is NOT a complex calculation. The SHA algo simply takes a set of inputs and applies them to a boolean logic table which then spits out a result. It is not doing any kind of usual add/subtract/multiply/divide/carry sequential math operations like how a typical computer does things. In a sense, you could say those functions happen all at once in a massively parallel operation. A good start on that using code that works with chip design software such as Cadence is http://stevengoldfeder.com/projects/circuits/sha2circuit.html or do a search using "sha256 logic gates" as the search term.

B. That logic table is hardwired into each ASIC core which is each comprised of thousands of AND, XOR and INV gates, each chip has from several hundred to maybe 2k of those cores and a miner can have from 1 to a few hundred of the ASIC chips in it.

C. Operation is:
 c1. Pool supplies work to the miner controller that creates the inputs to be fed to the ASIC's.
 c2. From that work, apply inputs in parallel, cycle clock to run through the logic table which will then immediately present the result which is a simple 1 or 0 (yes you found a block or no so try again).
 c3. If no block found, increment nonce value then repeat 'c2 & c3' until a block is found or until pool sends new work.

D. If a block is found, notify the pool you are using so it can verify and broadcast 'Block Found' to the BTC network.

The only real bottleneck is that yes, the CPU's in a miner are creating the inputs to the ASIC chips and responding to their output however that does not take much speed to do. That is why RasPi's, BeagleBones, etc can be used as the controller in a miner.

A good place for you to read up on what the input data is can be found starting here https://kano.is/index.php?k=mining
A rather fun live action page for what happens is here https://sha256algorithm.com/

[ABCbits]

2. What inputs are used for hashing? I've read about Nonce, timestamp, extra Nonce.

Block header, which have 80 bytes size. Aside from 3 field you mentioned, IIRC overt ASICBoost also manipulate nVersion.

6. Why hash rate keeps fluctuating? E.g. for my LV07, it averages 1.0TH, but fluctuates between 0.85TH - 1.2TH. Every 3-4 min or so it reaches low point, and in the next 3-4 min it reaches the high point. And on, and on.

Without additional information, i would speculate your LV07 is in loop due to overheating where it underclock itself temporarily.

[newb789]

Thank you for the detailed replies. After I read everything, and played with illustrations, I still cannot answer a simple question:

If I have two identical ASICs sitting in a room, pointed to the same pool, and using the same connectivity - will they be following the same sequence of input parameters? If not, who and how makes them different?

[NotFuzzyWarm]

...
If I have two identical ASICs sitting in a room, pointed to the same pool, and using the same connectivity - will they be following the same sequence of input parameters? If not, who and how makes them different?

Each miner receives shares from the pool typically at a rate of 18 shares per-min and each share is unique work.
ref  https://kano.is/index.php?k=minedet for how mining works.

From that link:

So for stratum we put a bunch of other 'nonces' in the coinbase "sig", that includes a unique number for each miner, and run the others from 0 up to some specified limit to generate a different coinbase transaction each time, that will generate a unique Merkle Root, to be put in a block header, to be hashed over the full Nonce range.

Thus with a unique number for each miner, and the miner itself following the stratum rules for creating coinbase transactions, the miner can keep generating a large amount of work that isn't expected to run out before the pool sends it new work.

The miner also doesn't need to waste time processing bitcoin network information or verifying transactions, that is the job of the pool and requires fast global internet access, a very fast CPU and lots of RAM. It only needs network access to the pool or even just a local mining proxy, and spends all it's effort on processing work and hashing nonces.

[newb789]

That was helpful. Thank you. It is now clear that two miners side-by-side would be using completely different inputs.

But what about ALL MINERS in the world? Is the number of different "sig" variations great enough so that each pool changing "sig" in a certain way leads to unique inputs for each miner across the board? I am guessing it is because mempool.space shows ScriptSig to be about 100 bytes in size, of which I am guessing about 90 bytes could be anything, and that's 2**180 - huge number. As a result Merkle Root becomes pretty random, and that's 2**256. Even a fraction of that is a huge number. I guess I answered my own question.

Completely different question... I remember reading something along the lines that a new block cannot be mined 2+ hours ahead of most recent block at the top of the blockchain. What happens when 12 freshly mined blocks are mined faster than in 2 hours? Do miners all over the world stop and wait until some time passes in order not to violate that rule? In other words, what happens to all miners if blocks are mined a little too fast for a few hours?

[NotFuzzyWarm]

...
Completely different question... I remember reading something along the lines that a new block cannot be mined 2+ hours ahead of most recent block at the top of the blockchain. What happens when 12 freshly mined blocks are mined faster than in 2 hours? Do miners all over the world stop and wait until some time passes in order not to violate that rule? In other words, what happens to all miners if blocks are mined a little too fast for a few hours?

1st of all, merit given for nicely targeted questions and being a quick learner.

AFAIK the 2+hrs limit only applies to Testnet and is to deal with folks abusing it by pointing high-end ASIC's at it just to mine tbtc.

Mainnet blocks has no such time limit because diff is too high to ever be mining that far ahead of the current block. If anything, I expect that would be rejected. That said, more than 6 blocks/hr (12/2hrs) happens all of the time - that is why diff usually keeps getting higher. All it will do is influence the diff adjustment that happens around every 14 days to maintain the defined average of 10min between block (6/hr).

[mikeywith]

Completely different question... I remember reading something along the lines that a new block cannot be mined 2+ hours ahead of most recent block at the top of the blockchain. What happens when 12 freshly mined blocks are mined faster than in 2 hours? Do miners all over the world stop and wait until some time passes in order not to violate that rule? In other words, what happens to all miners if blocks are mined a little too fast for a few hours?

That rule isn't exactly as you describe it. It's actually 2 hours ahead of the adjusted network time, which is the median of timestamps from the node’s connected peers. However, contrary to what many people believe, this is not a part of the consensus rules—i.e., blocks that are mined with a timestamp more than 2 hours in the future are not invalid blocks; they are just not accepted.

For example, if my node's adjusted network time is 1:00 AM and your block's timestamp is 3:01 AM, my node would simply ignore it (which is different from rejecting it). However, if your node relays that block again two minutes later, when my time is 1:02 AM, your block will be accepted.

And no, miners won't stop; everyone would continue mining, and eventually, the longest valid chain would win, assuming none of the blocks it was built on are invalid.

MAX_FUTURE_BLOCK_TIME was implemented to prevent miners from artificially increasing difficulty. If all miners falsely claim their block took 10 hours to mine, the difficulty would increase drastically. This rule ensures that even if every block timestamp was manipulated to the maximum extent, the highest possible difficulty increase per adjustment period would be 2/336; that is slightly above 0.5% for every difficulty adjustment..

[newb789]

LOL, I think I am good for now. Thank you guys for all detailed explanations.

Pretty amazing how it all works together, and how well it is all thought through. As a software developer of many years, I can only appreciate the beauty of the thing.

Now, back to my Lucky Miner which is stuck at 246M difficulty, LOL

Oh, another but unrelated question. Maybe someone gives me an idea what's going on...

My Internet speed is 300 Mb/s download and 20 Mb/s upload (was 20/20 before the upgrade - see further below). A few months ago, I set up a node. Right off the bat, my router (or external network) would start freezing after a day or a few. Specifically, connection to router for all home devices is fine, but no traffic goes though (including to/from the node). Rebooting the router clears the problem - until next time - in a day or a few. I also discovered that there was no need to reboot the router. Unplugging coax cable for a few seconds does the trick, and everything goes back to normal in a few seconds.

Never happened before I set up my node. If I shutdown the node, everything is fine for days and days. Started the node - same issue.

Upgraded to the newest router from ISP - same issue. Asked my ISP if there was a quota for uploads. They said NO.

Via experimentation, I determined that limiting upload to 5GB/day solves the problem - runs smoothly for weeks with no issues.

Any ideas?

[mikeywith]

LOL, I think I am good for now. Thank you guys for all detailed explanations.

Pretty amazing how it all works together, and how well it is all thought through. As a software developer of many years, I can only appreciate the beauty of the thing.

Now, back to my Lucky Miner which is stuck at 246M difficulty, LOL

Oh, another but unrelated question. Maybe someone gives me an idea what's going on...

My Internet speed is 300 Mb/s download and 20 Mb/s upload (was 20/20 before the upgrade - see further below). A few months ago, I set up a node. Right off the bat, my router (or external network) would start freezing after a day or a few. Specifically, connection to router for all home devices is fine, but no traffic goes though (including to/from the node). Rebooting the router clears the problem - until next time - in a day or a few. I also discovered that there was no need to reboot the router. Unplugging coax cable for a few seconds does the trick, and everything goes back to normal in a few seconds.

Never happened before I set up my node. If I shutdown the node, everything is fine for days and days. Started the node - same issue.

Upgraded to the newest router from ISP - same issue. Asked my ISP if there was a quota for uploads. They said NO.

Via experimentation, I determined that limiting upload to 5GB/day solves the problem - runs smoothly for weeks with no issues.

Any ideas?

Some routers store packets in thier DRAM, how does the DRAM reset is going to be different, obviously a reboot will clear it, so that may be your issue, check with the router manufacturer if it is the case, there will probably be away to disable that logging so that the DRAM doesn't get clogged.

I have never seen anyone complain about router issues running a node, but saw some people with larger amount of data face the same issue, so this might be it.

[newb789]

Interesting! Thank you. That could explain why re-plugging the coax cable could clear internal log of saved packets.

I don't have logging feature on my admin screen provided by my ISP, but will try calling them and asking whether there is any option like that which they can control on their end. I am reluctant to tell them what I am running, so will have to just tell them something generic.

Oh... One more question that I am not able to find an answer to. How the pool knows my hash rate if my miner only submits lower difficulty hashes it manages to find. Does the pool just estimate my hash rate based on that information?

[NotFuzzyWarm]

...
Oh... One more question that I am not able to find an answer to. How the pool knows my hash rate if my miner only submits lower difficulty hashes it manages to find. Does the pool just estimate my hash rate based on that information?

Not exactly, per Kano from his site:

All hash rates are a calculation based on shares submitted, they are not based on a count of hashes a miner has done.
Your miner doesn't know how many hashes it's done either, it also just does a calculation based on the Difficulty of the nonces it generates.

and how a pool more accurately calculates your hash rate:

How it is actually calculated
A 1 Difficulty share is considered proof of having done approximately 4 billion hashes (2^32 hashes).
So as an example, over a 1 minute time range, if a miner has submitted the expected average 18 Shares per Minute,
and the miner was mining at the default 8190 Difficulty, then it's hash rate would simply be 8190 * 2^32 * 18 / 60 H/s
With those numbers you'd get 10.55THs.

Of course the Difficulty you mine at will be set by the pool, and will be based on your miner hashrate, so that you are submitting, on average, 18 Shares per Minute.
See Help->Workers here for more information about Difficulty.

edit: fixed the missing exponent marker in 2^32 hashes, good catch by Mikey

[mikeywith]

All hash rates are a calculation based on shares submitted, they are not based on a count of hashes a miner has done.
Your miner doesn't know how many hashes it's done either, it also just does a calculation based on the Difficulty of the nonces it generates.

That's a perfect explanation, although I think one needs to really understand the difference between hashes and shares, the simplest way to think of it is that shares are essentially 'accepted' hashes, you could have a centillion number of hashes and 0 shares.

The equation is rather simple; the example posted by NFW is missing the ^ between 2 and 32, which was probably lost in the pasting process.

Estimated Hashrate= Shares Submitted×Share Difficulty×2^32 / Time
 
​Using the same numbers from NFW post you have


Estimated Hashrate= 18×8190×2^32 / 60
                             =18x 8190x4,294,967,296 / 60
                             = 633,164,078,776,320 /60
                             = 10,552,734,646,272 hashes per second

You can convert it to terahash by dividing it by  1 trillion (12 zeros), and you get 10.55 THs
​
This is the reason why it takes time for your reported hashrate to become accurate, and it's how the pool adjusts the share difficulty to avoid two things
1- you sumbiting a huge number of shares per x unit of time.
2- you are not sumbting any shares for a very long period.

Obviously, the targeted number of shares per x unit is going to differ from one pool to another, you can figure this out based on the share difficulty the pool sets vs your estimated hashrate.

 

[newb789]

Ugh.. Hmm.. A bit confused. I think I am missing something. I just watched my LV07 submit 8739 shares to ckpool in 1 minute, and it's been doing that consistently - this is what the pool shows (not the miner). This is a lot more than 18 shares from the example above. However, my hash rate (displayed by both, the pool and the miner) is around 1 TH/s.

Also, where did the number 8190 come from? Maybe it should be 8192 (2^13). But then, why 2^13?

On the other hand, miner itself shows 1 share about every 2-3-4 sec - which is consistent with 18 shares/min.

Here is the fragment of the miner log, which is consistent with about 18 shares per minute while watching it in real time:

₿ (149791716) asic_result: Ver: 2CAD2000 Nonce 6504004D diff 498.6 of 971.
₿ (149792476) asic_result: Ver: 2197C000 Nonce F31C000D diff 1494.4 of 971.
₿ (149792476) stratum_api: tx: {"id": 36816, "method": "mining.submit", "params": ["bc1rqk-bla-bla-bla", "67444bb00004c2b8", "0b1a010000000000", "67cc3f06", "f31c000d", "0197c000"]}
₿ (149792676) stratum_task: rx: {"result":true,"error":null,"id":36816}
₿ (149792686) stratum_task: message result accepted

I understand some of it, but would be nice to understand more:

a) minimum difficulty is set by the pool to be 971. Is it 971 times more difficult than the base difficulty of 1?
b) the first result is ignored because its hash's difficulty of 498.6 is below the set limit of 971.
c) nonce F31C000D is transmitted to the pool as well as the verification 2197C000 (although verification was transmitted as 0197c000).
d) the result was accepted by the pool.

What do the other numbers mean?

[mikeywith]

Ugh.. Hmm.. A bit confused. I think I am missing something. I just watched my LV07 submit 8739 shares to ckpool in 1 minute

You must be reading something wrong, you can't be submiting 8k shares in 1 minute.

Also, where did the number 8190 come from? Maybe it should be 8192 (2^13). But then, why 2^13?

8192 is the share diff in the example provided by NFW, no the 2^32 is correct, it is needed in the equation to "translate" share diff to hashrate.


fine, plot your numbers in the equation.

18×971×2^32/60

you get 1.25terahash

if your hashrate is indeed an exact 1terahash then it is not exactly 18 shares / min

you can use the same equation (basic math) you get =14.396 shares per minute.

clear?


 
​
 

[NotFuzzyWarm]

You must be reading something wrong, you can't be submiting 8k shares in 1 minute.

8192 is the share diff in the example provided by NFW, no the 2^32 is correct, it is needed in the equation to "translate" share diff to hashrate.
....
fine, plot your numbers in the equation.

18×971×2^32/60
you get 1.25terahash

if your hashrate is indeed an exact 1terahash then it is not exactly 18 shares / min
you can use the same equation (basic math) you get =14.396 shares per minute.

clear?

Also, where did the number 8190 come from? Maybe it should be 8192 (2^13). But then, why 2^13?

8192 is the default starting diff that KanoPool uses when a miner 1st connects to the pool after a reboot. The minimum diff his pool allows is 432, there is no maximum limit. As you submit shares over time that adjusts to whatever is needed to hit the 18 shares/min that his pool shoots for. eg. one of my 40TH/s miners there uses a work diff of 27,297. Other pools probably aim for a somewhat different share rate.

As Mikey said, you cannot be submitting 8k shares/min - that would suck up too much bandwidth. The whole point to a pool setting a working diff is to keep the bandwidth to a reasonable level. Remember, a pool has hundreds/thousands of miner connected to it and cannot tolerate that kind of data usage from them all.

[mikeywith]

you cannot be submitting 8k shares/min - that would suck up too much bandwidth. The whole point to a pool setting a working diff is to keep the bandwidth to a reasonable level. Remember, a pool has hundreds/thousands of miner connected to it and cannot tolerate that kind of data usage from them all.

Heck, at 8k share per connection those large pools will need to spend a fortune on infrastructure, it is not only the bandwidth, there is a more resource consuming part which is verifying the shares, verifying miner's work/share is the same as verifying a block propagated by another node.

1- The miner will send a block header containing the Merkle root, previous block hash, timestamp, and nonce.
2-The pool verifies that the Merkle root and other data match the work sent earlier to the miner.
3-The hash of the block header calculated with the nonce is lower than the target/higher than share diff set by pool or not?

If the hash is valid, the share is accepted and the piece of info is recorded somewhere on some database to be used in the payment, if the share is invalid, that info will also be recorded for future reference like hashrate graph on the user's dashboard.

It takes some serious resources to receive, verify and record shares,  if every single miner sends 8k shares that is going to suck.

[newb789]

Got it. Thanks.

To clarify, here is the recent snapshot from the pool:

=====================================
solo.ckpool.org/users/bc1bla-bla-bla

{
 "hashrate1m": "1.19T",
 "hashrate5m": "1.06T",
 "hashrate1hr": "1.01T",
 "hashrate1d": "1.02T",
 "hashrate7d": "873G",
 "lastshare": 1741555965,
 "workers": 1,
 "shares": 281621832,
 "bestshare": 245888437.7731853,
 "bestever": 245888437,
 "authorised": 1740352205,
 "worker": [
  {
   "workername": "bc1bla-bla-bla",
   "hashrate1m": "1.19T",
   "hashrate5m": "1.06T",
   "hashrate1hr": "1.01T",
   "hashrate1d": "1.02T",
   "hashrate7d": "873G",
   "lastshare": 1741555965,
   "shares": 281621832,
   "bestshare": 245888437.7731853,
   "bestever": 245888437
  }
 ]
}

====================================

Here is the info from the miner admin screen:

Model:   LV07
Uptime:   3 days
WiFi Status:   Connected!
Free Heap Memory:   8420560
Version:   1.2.0

Hash Rate
1.13 TH/s

Shares
65,400

Efficiency
19.56 J/TH

Best Difficulty
246M all-time best

============================

I am confused, and trying to reconcile different numbers. Here is what I mean.

1. My miner says it submitted 65,400 shares in roughly 3 days. Sounds about right - corresponds to around 15 shares/min. But the pool says: 281621832 shares. What?

2. Watching log file: minimum difficulty was 971, and I've seen actual difficulty around 1500 - 2000 - 3000. However, both the miner and the pool refer to 246M as all time best (the miner) and 245888437 (the pool). For comparison, recent block 887080 has difficulty of 112,149,504,190,349, or roughly 112T.

As an exercise, I did calculate minimum hash value for bits 0x17028281, checked a few blocks, and verified that their hash value was less than the minimum set by the bits, while the spread for about 10 blocks I checked was within that range and was clearly somewhat random. I also divided 00FFFF... value (for difficulty 1) by the actual hash for a couple of blocks, and got the same difficulty number (around 112T) as specified for each block. So, aside from inconsistencies above, I am getting a decent handle on basic understanding.

[NotFuzzyWarm]

Now you are in the realm of how -ck has setup his pool, I suggest you ask him about it in his pool thread.

[mikeywith]

the number at the top is the accumulated number of shares that were accepted from any miner that used this btc address , the one under the worker name is the number of shares that were ever accepted by this exact worker, given that you only used one worker the numbers are identical.

What you need to do is either count from the current number of shares going forward or use a fresh btc addy to watch your shares go from 0 to x.

except the hashrate and "bestshare" which resets whenever the pool finds a block the other numbers like shares and bestever will never reset unless done manually by ck, i do not recall anyone asking him publicly to reset those, besides, btc address are free and easy to create, so he will most likely reject any request of that nature.