[ANN] Stratum mining protocol - ASIC ready

[Roy Badami]

I have a question re Stratum, namely: does Stratum support rollntime? - I skimmed the spec and couldn't see a mention (and apologies if this has already been discussed and my search missed it).

What I'm thinking is this:  a 60GH/s miner (which we have every expectation is going to be pretty run-of-the mill once ASICs hit) is going to consume 14 work units per second, and it's reasonable to assume that they will continue to get faster over time.  Over time, as mining hardware improves, then on low-end hardware the CPU load on the mining host will start to become more significant, particularly if hosting a large farm of ASIC miners.  The USB I/O load may start to become significant, too, particularly if the mining host is actually an old laptop that only does USB 1.

ntime rolling can trivially be implemented in the hardware miner (even if in the microcontroller rather than in the ASIC), and doing so would reduce by almost two orders of magnitude the number of work units that have to be generated by the mining host and communicated to the hardware miners.

Longer term, I imagine that the microcontroller in ASIC miners will roll extranonce itself, but the extra CPU and memory requirements placed on the microcontroller, not to mention the extra firmware complexity, probably means this won't be a popular approach with hardware manufacturers until it becomes unavoidable.

Even once we get to that point, I can imagine that as ASIC cores get faster, and smaller (so one microcontroller potentially driving more of them) then hardware designers might opt for an architecture where ntime is incremented in the ASIC, with the microcontroller only rolling extranonce and calculating the Merkle root, just to reduce the CPU and IO requirements within the hardware miner.

This is all just me thinking aloud, and it might be that we really are convinced that the usefulness of ntime rolling is so far away that it's just not worth thinking about, but I just thought I'd throw it out there....

Thoughts?

roy

[1714974415]

1714974415

[1714974415]

1714974415

[1714974415]

1714974415

[1714974415]

1714974415

[eleuthria]

I have a question re Stratum, namely: does Stratum support rollntime? - I skimmed the spec and couldn't see a mention (and apologies if this has already been discussed and my search missed it).

What I'm thinking is this:  a 60GH/s miner (which we have every expectation is going to be pretty run-of-the mill once ASICs hit) is going to consume 14 work units per second, and it's reasonable to assume that they will continue to get faster over time.  Over time, as mining hardware improves, the CPU load on low-end hardware hosts will start to become more significant, particularly if hosting a large farm.  The USB I/O load may start to become significant, too, particularly if their host is actually an old laptop that only does USB 1.

ntime rolling can trivially be implemented in the hardware miner (even if in the microcontroller rather than in the ASIC), and doing so would reduce by almost two orders of magnitude the number of work units that have to be generated by the host and communicated to the miners.

Longer term, I imagine that the microcontroller in ASIC miners will roll extranonce itself, but the extra CPU and memory requirements placed on the microcontroller, not to mention the extra firmware complexity, probably means this won't be a popular approach with hardware manufacturers until it becomes unavoidable.

Even once we get to that point, I can imagine that as ASIC cores get faster, and smaller (so one MCU potentially driving more of them) then hardware designers might opt for an architecture where ntime is incremented in the ASIC, with the MCU only rolling extranonce and calculating the Merkle root, just to reduce the CPU and IO requirements within the hardware miner.

This is all just me thinking aloud, and it might be that we really are convinced that the usefulness of ntime rolling is so far away that it's just not worth thinking about, but I just thought I'd throw it out there....

Thoughts?

roy

Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit (by default, it can be increased and the spec supports it very easily).  However, Stratum also does support ntime rolling, the official spec is that ntime should not be increased faster than real-time.

nTime rolling is a hack solution to extend getworks, and if used improperly can be extremely bad for the network if a large number of blocks were produced and pushed the mean-time of recent blocks into the future.

[Roy Badami]

Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit

Yes, but it increases the size of the work unit at the expense of complexity.  Incrementing ntime extranonce and computing a new Merkle root is something that probably can't easily done in a hardware miner without increasing hardware cost, hence requiring work (and communication) by the host that the hardware miner is attached to everytime the nonce wraps.  On the other hand, ntime rolling could trivially be done in hardware (in the MCU, or even in future designs in the ASIC) massively reducing the work done by the host.

My thought is that you would do both: you'd incrememnt the nonce, naturally (this is done in hardware in the FPGA or ASIC).  Then when the nonce wraps round, you increment ntime (which could be done in the hardware miner's microcontroller, or longer term, in the ASIC).  Once ntime has been rolled by some configurable amount (30 seconds, 60 seconds, whatever), then ntime is reset, the host PC incremements extranonce, calculates a new work unit, and gives this work unit to the hardware miner.

Bottom line: the work unit for the protocol may be 4.2 billion times larger, but the work unit for the hardware miner will probably be unchanged in the short term.  ntime rolling is a far easier step for the hardware to take then full merkle branch support in the hardware.

[slush]

Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit (by default, it can be increased and the spec supports it very easily).  However, Stratum also does support ntime rolling, the official spec is that ntime should not be increased faster than real-time.

nTime rolling is a hack solution to extend getworks, and if used improperly can be extremely bad for the network if a large number of blocks were produced and pushed the mean-time of recent blocks into the future.

Stratum protocol don't need to mention ntime rolling, because block header for hashing is produced directly on the miner, not by the pool. But re: Roy's concern about ASIC miners - yes, ntime rolling in ASICs will be still possible with Stratum.

[-ck]

Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit (by default, it can be increased and the spec supports it very easily).  However, Stratum also does support ntime rolling, the official spec is that ntime should not be increased faster than real-time.

nTime rolling is a hack solution to extend getworks, and if used improperly can be extremely bad for the network if a large number of blocks were produced and pushed the mean-time of recent blocks into the future.

Stratum protocol don't need to mention ntime rolling, because block header for hashing is produced directly on the miner, not by the pool. But re: Roy's concern about ASIC miners - yes, ntime rolling in ASICs will be still possible with Stratum.

Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

[Roy Badami]

Stratum protocol don't need to mention ntime rolling, because block header for hashing is produced directly on the miner, not by the pool. But re: Roy's concern about ASIC miners - yes, ntime rolling in ASICs will be still possible with Stratum.

Hmm, are you saying that the need with getwork for the pool to explicitly permit rollntime (and to tell the miner how much it's allowed to roll ntime) is just a quirk of how traditional pools generate work?  And that the assumption is that a Stratum pool will just accept any valid block, regardless of the value of the ntime field within the block header (within reason)?

roy

[Roy Badami]

Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.

ETA: if cgminer running on my EEEPC 701 driving a couple of ASIC miners can keep the CPU load low enough that the CPU fan doesn't come on, I'd be very happy.  My guess is that it would need the miner to roll ntime to accomplish this.

[-ck]

Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.

Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

[Roy Badami]

Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Yeah, OK, you're right.  So assuming (for ease of maths) 2^10 transaction per block, you need 10 hashes per Merkle root, so you need a host CPU capable of 3.75kH/s.  Hell, my first gen EEEPC would do that more than comfortably (although it wouldn't surprise me if that's 10% CPU or so - I'm sure enough to make the CPU fan come on, even if only at low speed

Ok, so you've convinced me, we've got a little time before ASICs are fast enough to worry about this.... :-)

roy

ETA: But in 2 years time, when we have $99 600GH/s coffee warmers, then I'm going to be asking for rollntime support :-)

[kano]

Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Yeah, OK, you're right.  So assuming (for ease of maths) 2^10 transaction per block, you need 10 hashes per Merkle root, so you need a host CPU capable of 3.75kH/s.  Hell, my first gen EEEPC would do that more than comfortably (although it wouldn't surprise me if that's 10% CPU or so - I'm sure enough to make the CPU fan come on, even if only at low speed

Ok, so you've convinced me, we've got a little time before ASICs are fast enough to worry about this.... :-)

roy

ETA: But in 2 years time, when we have $99 600GH/s coffee warmers, then I'm going to be asking for rollntime support :-)

roll-n-time is only needed to change the data being hashed when you run out of the nonce range with getwork

Stratum uses the coinbase transaction to resolve this, so the n-time field is simply set to whatever the current time is.

For 1 second of time (n-time=now) you can generate an amount of work greater than ever will be possible to be done in 1 second by any device and thus there is no need to roll the n-time field

As devices hash faster, you simply increase the size of the 'secondary nonce' field in the coinbase

From the point of view of CPUs not being able to hash enough merkle trees - of course as hashing devices get faster you can also expect CPUs to get faster - but implementing something 'like Stratum' in the MCU is not really that big an issue and will resolve it anyway.

Of course at some point in the FAR distant future this will pose a problem, since cycling the 'secondary nonce' in the coinbase does not guarantee to generate unique block headers to hash - the merkle root generated from the transactions is a hash function and thus can produce duplicate output for different input - since the input size is of course MUCH greater than the output size
In fact there is a very small chance of people already hashing the same block header twice - but it is extremely small - the only way around that is how I've suggested increasing the size of the nonce field in the block header - that was rejected

Edit: of course this FAR DISTANT FUTURE problem is present (and exactly the same) in GBT also

[-ck]

Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Yeah, OK, you're right.  So assuming (for ease of maths) 2^10 transaction per block, you need 10 hashes per Merkle root, so you need a host CPU capable of 3.75kH/s.  Hell, my first gen EEEPC would do that more than comfortably (although it wouldn't surprise me if that's 10% CPU or so - I'm sure enough to make the CPU fan come on, even if only at low speed

Ok, so you've convinced me, we've got a little time before ASICs are fast enough to worry about this.... :-)

roy

ETA: But in 2 years time, when we have $99 600GH/s coffee warmers, then I'm going to be asking for rollntime support :-)

You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?

[P_Shep]

Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.

Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

That's not quite all the processing though.

I don't know if you send a target difficulty to the ASIC to work on, but at the moment FPGA's work at difficulty one, don't they?
So cgminer has to take these diff 1 nonces and rehash them to find their actual difficulty, and compare them to the target.
If ASICs don't have a settable target, it could add quite a bit more of a load to the CPU.

[eleuthria]

Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.

Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

That's not quite all the processing though.

I don't know if you send a target difficulty to the ASIC to work on, but at the moment FPGA's work at difficulty one, don't they?
So cgminer has to take these diff 1 nonces and rehash them to find their actual difficulty, and compare them to the target.
If ASICs don't have a settable target, if could add quite a bit more of a load to the CPU.

Validating the difficulty of a solution should only require 2 SHA256 passes and a simple evaluation to determine difficulty.  It would be significantly less work than preparing the work to be sent to the ASIC in the first place.

[-ck]

Early on I spoke to some of the ASIC-to-be manufacturers and suggested they do target testing for higher diffs internally as well, so I expect at least some of them to take a target as well That and as Eleuthria said, it's still insignificant CPU to test them anyway.

[kano]

Early on I spoke to some of the ASIC-to-be manufacturers and suggested they do target testing for higher diffs internally as well, so I expect at least some of them to take a target as well That and as Eleuthria said, it's still insignificant CPU to test them anyway.

... and hopefully every ASIC that does this will also return the amount of work actually done inside the device ...

I hate estimates

[Roy Badami]

Early on I spoke to some of the ASIC-to-be manufacturers and suggested they do target testing for higher diffs internally as well, so I expect at least some of them to take a target as well That and as Eleuthria said, it's still insignificant CPU to test them anyway.

I'd just taken that as a given - it would seem to be madness not to do that in the ASIC.

[Roy Badami]

You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?

Why only once a second?  An ASIC is going to run through all nonces in far less than a second - why can't it roll ntime rapidly at least for a few seconds?  No one cares if ntime is exact to the second, as long as it's roughly right.

[slush]

No one cares if ntime is exact to the second, as long as it's roughly right.

ntime should reflect current time. If there's another way of creating unique data header for hashing, it should be prioritized over ntime rolling.

However ntime rolling is still possible, so don't worry, everything will be fine :-).

[-ck]

You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?

Why only once a second?  An ASIC is going to run through all nonces in far less than a second - why can't it roll ntime rapidly at least for a few seconds?  No one cares if ntime is exact to the second, as long as it's roughly right.

You still haven't understood? To go through all the nonces with stratum currently as is, would require it to check 18,446,744,073,709,551,616 hashes. If that's not enough, you can just increase the size of nonce2 from 4 to 6 and get 1,208,925,819,614,629,174,706,176 hashes. This is without rolling the ntime at all. If that's not enough, rolling the ntime just once every second will give you 1,208,925,819,614,629,174,706,176 hashes per second.

[eleuthria]

You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?

Why only once a second?  An ASIC is going to run through all nonces in far less than a second - why can't it roll ntime rapidly at least for a few seconds?  No one cares if ntime is exact to the second, as long as it's roughly right.

You still haven't understood? To go through all the nonces with stratum currently as is, would require it to check 18,446,744,073,709,551,616 hashes. If that's not enough, you can just increase the size of nonce2 from 4 to 6 and get 1,208,925,819,614,629,174,706,176 hashes. This is without rolling the ntime at all. If that's not enough, rolling the ntime just once every second will give you 1,208,925,819,614,629,174,706,176 hashes per second.

The argument (I think) is that you'd have to rebuild the merkleroot with each extranonce, which requires extra work to either be done on the CPU, or on-board future ASICs (would be recommended, due to latency of relaying new work between the miner and the ASIC), meanwhile with ntime rolling there is no extra building of work, just increasing a secondary counter in the header.

While this is true, I think a better rebuttal would be:
An average piece of work would only take ~10 SHA256 hashes to rebuild the merkleroot.  The # of transactions in a block has to double to require each extra hash to build the merkleroot.  Those ~10 SHA256 hashes produce 4.2 billion hashes worth of work.  So if the chip implements nonce rolling internally, it would only decrease effective speed by ~0.000000238%.

It really all comes down to how ASICs are designed.  If they ever reach the 1 TH/s point in a single chip/set of chips (1 device as the miner software would see it), it could be an issue due to needing 250 pieces of work per second.  A 1 ms latency of providing new work would mean the device is idle roughly 20% of the time since.  Of course, I assume by that point the ASIC hardware would have more internal support for mining protocols to handle extranonce rolling+ntime.