Bitcoin Forum
November 20, 2017, 09:38:09 PM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [18] 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
Author Topic: [ANN] Stratum mining protocol - ASIC ready  (Read 143485 times)
Roy Badami
Hero Member
*****
Offline Offline

Activity: 530


View Profile
December 10, 2012, 09:02:32 PM
 #341

I have a question re Stratum, namely: does Stratum support rollntime? - I skimmed the spec and couldn't see a mention (and apologies if this has already been discussed and my search missed it).

What I'm thinking is this:  a 60GH/s miner (which we have every expectation is going to be pretty run-of-the mill once ASICs hit) is going to consume 14 work units per second, and it's reasonable to assume that they will continue to get faster over time.  Over time, as mining hardware improves, then on low-end hardware the CPU load on the mining host will start to become more significant, particularly if hosting a large farm of ASIC miners.  The USB I/O load may start to become significant, too, particularly if the mining host is actually an old laptop that only does USB 1.

ntime rolling can trivially be implemented in the hardware miner (even if in the microcontroller rather than in the ASIC), and doing so would reduce by almost two orders of magnitude the number of work units that have to be generated by the mining host and communicated to the hardware miners.

Longer term, I imagine that the microcontroller in ASIC miners will roll extranonce itself, but the extra CPU and memory requirements placed on the microcontroller, not to mention the extra firmware complexity, probably means this won't be a popular approach with hardware manufacturers until it becomes unavoidable.

Even once we get to that point, I can imagine that as ASIC cores get faster, and smaller (so one microcontroller potentially driving more of them) then hardware designers might opt for an architecture where ntime is incremented in the ASIC, with the microcontroller only rolling extranonce and calculating the Merkle root, just to reduce the CPU and IO requirements within the hardware miner.

This is all just me thinking aloud, and it might be that we really are convinced that the usefulness of ntime rolling is so far away that it's just not worth thinking about, but I just thought I'd throw it out there....

Thoughts?

roy
1511213889
Hero Member
*
Offline Offline

Posts: 1511213889

View Profile Personal Message (Offline)

Ignore
1511213889
Reply with quote  #2

1511213889
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1511213889
Hero Member
*
Offline Offline

Posts: 1511213889

View Profile Personal Message (Offline)

Ignore
1511213889
Reply with quote  #2

1511213889
Report to moderator
1511213889
Hero Member
*
Offline Offline

Posts: 1511213889

View Profile Personal Message (Offline)

Ignore
1511213889
Reply with quote  #2

1511213889
Report to moderator
1511213889
Hero Member
*
Offline Offline

Posts: 1511213889

View Profile Personal Message (Offline)

Ignore
1511213889
Reply with quote  #2

1511213889
Report to moderator
eleuthria
Legendary
*
Offline Offline

Activity: 1750



View Profile
December 10, 2012, 09:09:34 PM
 #342

I have a question re Stratum, namely: does Stratum support rollntime? - I skimmed the spec and couldn't see a mention (and apologies if this has already been discussed and my search missed it).

What I'm thinking is this:  a 60GH/s miner (which we have every expectation is going to be pretty run-of-the mill once ASICs hit) is going to consume 14 work units per second, and it's reasonable to assume that they will continue to get faster over time.  Over time, as mining hardware improves, the CPU load on low-end hardware hosts will start to become more significant, particularly if hosting a large farm.  The USB I/O load may start to become significant, too, particularly if their host is actually an old laptop that only does USB 1.

ntime rolling can trivially be implemented in the hardware miner (even if in the microcontroller rather than in the ASIC), and doing so would reduce by almost two orders of magnitude the number of work units that have to be generated by the host and communicated to the miners.

Longer term, I imagine that the microcontroller in ASIC miners will roll extranonce itself, but the extra CPU and memory requirements placed on the microcontroller, not to mention the extra firmware complexity, probably means this won't be a popular approach with hardware manufacturers until it becomes unavoidable.

Even once we get to that point, I can imagine that as ASIC cores get faster, and smaller (so one MCU potentially driving more of them) then hardware designers might opt for an architecture where ntime is incremented in the ASIC, with the MCU only rolling extranonce and calculating the Merkle root, just to reduce the CPU and IO requirements within the hardware miner.

This is all just me thinking aloud, and it might be that we really are convinced that the usefulness of ntime rolling is so far away that it's just not worth thinking about, but I just thought I'd throw it out there....

Thoughts?

roy

Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit (by default, it can be increased and the spec supports it very easily).  However, Stratum also does support ntime rolling, the official spec is that ntime should not be increased faster than real-time.

nTime rolling is a hack solution to extend getworks, and if used improperly can be extremely bad for the network if a large number of blocks were produced and pushed the mean-time of recent blocks into the future.

RIP BTC Guild, April 2011 - June 2015
Roy Badami
Hero Member
*****
Offline Offline

Activity: 530


View Profile
December 10, 2012, 09:22:26 PM
 #343

Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit

Yes, but it increases the size of the work unit at the expense of complexity.  Incrementing ntime extranonce and computing a new Merkle root is something that probably can't easily done in a hardware miner without increasing hardware cost, hence requiring work (and communication) by the host that the hardware miner is attached to everytime the nonce wraps.  On the other hand, ntime rolling could trivially be done in hardware (in the MCU, or even in future designs in the ASIC) massively reducing the work done by the host.

My thought is that you would do both: you'd incrememnt the nonce, naturally (this is done in hardware in the FPGA or ASIC).  Then when the nonce wraps round, you increment ntime (which could be done in the hardware miner's microcontroller, or longer term, in the ASIC).  Once ntime has been rolled by some configurable amount (30 seconds, 60 seconds, whatever), then ntime is reset, the host PC incremements extranonce, calculates a new work unit, and gives this work unit to the hardware miner.

Bottom line: the work unit for the protocol may be 4.2 billion times larger, but the work unit for the hardware miner will probably be unchanged in the short term.  ntime rolling is a far easier step for the hardware to take then full merkle branch support in the hardware.
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
December 10, 2012, 09:29:39 PM
 #344

Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit (by default, it can be increased and the spec supports it very easily).  However, Stratum also does support ntime rolling, the official spec is that ntime should not be increased faster than real-time.

nTime rolling is a hack solution to extend getworks, and if used improperly can be extremely bad for the network if a large number of blocks were produced and pushed the mean-time of recent blocks into the future.

Stratum protocol don't need to mention ntime rolling, because block header for hashing is produced directly on the miner, not by the pool. But re: Roy's concern about ASIC miners - yes, ntime rolling in ASICs will be still possible with Stratum.

-ck
Moderator
Legendary
*
Offline Offline

Activity: 2338


Ruu \o/


View Profile WWW
December 10, 2012, 09:40:37 PM
 #345

Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit (by default, it can be increased and the spec supports it very easily).  However, Stratum also does support ntime rolling, the official spec is that ntime should not be increased faster than real-time.

nTime rolling is a hack solution to extend getworks, and if used improperly can be extremely bad for the network if a large number of blocks were produced and pushed the mean-time of recent blocks into the future.

Stratum protocol don't need to mention ntime rolling, because block header for hashing is produced directly on the miner, not by the pool. But re: Roy's concern about ASIC miners - yes, ntime rolling in ASICs will be still possible with Stratum.
Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Roy Badami
Hero Member
*****
Offline Offline

Activity: 530


View Profile
December 10, 2012, 09:44:21 PM
 #346

Stratum protocol don't need to mention ntime rolling, because block header for hashing is produced directly on the miner, not by the pool. But re: Roy's concern about ASIC miners - yes, ntime rolling in ASICs will be still possible with Stratum.

Hmm, are you saying that the need with getwork for the pool to explicitly permit rollntime (and to tell the miner how much it's allowed to roll ntime) is just a quirk of how traditional pools generate work?  And that the assumption is that a Stratum pool will just accept any valid block, regardless of the value of the ntime field within the block header (within reason)?

roy
Roy Badami
Hero Member
*****
Offline Offline

Activity: 530


View Profile
December 10, 2012, 09:47:02 PM
 #347

Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.

ETA: if cgminer running on my EEEPC 701 driving a couple of ASIC miners can keep the CPU load low enough that the CPU fan doesn't come on, I'd be very happy.  My guess is that it would need the miner to roll ntime to accomplish this.
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2338


Ruu \o/


View Profile WWW
December 10, 2012, 09:53:01 PM
 #348

Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.
Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Roy Badami
Hero Member
*****
Offline Offline

Activity: 530


View Profile
December 10, 2012, 10:02:52 PM
 #349

Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Yeah, OK, you're right.  So assuming (for ease of maths) 2^10 transaction per block, you need 10 hashes per Merkle root, so you need a host CPU capable of 3.75kH/s.  Hell, my first gen EEEPC would do that more than comfortably (although it wouldn't surprise me if that's 10% CPU or so - I'm sure enough to make the CPU fan come on, even if only at low speed Smiley

Ok, so you've convinced me, we've got a little time before ASICs are fast enough to worry about this.... :-)

roy

ETA: But in 2 years time, when we have $99 600GH/s coffee warmers, then I'm going to be asking for rollntime support :-)
kano
Legendary
*
Offline Offline

Activity: 2268


Linux since 1997 RedHat 4


View Profile
December 10, 2012, 11:21:51 PM
 #350

Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Yeah, OK, you're right.  So assuming (for ease of maths) 2^10 transaction per block, you need 10 hashes per Merkle root, so you need a host CPU capable of 3.75kH/s.  Hell, my first gen EEEPC would do that more than comfortably (although it wouldn't surprise me if that's 10% CPU or so - I'm sure enough to make the CPU fan come on, even if only at low speed Smiley

Ok, so you've convinced me, we've got a little time before ASICs are fast enough to worry about this.... :-)

roy

ETA: But in 2 years time, when we have $99 600GH/s coffee warmers, then I'm going to be asking for rollntime support :-)
roll-n-time is only needed to change the data being hashed when you run out of the nonce range with getwork

Stratum uses the coinbase transaction to resolve this, so the n-time field is simply set to whatever the current time is.

For 1 second of time (n-time=now) you can generate an amount of work greater than ever will be possible to be done in 1 second by any device and thus there is no need to roll the n-time field

As devices hash faster, you simply increase the size of the 'secondary nonce' field in the coinbase

From the point of view of CPUs not being able to hash enough merkle trees - of course as hashing devices get faster you can also expect CPUs to get faster - but implementing something 'like Stratum' in the MCU is not really that big an issue and will resolve it anyway.

Of course at some point in the FAR distant future this will pose a problem, since cycling the 'secondary nonce' in the coinbase does not guarantee to generate unique block headers to hash - the merkle root generated from the transactions is a hash function and thus can produce duplicate output for different input - since the input size is of course MUCH greater than the output size
In fact there is a very small chance of people already hashing the same block header twice - but it is extremely small - the only way around that is how I've suggested increasing the size of the nonce field in the block header - that was rejected

Edit: of course this FAR DISTANT FUTURE problem is present (and exactly the same) in GBT also

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2338


Ruu \o/


View Profile WWW
December 10, 2012, 11:44:17 PM
 #351

Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Yeah, OK, you're right.  So assuming (for ease of maths) 2^10 transaction per block, you need 10 hashes per Merkle root, so you need a host CPU capable of 3.75kH/s.  Hell, my first gen EEEPC would do that more than comfortably (although it wouldn't surprise me if that's 10% CPU or so - I'm sure enough to make the CPU fan come on, even if only at low speed Smiley

Ok, so you've convinced me, we've got a little time before ASICs are fast enough to worry about this.... :-)

roy

ETA: But in 2 years time, when we have $99 600GH/s coffee warmers, then I'm going to be asking for rollntime support :-)
You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
P_Shep
Legendary
*
Offline Offline

Activity: 1106


View Profile WWW
December 10, 2012, 11:51:44 PM
 #352

Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.
Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

That's not quite all the processing though.

I don't know if you send a target difficulty to the ASIC to work on, but at the moment FPGA's work at difficulty one, don't they?
So cgminer has to take these diff 1 nonces and rehash them to find their actual difficulty, and compare them to the target.
If ASICs don't have a settable target, it could add quite a bit more of a load to the CPU.
eleuthria
Legendary
*
Offline Offline

Activity: 1750



View Profile
December 10, 2012, 11:55:55 PM
 #353

Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.
Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

That's not quite all the processing though.

I don't know if you send a target difficulty to the ASIC to work on, but at the moment FPGA's work at difficulty one, don't they?
So cgminer has to take these diff 1 nonces and rehash them to find their actual difficulty, and compare them to the target.
If ASICs don't have a settable target, if could add quite a bit more of a load to the CPU.

Validating the difficulty of a solution should only require 2 SHA256 passes and a simple evaluation to determine difficulty.  It would be significantly less work than preparing the work to be sent to the ASIC in the first place.

RIP BTC Guild, April 2011 - June 2015
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2338


Ruu \o/


View Profile WWW
December 11, 2012, 12:02:28 AM
 #354

Early on I spoke to some of the ASIC-to-be manufacturers and suggested they do target testing for higher diffs internally as well, so I expect at least some of them to take a target as well That and as Eleuthria said, it's still insignificant CPU to test them anyway.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
kano
Legendary
*
Offline Offline

Activity: 2268


Linux since 1997 RedHat 4


View Profile
December 11, 2012, 12:19:01 AM
 #355

Early on I spoke to some of the ASIC-to-be manufacturers and suggested they do target testing for higher diffs internally as well, so I expect at least some of them to take a target as well That and as Eleuthria said, it's still insignificant CPU to test them anyway.
... and hopefully every ASIC that does this will also return the amount of work actually done inside the device ...

I hate estimates Tongue

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
Roy Badami
Hero Member
*****
Offline Offline

Activity: 530


View Profile
December 11, 2012, 08:44:40 PM
 #356

Early on I spoke to some of the ASIC-to-be manufacturers and suggested they do target testing for higher diffs internally as well, so I expect at least some of them to take a target as well That and as Eleuthria said, it's still insignificant CPU to test them anyway.

I'd just taken that as a given - it would seem to be madness not to do that in the ASIC.
Roy Badami
Hero Member
*****
Offline Offline

Activity: 530


View Profile
December 11, 2012, 08:52:12 PM
 #357

You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?

Why only once a second?  An ASIC is going to run through all nonces in far less than a second - why can't it roll ntime rapidly at least for a few seconds?  No one cares if ntime is exact to the second, as long as it's roughly right.
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
December 11, 2012, 09:17:43 PM
 #358

No one cares if ntime is exact to the second, as long as it's roughly right.

ntime should reflect current time. If there's another way of creating unique data header for hashing, it should be prioritized over ntime rolling.

However ntime rolling is still possible, so don't worry, everything will be fine :-).

-ck
Moderator
Legendary
*
Offline Offline

Activity: 2338


Ruu \o/


View Profile WWW
December 11, 2012, 10:12:27 PM
 #359

You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?

Why only once a second?  An ASIC is going to run through all nonces in far less than a second - why can't it roll ntime rapidly at least for a few seconds?  No one cares if ntime is exact to the second, as long as it's roughly right.
You still haven't understood? To go through all the nonces with stratum currently as is, would require it to check 18,446,744,073,709,551,616 hashes. If that's not enough, you can just increase the size of nonce2 from 4 to 6 and get 1,208,925,819,614,629,174,706,176 hashes. This is without rolling the ntime at all. If that's not enough, rolling the ntime just once every second will give you 1,208,925,819,614,629,174,706,176 hashes per second.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
eleuthria
Legendary
*
Offline Offline

Activity: 1750



View Profile
December 11, 2012, 10:38:40 PM
 #360

You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?

Why only once a second?  An ASIC is going to run through all nonces in far less than a second - why can't it roll ntime rapidly at least for a few seconds?  No one cares if ntime is exact to the second, as long as it's roughly right.
You still haven't understood? To go through all the nonces with stratum currently as is, would require it to check 18,446,744,073,709,551,616 hashes. If that's not enough, you can just increase the size of nonce2 from 4 to 6 and get 1,208,925,819,614,629,174,706,176 hashes. This is without rolling the ntime at all. If that's not enough, rolling the ntime just once every second will give you 1,208,925,819,614,629,174,706,176 hashes per second.

The argument (I think) is that you'd have to rebuild the merkleroot with each extranonce, which requires extra work to either be done on the CPU, or on-board future ASICs (would be recommended, due to latency of relaying new work between the miner and the ASIC), meanwhile with ntime rolling there is no extra building of work, just increasing a secondary counter in the header.

While this is true, I think a better rebuttal would be:
An average piece of work would only take ~10 SHA256 hashes to rebuild the merkleroot.  The # of transactions in a block has to double to require each extra hash to build the merkleroot.  Those ~10 SHA256 hashes produce 4.2 billion hashes worth of work.  So if the chip implements nonce rolling internally, it would only decrease effective speed by ~0.000000238%.

It really all comes down to how ASICs are designed.  If they ever reach the 1 TH/s point in a single chip/set of chips (1 device as the miner software would see it), it could be an issue due to needing 250 pieces of work per second.  A 1 ms latency of providing new work would mean the device is idle roughly 20% of the time since.  Of course, I assume by that point the ASIC hardware would have more internal support for mining protocols to handle extranonce rolling+ntime.

RIP BTC Guild, April 2011 - June 2015
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [18] 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!