Bitcoin Forum
November 20, 2017, 09:39:12 PM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 [23] 24 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
Author Topic: [ANN] Stratum mining protocol - ASIC ready  (Read 143488 times)
DrHaribo
Legendary
*
Offline Offline

Activity: 2240


Bitminter.com Operator


View Profile WWW
February 11, 2013, 09:55:14 PM
 #441

I believe I saw somewhere a way for a server to ask a client to identify itself (software name and version). Is this something the current crop of clients support?

▶▶▶ Bitminter.com - Your trusted mining pool since 2011.
1511213952
Hero Member
*
Offline Offline

Posts: 1511213952

View Profile Personal Message (Offline)

Ignore
1511213952
Reply with quote  #2

1511213952
Report to moderator
Join ICO Now A blockchain platform for effective freelancing
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1511213952
Hero Member
*
Offline Offline

Posts: 1511213952

View Profile Personal Message (Offline)

Ignore
1511213952
Reply with quote  #2

1511213952
Report to moderator
1511213952
Hero Member
*
Offline Offline

Posts: 1511213952

View Profile Personal Message (Offline)

Ignore
1511213952
Reply with quote  #2

1511213952
Report to moderator
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2338


Ruu \o/


View Profile WWW
February 11, 2013, 09:57:50 PM
 #442

I believe I saw somewhere a way for a server to ask a client to identify itself (software name and version). Is this something the current crop of clients support?

client.get_version

Supported by cgminer.

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
fireduck
Sr. Member
****
Offline Offline

Activity: 366



View Profile
February 14, 2013, 12:23:59 AM
 #443

I have made a java stratum pool server implementation.  It seems to be working pretty well.  Once I pull my passwords and settings out of the source into a config file I'll put it up on github.

Questions:

1) Should a work unit (from a mining.notify message) be usable by a client if the TCP connection breaks and they reconnect?  It makes sense to me that it should, but I am having trouble because the mining.subscribe seems to occur before the mining.authorize so I need to issue (or reissue) the same extranonce1 before I know which user it is.

2) I should be able to support a bunch of authenticated workers on a single TCP connection, right?  This means they are all going to work with the same mining.notify messages and with the same extranonce1 values?

If I do this, it makes me dedup code a little more complex.  If I don't do it, it probably disables some interesting proxy use cases.

I could also see an issue with select-able difficulty pools (like mine) where a user creates a stratum connection with a bunch of workers at different difficulty levels and then when they find a share they submit it as whichever worker has the highest difficulty that it matches, thus greatly increasing their pay at my expense.
kano
Legendary
*
Offline Offline

Activity: 2268


Linux since 1997 RedHat 4


View Profile
February 14, 2013, 01:43:24 AM
 #444

...
Questions:

1) Should a work unit (from a mining.notify message) be usable by a client if the TCP connection breaks and they reconnect?  It makes sense to me that it should, but I am having trouble because the mining.subscribe seems to occur before the mining.authorize so I need to issue (or reissue) the same extranonce1 before I know which user it is.
...
As far as I know, none of the pools support this yet Sad

cgminer currently assumes that a disconnect means throw away the work, since all pools do that.

Be the first! Make the others do it too!

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
eleuthria
Legendary
*
Offline Offline

Activity: 1750



View Profile
February 14, 2013, 02:04:33 AM
 #445

I have made a java stratum pool server implementation.  It seems to be working pretty well.  Once I pull my passwords and settings out of the source into a config file I'll put it up on github.

Questions:

1) Should a work unit (from a mining.notify message) be usable by a client if the TCP connection breaks and they reconnect?  It makes sense to me that it should, but I am having trouble because the mining.subscribe seems to occur before the mining.authorize so I need to issue (or reissue) the same extranonce1 before I know which user it is.

2) I should be able to support a bunch of authenticated workers on a single TCP connection, right?  This means they are all going to work with the same mining.notify messages and with the same extranonce1 values?

If I do this, it makes me dedup code a little more complex.  If I don't do it, it probably disables some interesting proxy use cases.

I could also see an issue with select-able difficulty pools (like mine) where a user creates a stratum connection with a bunch of workers at different difficulty levels and then when they find a share they submit it as whichever worker has the highest difficulty that it matches, thus greatly increasing their pay at my expense.


For #2)  Yes, work with Stratum is defined per connection, so is difficulty.  If people pool multiple workers over one connection, they have to work at whatever the highest difficulty worker is set to.

RIP BTC Guild, April 2011 - June 2015
fireduck
Sr. Member
****
Offline Offline

Activity: 366



View Profile
February 14, 2013, 06:18:15 AM
 #446

I have made a java stratum pool server implementation.  It seems to be working pretty well.  Once I pull my passwords and settings out of the source into a config file I'll put it up on github.

Questions:

1) Should a work unit (from a mining.notify message) be usable by a client if the TCP connection breaks and they reconnect?  It makes sense to me that it should, but I am having trouble because the mining.subscribe seems to occur before the mining.authorize so I need to issue (or reissue) the same extranonce1 before I know which user it is.

2) I should be able to support a bunch of authenticated workers on a single TCP connection, right?  This means they are all going to work with the same mining.notify messages and with the same extranonce1 values?

If I do this, it makes me dedup code a little more complex.  If I don't do it, it probably disables some interesting proxy use cases.

I could also see an issue with select-able difficulty pools (like mine) where a user creates a stratum connection with a bunch of workers at different difficulty levels and then when they find a share they submit it as whichever worker has the highest difficulty that it matches, thus greatly increasing their pay at my expense.


I've put my code on github in case anyone is interested:
https://github.com/fireduck64/SockThing

It seems to work but is certainly not done.  I need to add metrics, probably some job management and clean up the multi-connection same user story a bit.
eleuthria
Legendary
*
Offline Offline

Activity: 1750



View Profile
February 14, 2013, 06:20:44 AM
 #447

I've put my code on github in case anyone is interested:
https://github.com/fireduck64/SockThing

It seems to work but is certainly not done.  I need to add metrics, probably some job management and clean up the multi-connection same user story a bit.


Ahh, the Java, it burns!  Glad to see another fresh implementation of Stratum.  I'll try to take a deeper look at it tomorrow, but so far I must say it's one of the cleaner java-based pool servers I've seen in quite some time.

RIP BTC Guild, April 2011 - June 2015
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
February 14, 2013, 08:01:26 AM
 #448

1) Should a work unit (from a mining.notify message) be usable by a client if the TCP connection breaks and they reconnect?  It makes sense to me that it should, but I am having trouble because the mining.subscribe seems to occur before the mining.authorize so I need to issue (or reissue) the same extranonce1 before I know which user it is.

Please propose some protocol extension before you implement anything in this area. I was thinking about some session identifiers and resuming the connection, but I didn't find any usable (simple and scalable) solution yet. From my perspective this is actually much smaller problem that it seems to be; properly implemented server and client and standard internet connection won't drop the connection for days. And one second of lost work per day is definitely NOT worth all the effort.

kano
Legendary
*
Offline Offline

Activity: 2268


Linux since 1997 RedHat 4


View Profile
February 14, 2013, 09:30:58 AM
 #449

1) Should a work unit (from a mining.notify message) be usable by a client if the TCP connection breaks and they reconnect?  It makes sense to me that it should, but I am having trouble because the mining.subscribe seems to occur before the mining.authorize so I need to issue (or reissue) the same extranonce1 before I know which user it is.

Please propose some protocol extension before you implement anything in this area. I was thinking about some session identifiers and resuming the connection, but I didn't find any usable (simple and scalable) solution yet. From my perspective this is actually much smaller problem that it seems to be; properly implemented server and client and standard internet connection won't drop the connection for days. And one second of lost work per day is definitely NOT worth all the effort.
Except ... for anyone with an unreliable internet connection (happens with wireless and 3g and such) they lose more shares the more often they lose the connection.
Of course if you have a dynamic IP address, you also lose shares during that change, however often that happens - but obviously that's not often.
... pity it's a new problem added by Stratum (that didn't already exist)

Yes in general it is a small number of shares, but it also means that if the pool itself does a restart, everyone loses the work they are working on and can never submit it.
Yes a proper implementation would have to handle a pool restart also - but that is the pools problem not the protocol's problem.

However, the problem is simply that it is hard for the pools to implement, so the pools have shied away from doing it or are unable to do it due to their own design.

Pool: https://kano.is Here on Bitcointalk: Forum BTC: 1KanoPb8cKYqNrswjaA8cRDk4FAS9eDMLU
FreeNode IRC: irc.freenode.net channel #kano.is Majority developer of the ckpool code
Help keep Bitcoin secure by mining on pools with full block verification on all blocks - and NO empty blocks!
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
February 14, 2013, 12:51:00 PM
 #450

properly implemented server and client and standard internet connection won't drop the connection for days.
Except ... for anyone with an unreliable internet connection (happens with wireless and 3g and such) they lose more shares the more often they lose the connection.

Important part highlighted.

Quote
Of course if you have a dynamic IP address, you also lose shares during that change

Why? Providers are dropping TCP connection because of moving the customer to new IP address?

Quote
... pity it's a new problem added by Stratum (that didn't already exist)

A lot of other and more important issues existed before, which are solved by Stratum.

Quote
Yes a proper implementation would have to handle a pool restart also - but that is the pools problem not the protocol's problem.

Ok, adding session identifier to the response of mining.subscribe() is trivial, this identifier can be used by mining.resume() on the beginning of another TCP connection. But you're literally opening Pandora box:

a) Backends must have some backbone channel for retrieving session data of given session identifier
or
b) Backends must use exactly the same template, so session identifier can be the same as extranonce1, but backends must have backbone for validating duplicated submissions.

Both solutions have drastical impact to scalability and both solutions are extremely hard to implement. It also open another doors for attacks (DoS, share double spends). That additional infrastructure is definitely not worth of <0.001% lost shares of users with non-standard mining setup.

I agree with you kano, that it's not perfect. But adding another magnitude of complexity won't help much.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2268



View Profile
February 14, 2013, 12:58:16 PM
 #451

I'd suggest just adding an optional field to the end of mining.submit for extranonce1 that the miner can use for old share submissions on reconnect.
Let the pool decide whether to honour them or not, and for how long.

slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
February 14, 2013, 01:16:27 PM
 #452

I'd suggest just adding an optional field to the end of mining.submit for extranonce1 that the miner can use for old share submissions on reconnect.
Let the pool decide whether to honour them or not, and for how long.

But pool cannot validate the share unless the backend has the same block template (with the same job ID). Yes, we can start adding "optional fields", but that's not the direction of development which I want to go.

If something, then go with mining.resume() and let the pool decide if he'll accept it or not. However I don't think anybody sane will want to implement full mechanism of session resuming.

-ck
Moderator
Legendary
*
Offline Offline

Activity: 2338


Ruu \o/


View Profile WWW
February 14, 2013, 01:29:39 PM
 #453

If something, then go with mining.resume() and let the pool decide if he'll accept it or not. However I don't think anybody sane will want to implement full mechanism of session resuming.
You have certainly expressed your disinterest in implementing it, but that is not the case for all other pool ops. If just one pool op decides to implement this, I will support it in cgminer, and as per always, once one pool has set the standard, it would be crazy for others to not follow. So who's first?

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
February 14, 2013, 02:03:12 PM
 #454

If just one pool op decides to implement this, I will support it in cgminer.

No problem. I just hope that the first daredevil will implement it properly, so response of mining.subscribe will have one extra argument 'session-id':

Quote
{"id": 1, "result": [[], "08000002", 4, "some-session-id"], "error": null}\n

If this parameter is set and non-null, the miner will store sesssion id internally and then, after the crash of the connection, miner will call mining.resume('session-id') instead of mining.subscribe(...). Pool will respond with true/false. If true, miner can continue in the previous session (so re-upload pending shares, ...), if false, miner must call mining.subscribe and throw away pending shares.

Correct?

fireduck
Sr. Member
****
Offline Offline

Activity: 366



View Profile
February 14, 2013, 03:17:24 PM
 #455

1) Should a work unit (from a mining.notify message) be usable by a client if the TCP connection breaks and they reconnect?  It makes sense to me that it should, but I am having trouble because the mining.subscribe seems to occur before the mining.authorize so I need to issue (or reissue) the same extranonce1 before I know which user it is.

Please propose some protocol extension before you implement anything in this area. I was thinking about some session identifiers and resuming the connection, but I didn't find any usable (simple and scalable) solution yet. From my perspective this is actually much smaller problem that it seems to be; properly implemented server and client and standard internet connection won't drop the connection for days. And one second of lost work per day is definitely NOT worth all the effort.

So, I've just made SockThing use the same extranonce1 for all connections.  I am also adding an extra random number to the coinbase and generate a new coinbase for each job.

Now reconnects should work fine, except clients have no way of knowing that they will.

I'm thinking maybe adding a field to mining.subscribe that indicates this feature exists.

However, I don't imagine other pool operators are inclined to build a new coinbase for each user so I don't imagine my solution will be for everyone.
fireduck
Sr. Member
****
Offline Offline

Activity: 366



View Profile
February 14, 2013, 03:21:52 PM
 #456

If just one pool op decides to implement this, I will support it in cgminer.

No problem. I just hope that the first daredevil will implement it properly, so response of mining.subscribe will have one extra argument 'session-id':

Quote
{"id": 1, "result": [[], "08000002", 4, "some-session-id"], "error": null}\n

If this parameter is set and non-null, the miner will store sesssion id internally and then, after the crash of the connection, miner will call mining.resume('session-id') instead of mining.subscribe(...). Pool will respond with true/false. If true, miner can continue in the previous session (so re-upload pending shares, ...), if false, miner must call mining.subscribe and throw away pending shares.

Correct?

That makes sense to me.  I know the ship has long since sailed, but I kinda with these weren't json arrays but instead where key value pairs so that it would be a little more self describing and easier to add various extensions.  Maybe any extensions should be done as pairs rather than tacking values on to keep things clear?
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
February 14, 2013, 03:25:01 PM
 #457

So, I've just made SockThing use the same extranonce1 for all connections.  I am also adding an extra random number to the coinbase and generate a new coinbase for each job.

Why is this better than generating custom extranonce1? Extranonce1 has been designed exactly for this. If you can achieve the same block template on all backends, then you can make session_id==extranonce1 and my proposal will work for you.

slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
February 14, 2013, 03:27:19 PM
 #458

That makes sense to me.  I know the ship has long since sailed, but I kinda with these weren't json arrays but instead where key value pairs so that it would be a little more self describing and easier to add various extensions.  Maybe any extensions should be done as pairs rather than tacking values on to keep things clear?

Then next logical step is to get rid of json entirely and propose protobuf messages. I picked json for good reasons and I still think it was smart choice, but for future development protobuf would be better. If anyone thinks about protocol change, then please take protobuf in account.

fireduck
Sr. Member
****
Offline Offline

Activity: 366



View Profile
February 14, 2013, 03:34:27 PM
 #459

So, I've just made SockThing use the same extranonce1 for all connections.  I am also adding an extra random number to the coinbase and generate a new coinbase for each job.

Why is this better than generating custom extranonce1? Extranonce1 has been designed exactly for this. If you can achieve the same block template on all backends, then you can make session_id==extranonce1 and my proposal will work for you.

Yeah, your proposal is a clean way of doing it.  Mine was just a hack to work within the existing protocol.
-ck
Moderator
Legendary
*
Offline Offline

Activity: 2338


Ruu \o/


View Profile WWW
February 15, 2013, 12:39:30 AM
 #460

If just one pool op decides to implement this, I will support it in cgminer.

No problem. I just hope that the first daredevil will implement it properly, so response of mining.subscribe will have one extra argument 'session-id':

Quote
{"id": 1, "result": [[], "08000002", 4, "some-session-id"], "error": null}\n

If this parameter is set and non-null, the miner will store sesssion id internally and then, after the crash of the connection, miner will call mining.resume('session-id') instead of mining.subscribe(...). Pool will respond with true/false. If true, miner can continue in the previous session (so re-upload pending shares, ...), if false, miner must call mining.subscribe and throw away pending shares.

Correct?
Nice.

If mining.resume is successful, does this mean the miner does not need to do mining.authorize afterwards?

Primary developer/maintainer for cgminer and ckpool/ckproxy.
ZERO FEE Pooled mining at ckpool.org 1% Fee Solo mining at solo.ckpool.org
-ck
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 [23] 24 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!