[ANN] Stratum mining protocol - ASIC ready

[-ck]

Thanks. However what if it's the first command after the socket is opened?

Why there should be any exception for first message?

If the miner knows any recent session id, it should send mining.resume(id) firstly. If miner don't have any session id, it should send mining.subscribe(). When the pool doesn't understand mining.resume message, it should respond with standard error, not by closing the connection.

Is there really any pool which closes the connection on unknown message?

I was mistaken. I checked with Eleuthria and he said his pool simply doesn't respond to messages it doesn't recognise. This is a problem for me since I would have to wait for a response to time out and that would not be graceful or efficient. Perhaps for important auth messages I should let cgminer time out relatively quickly like 5 seconds since it really shouldn't take the pool longer than that to respond.

[slush]

It will be better to ask Eleuthria to add standard error messages for unknown methods. It's actually pretty easy and it will make the life easier.

Auth messages are a bit sensitive, because pool may need database lookup. Especially during the server restart authorization messages may take longer time than usual, because of connection storms.

[DrHaribo]

Is it possible to use the existing unique subscription ID as the session ID with mining.resume instead?

Sounds good to me, mine is just hardcoded to "1" at the moment. I'll make it unique when it is actually used for something.

Pool should not close the connection on unknown message.

Can this go in the docs?

As well as mining.resume and mining.identify or whatever the "which client version are you" message is called.

[-ck]

Is it possible to use the existing unique subscription ID as the session ID with mining.resume instead?

Sounds good to me, mine is just hardcoded to "1" at the moment. I'll make it unique when it is actually used for something.

We had a chat about this on IRC and agreed this would be the most seamless conversion without breaking existing clients. If the client gets a different id or enonce1 they assume it didn't resume.

[techwtf]

Any chance to have the functionality like "X-Switch-To" in getwork protocol extension?
this may allow pool operators do some maintenance work & ask the miners to migrate to a fallback server.

I'm not sure if similar idea is asked before.

[eleuthria]

Any chance to have the functionality like "X-Switch-To" in getwork protocol extension?
this may allow pool operators do some maintenance work & ask the miners to migrate to a fallback server.

I'm not sure if similar idea is asked before.

This was actually suggested by me on the very first reply to the thread.  I'm not sure if we'll see it standardized though.

[slush]

There's mining.reconnect(). It should work in proxy, poclbm and cgminer too.

[techwtf]

There's mining.reconnect(). It should work in proxy, poclbm and cgminer too.

thank you and eleuthria I didnt come up with "reconnect" so it is ignored

[DrHaribo]

There's mining.reconnect(). It should work in proxy, poclbm and cgminer too.

Is this an extension? Where are those documented?

[slush]

They're documented in not-yet-written BIP 40/41 O:-). I recommend to check stratum proxy as a reference implementation: https://github.com/slush0/stratum-mining-proxy/blob/master/mining_libs/client_service.py

[-ck]

I've implemented the modified protocol for reconnection and it's in the current cgminer master git. Not sure if anyone's interested in testing it as you all could be too busy fapping to ASIC hashrates right about now. Either way it should work in a way that doesn't break existing pool stratum support or slush's proxy.

[slush]

Because Github no longer provides "Download section", I moved Windows binaries of Stratum Proxy here: https://mining.bitcoin.cz/media/download/mining_proxy.exe

This URL will always serve the latest Windows binary. I tried to update links on websites to this new URL, but please let me know if you'll find old link to Github download section somewhere.

[kano]

Because Github no longer provides "Download section", I moved Windows binaries of Stratum Proxy here: https://mining.bitcoin.cz/media/download/mining_proxy.exe

This URL will always serve the latest Windows binary. I tried to update links on websites to this new URL, but please let me know if you'll find old link to Github download section somewhere.

If you wish to see the old downloads you put in your git then just add "/downloads/" to your git address.
GutHub has disabled being able to add new downloads and will delete all download files soon.

[slush]

If you wish to see the old downloads you put in your git then just add "/downloads/" to your git address.
GutHub has disabled being able to add new downloads and will delete all download files soon.

I know that "/download" trick, I just wanted to make the announcement simple :-). But the important part is highlighted.

[kano]

If you wish to see the old downloads you put in your git then just add "/downloads/" to your git address.
GutHub has disabled being able to add new downloads and will delete all download files soon.

I know that "/download" trick, I just wanted to make the announcement simple :-). But the important part is highlighted.

I solved it by creating a cgminer-binaries git for me

[-ck]

Ok for those not watching the discussion on #stratum on IRC, I believe we have a final protocol for mining resume which should not break any clients or pools.

It was decided that the parameters could include both the user agent and the session ID as the first and second parameters. If this fails cgminer/other clients should resort to sending blank set of parameters as previously.

The session Id, it was decided, would be encoded along with the notification message "mining.notify" as a member of the 3 deep array parameters returned for "result" by the pool.

See the following for an example of an initial connect followed by a reconnect.

Code:

 [2013-02-25 10:30:58] SEND: {"id": 0, "method": "mining.subscribe", "params": ["cgminer/2.10.5"]}
 [2013-02-25 10:30:58] RECVD: {"result": [[["mining.notify", "02000000b507a8fd1ea2b7d9cdec867086f6935228aba1540154f83930377ea5a2e37108"], ["mining.set_difficulty", "02000000b507a8fd1ea2b7d9cdec867086f6935228aba1540154f83930377ea5a2e371082"]], "02000000", 4], "id": 0, "error": null}
 [2013-02-25 10:30:58] Pool 0 stratum session id: 02000000b507a8fd1ea2b7d9cdec867086f6935228aba1540154f83930377ea5a2e37108
 [2013-02-25 10:30:58] Pool 0 confirmed mining.subscribe with extranonce1 02000000 extran2size 4

 [2013-02-25 10:33:00] SEND: {"id": 2, "method": "mining.subscribe", "params": ["cgminer/2.10.5", "02000000b507a8fd1ea2b7d9cdec867086f6935228aba1540154f83930377ea5a2e37108"]}
 [2013-02-25 10:33:00] RECVD: {"result": [[["mining.notify", "02000000c33cfaa37a964c2ba76c78b99dc170a1b1fe7a5fe025f72e89afba7fc6f23d0e"], ["mining.set_difficulty", "02000000c33cfaa37a964c2ba76c78b99dc170a1b1fe7a5fe025f72e89afba7fc6f23d0e2"]], "02000000", 4], "id": 2, "error": null}
 [2013-02-25 10:33:00] Pool 0 stratum session id: 02000000c33cfaa37a964c2ba76c78b99dc170a1b1fe7a5fe025f72e89afba7fc6f23d0e
 [2013-02-25 10:33:00] Pool 0 confirmed mining.subscribe with extranonce1 02000000 extran2size 4

Note the session ID has changed between the initial subscription and the resume, BUT the extranonce1 has remained the same. This tells the miner it can (re)submit shares before the disconnection.

Hopefully we will not need to revise this any further.

[eleuthria]

Ok for those not watching the discussion on #stratum on IRC, I believe we have a final protocol for mining resume which should not break any clients or pools.

It was decided that the parameters could include both the user agent and the session ID as the first and second parameters. If this fails cgminer/other clients should resort to sending blank set of parameters as previously.

The session Id, it was decided, would be encoded along with the notification message "mining.notify" as a member of the 3 deep array parameters returned for "result" by the pool.

See the following for an example of an initial connect followed by a reconnect.

Code:

 [2013-02-25 10:30:58] SEND: {"id": 0, "method": "mining.subscribe", "params": ["cgminer/2.10.5"]}
 [2013-02-25 10:30:58] RECVD: {"result": [[["mining.notify", "02000000b507a8fd1ea2b7d9cdec867086f6935228aba1540154f83930377ea5a2e37108"], ["mining.set_difficulty", "02000000b507a8fd1ea2b7d9cdec867086f6935228aba1540154f83930377ea5a2e371082"]], "02000000", 4], "id": 0, "error": null}
 [2013-02-25 10:30:58] Pool 0 stratum session id: 02000000b507a8fd1ea2b7d9cdec867086f6935228aba1540154f83930377ea5a2e37108
 [2013-02-25 10:30:58] Pool 0 confirmed mining.subscribe with extranonce1 02000000 extran2size 4

 [2013-02-25 10:33:00] SEND: {"id": 2, "method": "mining.subscribe", "params": ["cgminer/2.10.5", "02000000b507a8fd1ea2b7d9cdec867086f6935228aba1540154f83930377ea5a2e37108"]}
 [2013-02-25 10:33:00] RECVD: {"result": [[["mining.notify", "02000000c33cfaa37a964c2ba76c78b99dc170a1b1fe7a5fe025f72e89afba7fc6f23d0e"], ["mining.set_difficulty", "02000000c33cfaa37a964c2ba76c78b99dc170a1b1fe7a5fe025f72e89afba7fc6f23d0e2"]], "02000000", 4], "id": 2, "error": null}
 [2013-02-25 10:33:00] Pool 0 stratum session id: 02000000c33cfaa37a964c2ba76c78b99dc170a1b1fe7a5fe025f72e89afba7fc6f23d0e
 [2013-02-25 10:33:00] Pool 0 confirmed mining.subscribe with extranonce1 02000000 extran2size 4

Note the session ID has changed between the initial subscription and the resume, BUT the extranonce1 has remained the same. This tells the miner it can (re)submit shares before the disconnection.

Hopefully we will not need to revise this any further.

Looks like what we discussed, and I will work on updating BTC Guild stratum servers in the future to support unique session IDs so disconnected sessions can resume without any shares completed during the disconnect period.  Also glad you decided on pushing the client software with the subscribe, it feels much better to push the data once at the initial handshake rather than have a ask/response method executed after.

[DrHaribo]

This looks good. Thanks for including the user-agent in there as well.

Note the session ID has changed between the initial subscription and the resume, BUT the extranonce1 has remained the same. This tells the miner it can (re)submit shares before the disconnection.

If the resume fails, how should the server respond? Give an error, or just a new extranonce1?

If the old connection is still there, I guess the server could disconnect it. That means the session ID must be hard to guess so you can't disconnect other people. You must also be very sure that you don't give out the same session ID twice, or you'll have two workers disconnecting each other all day.

It's safer to never resume a connection that still appears up to the server. But I wonder how often it will happen that a client tries to resume before the server discovers that the old connection is dead.

Letting both connections stay seems like a bad idea. They'd both be doing the same hashes in the unlikely event that they both keep hashing.

[-ck]

This looks good. Thanks for including the user-agent in there as well.

Note the session ID has changed between the initial subscription and the resume, BUT the extranonce1 has remained the same. This tells the miner it can (re)submit shares before the disconnection.

If the resume fails, how should the server respond? Give an error, or just a new extranonce1?

If the old connection is still there, I guess the server could disconnect it. That means the session ID must be hard to guess so you can't disconnect other people. You must also be very sure that you don't give out the same session ID twice, or you'll have two workers disconnecting each other all day.

It's safer to never resume a connection that still appears up to the server. But I wonder how often it will happen that a client tries to resume before the server discovers that the old connection is dead.

Letting both connections stay seems like a bad idea. They'd both be doing the same hashes in the unlikely event that they both keep hashing.

I think submitting a new extranonce1 is all that is required. We should view the sessionid submission as a hint only and let the pool decide what to do. You will indeed need unique session IDs for this support to work. When luke-Jr tested it, he found that the old connection was not actually detached before I was requesting a new one with the sessionID. This is not surprising given how difficult it can be to tell that a raw socket is actually broken or not. I suspect that dropping the first connection there is the answer.

[zvs]

so, earlier, I posted a query about stratum causing too much cpu load and jerkiness in computers that didnt have problem with longpolling.  turning back on nagle seems to have solved that for the most part

but, i also noticed this



is minerd just really fast, is cgminer slow, or is stratum slow?  one of them was 2s late