[ANN] Stratum mining protocol - ASIC ready

[Graet]

Ozcoin has a stratum server up for testing
details
https://bitcointalk.org/index.php?topic=14085.1180

[P_Shep]

Just added that to my pools list and it totally messed up cgminer - Ozcoin is the only pool I use to have stratum.
I suspect it's my own doing, as I have a homebrew version of cgminer that I use.
Very strange failure though.
Added the pool through the API, then the API went off line for 30 seconds. thought cgminer crashed, but it came back.
Then the pool mined 457 shares and stopped
The 5s rolling average dropped to one BFL of the 6, but 6/6 devices were still reported. Hash rate on some BFLs dropped to 0, others were some fraction of one BFL.
I saved the config file through the API, then reset cgminer.
The first 2 BFLs where then reported sick, with garbled messages being returned in the comms (reporting 'USY').
The other BFLS were showing low, or no hash rates.
The pool I added was missing the 'stratum+tcp://' part when displayed through the API (when I added it via API the whole string was there).
Stopped cgminer, manually removed the pool from the conf. file, and restarted. Everything back to normal now.

I'm guessing there's some sort of buffer over-run going on somewhere.

[Graet]

Just added that to my pools list and it totally messed up cgminer - Ozcoin is the only pool I use to have stratum.
I suspect it's my own doing, as I have a homebrew version of cgminer that I use.
Very strange failure though.
Added the pool through the API, then the API went off line for 30 seconds. though cgminer crashed but it came back.
Then the pool mined 457 shares and stopped
The 5s rolling average dropped to one BFL of the 6, but 6/6 devices were still reported. Hash rate on some BFLs dropped to 0, others were some fraction of one BFL.
I saved the config file through the API, then reset cgminer.
The first 2 BFLs where then reported sick, with garbled messages being returned in the comms (reporting 'USY').
The other BFLS were showing low, or no hash rates.
The pool I added was missing the 'stratum+tcp://' part when displayed through the API (when I added it via API the whole string was there).
Stopped cgminer, manually removed the pool from the conf. file, and restarted. Everything back to normal now.

I'm guessing there's some sort of buffer over-run going on somewhere.

not sure of the timing of it all but we have had to restart our stratum a couple of times, might explain some of it

[DavinciJ15]

Just added that to my pools list and it totally messed up cgminer - Ozcoin is the only pool I use to have stratum.
I suspect it's my own doing, as I have a homebrew version of cgminer that I use.
Very strange failure though.
Added the pool through the API, then the API went off line for 30 seconds. though cgminer crashed but it came back.
Then the pool mined 457 shares and stopped
The 5s rolling average dropped to one BFL of the 6, but 6/6 devices were still reported. Hash rate on some BFLs dropped to 0, others were some fraction of one BFL.
I saved the config file through the API, then reset cgminer.
The first 2 BFLs where then reported sick, with garbled messages being returned in the comms (reporting 'USY').
The other BFLS were showing low, or no hash rates.
The pool I added was missing the 'stratum+tcp://' part when displayed through the API (when I added it via API the whole string was there).
Stopped cgminer, manually removed the pool from the conf. file, and restarted. Everything back to normal now.

I'm guessing there's some sort of buffer over-run going on somewhere.

not sure of the timing of it all but we have had to restart our stratum a couple of times, might explain some of it

Don't you just love dripping wet, freshly released software?

[Luke-Jr]

not sure of the timing of it all but we have had to restart our stratum a couple of times, might explain some of it

Don't you just love dripping wet, freshly released software?

When/if I implement Stratum or another similar protocol in Eloipool, I'd plan to save sockets across restarts :p

[DavinciJ15]

not sure of the timing of it all but we have had to restart our stratum a couple of times, might explain some of it

Don't you just love dripping wet, freshly released software?

When/if I implement Stratum or another similar protocol in Eloipool, I'd plan to save sockets across restarts :p

Does Eloipool support merged mining?  I have not looked at it yet.

[Luke-Jr]

not sure of the timing of it all but we have had to restart our stratum a couple of times, might explain some of it

Don't you just love dripping wet, freshly released software?

When/if I implement Stratum or another similar protocol in Eloipool, I'd plan to save sockets across restarts :p

Does Eloipool support merged mining?  I have not looked at it yet.

Kindof. It supports the setworkaux and gotwork JSON-RPC pair, which can be used for merged mining.
It's not the simplest thing to setup, though.

My hope going forward is that miners will setup their own merged mining, instead of relying on the pool for it, now that GBT enables this.

[-ck]

Just added that to my pools list and it totally messed up cgminer - Ozcoin is the only pool I use to have stratum.
I suspect it's my own doing, as I have a homebrew version of cgminer that I use.
Very strange failure though.
Added the pool through the API, then the API went off line for 30 seconds. thought cgminer crashed, but it came back.
Then the pool mined 457 shares and stopped
The 5s rolling average dropped to one BFL of the 6, but 6/6 devices were still reported. Hash rate on some BFLs dropped to 0, others were some fraction of one BFL.
I saved the config file through the API, then reset cgminer.
The first 2 BFLs where then reported sick, with garbled messages being returned in the comms (reporting 'USY').
The other BFLS were showing low, or no hash rates.
The pool I added was missing the 'stratum+tcp://' part when displayed through the API (when I added it via API the whole string was there).
Stopped cgminer, manually removed the pool from the conf. file, and restarted. Everything back to normal now.

I'm guessing there's some sort of buffer over-run going on somewhere.

Yeah it seems at the time ozcoin kept the connection open but its own buffer was full and was not accepting anything the miner was trying to send, delaying submission. There currently is a very long timeout for submission of data in cgminer and it probably queued up heaps of shares it was trying to submit in that time and overflowed. I will address that in the future. Even if the connection is open, cgminer will have to assume a pool is as good as dead if it's failing to accept data for some arbitrary period.

[Luke-Jr]

Today I implemented and released to my pool new method called mining.get_transactions(job_id). This call simply dumps transactions used for given job. Thanks to this, miners now have everything needed to reconstruct source block template used by the pool and they can check online if pool isn't doing something nasty.

This doesn't seem to actually work. I'm receiving some transaction data, but then the connection goes silent before the response is complete. By "silent" I mean it remains open and there are TCP keepalives going on, but no more data arrives. Some writebuffer-related bug perhaps?

[slush]

This doesn't seem to actually work. I'm receiving some transaction data, but then the connection goes silent before the response is complete. By "silent" I mean it remains open and there are TCP keepalives going on, but no more data arrives. Some writebuffer-related bug perhaps?

Hrm, works for me. How big response you got when it crashed? And did you received complete response for get_transaction call?

[-ck]

Almost implemented the first working version of GBT within cgminer.

This sort of behaviour tells the story well though:

Code:

 [2012-11-05 22:28:08] Accepted 23c0ba9a Diff 7/1 GPU 2 pool 2
 [2012-11-05 22:28:08] Accepted 163ec800 Diff 11/1 GPU 0 pool 2
 [2012-11-05 22:28:13] Stratum from pool 3 detected new block
 [2012-11-05 22:28:13] Accepted 61bdf4a7 Diff 2/1 GPU 3 pool 2
 [2012-11-05 22:28:13] Stale share detected, submitting as user requested
 [2012-11-05 22:28:14] Accepted 00feebca Diff 257/1 GPU 1 pool 2
 [2012-11-05 22:28:19] Rejected 70b76eb2 Diff 2/1 GPU 1 pool 2 (stale-prevblk)
 [2012-11-05 22:28:19] Rejected 47b723ef Diff 3/1 GPU 3 pool 2 (stale-prevblk)
 [2012-11-05 22:28:20] Rejected 9644eef7 Diff 1/1 GPU 2 pool 2 (stale-prevblk)
 [2012-11-05 22:28:21] Rejected 4cafa99d Diff 3/1 GPU 1 pool 2 (stale-prevblk)
 [2012-11-05 22:28:22] Rejected 093dc4cd Diff 27/1 GPU 0 pool 2 (stale-prevblk)
 [2012-11-05 22:28:23] GBT LONGPOLL from pool 2 requested work restart
 [2012-11-05 22:28:24] Accepted 375a5178 Diff 4/1 GPU 1 pool 2
 [2012-11-05 22:28:25] Accepted 047206f4 Diff 57/1 GPU 3 pool 2
 [2012-11-05 22:28:25] GBT LONGPOLL from pool 2 requested work restart
 [2012-11-05 22:28:30] Accepted 72bee5f8 Diff 2/1 GPU 0 pool 2
 [2012-11-05 22:28:31] Accepted 8e63856d Diff 1/1 GPU 2 pool 2

Now pool 3 here is slush on stratum (ping time 335ms), while pool 2 is EMC on GBT (ping time 225ms).

Note how stratum picks up the block change and notifies me 10 seconds before GBT does. However, this is not the GBT pool simply learning of the block change later, because it starts rejecting my shares as being from the previous block before I even got the longpoll from the GBT pool. Then of course there's a second longpoll with the transactions, which is not an unusual practice.

[Luke-Jr]

Code:

I: {"params": ["b57e", "3cc177a87a3bedf6f6d25d8a09801c4450ebccf3cb5d02a0000002a600000000", "01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff21033b2703062f503253482f04f03b985008", "072f736c7573682f000000000100f2052a010000001976a914e8e6ace10f10ce5ed479c7188c9b4061e53aa90688ac00000000", [], "00000002", "1a0513c5", "50983bf0", true], "id": null, "method": "mining.notify"}
O: {"params": ["b57e"], "id": "txlistb57e", "method": "mining.get_transactions"}
I: {"error": null, "id": "txlistb57e", "result": ["0100000001e45eb5311f14f66eff9351134265e33dbd41daa8110acc5b031e80de38893a65000000006b483045022028837a004078689c927fdc794a1e1a15b90671e90cd77e3f98c1c8776b7a9054022100c5c803954112f0ce183041c38b2e31e38bf91c61c3a5cc2b981b23665e7c1d240121029b497b642311bb84f184c0534d3874182035e77c8b1a48c35d3e9e8a390c93c3ffffffff0200a78c13180000001976a914b4eac5b7bba166b7f8c8cd195d6cb3f4680eede788ac80267241000000001976a9147ff701619769fbd1a12f8d0f440c3cfa50adfefe88ac00000000", ...]}

Why does this job have transactions, but no merkle links? :/

[slush]

Why does this job have transactions, but no merkle links? :/

Heh, thank you for pointing at it. It is bug with caching of transaction list in my implementation. It affects only get_transaction method and only for first template after new block announcement. I'll fix it tomorrow!

[generalfault]

A hopefully quick question, and pardon my ignorance.

in stratum-mining/mining/interfaces.py I see where we see submitted shares, but how do we know if the result is accepted as a share? Is is related to on_submit_block?

[slush]

def on_submit_share(self, worker_name, block_header, block_hash, shares, timestamp, is_valid):

is_valid is boolean indicating that share has been accepted.

[generalfault]

def on_submit_share(self, worker_name, block_header, block_hash, shares, timestamp, is_valid):

is_valid is boolean indicating that share has been accepted.

I am so sorry, I didn't quite ask that right.... how do we know if the share solves/generates a block? i.e. how do we know when we generated a block.

[slush]

I am so sorry, I didn't quite ask that right.... how do we know if the share solves/generates a block? i.e. how do we know when we generated a block.

Oh, this. Yes, it is on_submit_block, as you guessed by self:

on_submit_block(self, is_accepted, worker_name, block_header, block_hash, timestamp):

Share is sent to the bitcoind everytime its share difficulty>network difficulty and result is passed into this method. is_accepted indicates that the solution has been accepted and you "won" a block.

[generalfault]

Slush,

I just want to say THANK YOU! You have done so much for the community. (And I know that thank you's tend to be few and far between.)

I'm using the stratum mining server and it's working very well.
I solomine (slush as backup of course) with about 20 clients (some run bfl kit) and using pool software is a good way to monitor/manage them.
I hacked in the db code to get it reporting. Do you want any contributions to the code? I could put in some db code and a simple setup how-to (since it's a bit confusing.) If not I understand.

[slush]

generalfault, I want to keep stratum-mining as simple as possible, to be still usable just as stratum protocol demonstration. However I'd like to ask you to fork stratum-mining repo and publish your changes there. M0mchil did the same and I'm sending people there when they asked me for some DB layer.

[DrHaribo]

I have read the discussion about separating target/difficulty and work data. I have to say I think this was a mistake as it opens possibilities for abuse. I think something about these issues should go in the docs. Otherwise someone is going to create a naive implementation with some nasty issues.

Issue #1: As Eleuthria mentioned miners can hold back proofs of work. Keep results that are above the target, and submit them later if the difficulty decreases. Keep results that are far below the target. Submit them later if difficulty increases, getting better pay for them. By varying your hashrate between two Stratum pools you can make the two pools raise and lower your difficulty frequently. This enables a kind of Stratum pool hopping.

You can prevent this on the server side by always following the change of difficulty with sending out new work and flushing old work. Effectively negating the separation of diff/target and work data.

Issue #2: As Kano mentioned an evil pool op can steal from you. When the miner sends in a hash that is below target but not too far below it, simply reply with "1. I'm raising the difficulty. 2. your share is rejected due to the new difficulty". This will happen naturally once in a while as there is a race condition inherent in the protocol. But as a pool op you can make it happen on purpose to steal from miners. Then reduce difficulty again, and repeat. Raising difficulty once in a while will also make the miner throw away work it was just about to send in. Lowering the difficulty again could make up for it if the miner has kept old low difficulty shares. But oops, the server is flushing old work to prevent issue #1, so they are no longer valid.

With getwork and getblocktemplate a reject due to the hash being above target means there is a bug in your miner. With Stratum it means "it could be nothing - but maybe the pool op is stealing from you."

While issue #1 is fixable, issue #2 can't be fixed 100% without changing the protocol. The best you can do is to always follow up every difficulty change with quick new work data that flush the old. This should reduce the frequency of the problem. Then just hope the miner won't tell the user too often that you are stealing.

Is it ok to pipeline commands to the client? That is, send the difficulty change and new work (2 RPC calls) in quick succession without waiting for a reply to the first.

That should at least reduce the amount of visible rejects that cause miners to think you are stealing. However it won't fix it 100% and having to flush old work to change the difficulty will also waste some hashpower.