[ANN] Stratum mining protocol - ASIC ready

[Bitsilver.us]

User defined Stratum diff is already available. So I don't see the rush to implement it into a miner.

I do see reasons to implement a difficulty setting in the miner and hope this bounces up the list of priorities for BFGMiner.

[DrHaribo]

I do see reasons to implement a difficulty setting in the miner

What are those reasons?

How would it work? Would it be a suggestion that the server can take under advisement and choose to ignore?

[slush]

Would it be a suggestion that the server can take under advisement and choose to ignore?

I suggested this already, but there was almost no response by miner developers (cgminer). In oposite to mining.resume, mining.suggest is trivial to implement on pool side or it can be ignored if pool doesn't understand it.

[jddebug]

I do see reasons to implement a difficulty setting in the miner

What are those reasons?

I had brought it up with Luke Jr. because I am mining at Eligius Diff 64 and I was having network problems. I'd be dropping to backup pool Diff 1 and then slowly climb to 64 with massive bandwidth usage in the meantime and then a while later I'd be back at Eligius at Diff 1 again with massive bandwidth usage while it made its way back to Diff 64 again. Back and forth with 168GH I was seeing the drawback to having to wait for the pool to figure out my Difficulty and I asked Luke Jr. if I could just set it myself.

Luke Jr. explained thats not possible with Stratum at this time. I have 10TH on order. If I had that hashrate at this time and I was dropping to and from backup pools it  appears I would have a major problem and it would appear the pool would too.

Am I mistaken?

[-ck]

Would it be a suggestion that the server can take under advisement and choose to ignore?

I suggested this already, but there was almost no response by miner developers (cgminer). In oposite to mining.resume, mining.suggest is trivial to implement on pool side or it can be ignored if pool doesn't understand it.

Just tell me what to do and I'd implement it, bugs and all.

[slush]

Basically the idea is that miner remember the last difficulty given by the previous mining session and it sends mining.suggest_difficulty(difficulty) on the beginning of the next session (it may be sent before mining.subscribe or mining.resume, but it should not be a requirement). The response to the call is irrelevant for the miner, but if the pool accept the suggestion in any way, it will send standard mining.set_difficulty with new computed difficulty.

Obviously the difficulty can differ from pool to pool while switching between backup pools, because every pool can have different SPM target . The main goal is to give at least some approximation for the newly connected pool, so it won't start with difficulty 1 for 10 THash/s monster rig...

[jamesg]

Is it OK for a pool to remember a minimum diff per worker and when the worker reconnects set their diff to the remembered value?

The chain of events would be something like:

- mining.subscribe (default diff of the pool is given)
- mining.authorize (this is where the worker would be known and a remembered diff could be determined.)
- if remembered worker diff higher than default send mining.set_difficulty

Thoughts?

[slush]

Thoughts?

This is one of possible solutions. Unfortunately many users use one worker accross all miners, because they were lazy to set up unique worker per miner...

[jamesg]

Thoughts?

This is one of possible solutions. Unfortunately many users use one worker accross all miners, because they were lazy to set up unique worker per miner...

Does the break any of the current rules for stratum?

[slush]

No, except that slow devices using the same worker name as fast ones will have huge difficulty and it will take some time to settle on natural level.

[jddebug]

No, except that slow devices using the same worker name as fast ones will have huge difficulty and it will take some time to settle on natural level.

I use one worker at ozcoin and it sets each rig to a unique difficulty somehow.

[eleuthria]

Is it OK for a pool to remember a minimum diff per worker and when the worker reconnects set their diff to the remembered value?

The chain of events would be something like:

- mining.subscribe (default diff of the pool is given)
- mining.authorize (this is where the worker would be known and a remembered diff could be determined.)
- if remembered worker diff higher than default send mining.set_difficulty

Thoughts?

That is certainly one way a pool can handle it, and one way I've been considering for BTC Guild.  However, I think simple minimum difficulty settings available through the frontend solve problems for large workers.  And at least with my implementation, the pool will adjust very high speed workers to an appropriate difficulty before bandwidth becomes an issue.  They'll see a spike at the very start, but with aggressive difficulty adjustments you can prevent it from being enough to saturate even on a very slow connection (3g/dial-up).

I use one worker at ozcoin and it sets each rig to a unique difficulty somehow.

Stratum by definition is per-connection difficulty.  gigavps's suggestion would only affect pools which try to remember/re-instate difficulty on a after worker authorization.

[slush]

However, I think simple minimum difficulty settings available through the frontend solve problems for large workers.  And at least with my implementation, the pool will adjust very high speed workers to an appropriate difficulty before bandwidth becomes an issue.  They'll see a spike at the very start, but with aggressive difficulty adjustments you can prevent it from being enough to saturate even on a very slow connection (3g/dial-up).

I agree that this will work, too, but it requires that user knows what's the appropriate difficulty for his miners. For the most users difficulty is simply abstract number.

In the end, pools will use some mixed solutions like
a) suggestion provided by mining software
b) minimum difficulty set by the user on pool profile
c) using higher default difficulty for unknown connections (will affect slow miners until the difficulty settle down, but it will help fast miners during startup).

[-ck]

Basically the idea is that miner remember the last difficulty given by the previous mining session and it sends mining.suggest_difficulty(difficulty) on the beginning of the next session (it may be sent before mining.subscribe or mining.resume, but it should not be a requirement). The response to the call is irrelevant for the miner, but if the pool accept the suggestion in any way, it will send standard mining.set_difficulty with new computed difficulty.

Obviously the difficulty can differ from pool to pool while switching between backup pools, because every pool can have different SPM target . The main goal is to give at least some approximation for the newly connected pool, so it won't start with difficulty 1 for 10 THash/s monster rig...

Ok, I can easily implement this.

Moving back to the resume support, since mining.resume breaks your proxy when implemented pool-side with the current format, and there will be lots of miners still using your proxy and this makes the pools not interested in supporting the feature, can I suggest a modification to a use of the mining.subscribe response?

SEND: {"id": 0, "method": "mining.subscribe", "params": []}
RECV: {"error": null, "id": 0, "result": [["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], "f8019658", 4]}

Is it possible to use the existing unique subscription ID as the session ID with mining.resume instead? I know some pools use this for different reasons, so I'm not sure if that's possible, but just a suggestion. Alternatively, can you put the session ID somewhere else in the JSON response that does NOT break your proxy? While I have committed the changes to cgminer git with the existing format, there are no users currently depending on that format so we can and should change it.

[slush]

The first parameter is the rest of original Stratum idea; to be universal overlay protocol with some extended publish/subscribe features. To be honest, there's still no real use for initial meaning of the first parameter and the whole idea seems to be a bit overcomplicated.

So I'm offering to change meaning of the first parameter, let's call it session_id :-). I checked all known implementations (stratum proxy, poclbm, cgminer/bfgminer) and all three seems to completely ignore the first parameter. Using first argument as session_id seems to be fully backward compatible.

Any ideas?

[slush]

...and I'll also fix proxy to ignore any unknown parameters in subscribe notification. It wasn't intentional, I'm sorry for such kind of issue :-/.

[-ck]

The first parameter is the rest of original Stratum idea; to be universal overlay protocol with some extended publish/subscribe features. To be honest, there's still no real use for initial meaning of the first parameter and the whole idea seems to be a bit overcomplicated.

So I'm offering to change meaning of the first parameter, let's call it session_id :-). I checked all known implementations (stratum proxy, poclbm, cgminer/bfgminer) and all three seems to completely ignore the first parameter. Using first argument as session_id seems to be fully backward compatible.

Any ideas?

Well of course I like that (since I suggested it) but the only disadvantage I see is the pool really does not have a flag now in its parameters to tell the mining software whether it supports resume or not, so the only way to find out is if cgminer tries the command first. That's ok too, but the issue there is what the pool does in response to a command it does not recognise: some will drop the connection, and I'm not sure what others do. I need some comments from the pool ops please?

[slush]

Well of course I like that (since I suggested it) but the only disadvantage I see is the pool really does not have a flag now in its parameters to tell the mining software whether it supports resume or not, so the only way to find out is if cgminer tries the command first. That's ok too, but the issue there is what the pool does in response to a command it does not recognise: some will drop the connection, and I'm not sure what others do. I need some comments from the pool ops please?

Pool should not close the connection on unknown message. Like this:

Code:

{"id": 1, "method": "mining.resume", "params": []}
{"error": [-3, "Method 'resume' not found for service 'mining'", null], "id": 1, "result": null}

Server should close connection only when the payload is corrupted (thus there's no way to identify messages and respond with standard error).

[-ck]

Well of course I like that (since I suggested it) but the only disadvantage I see is the pool really does not have a flag now in its parameters to tell the mining software whether it supports resume or not, so the only way to find out is if cgminer tries the command first. That's ok too, but the issue there is what the pool does in response to a command it does not recognise: some will drop the connection, and I'm not sure what others do. I need some comments from the pool ops please?

Pool should not close the connection on unknown message. Like this:

Code:

{"id": 1, "method": "mining.resume", "params": []}
{"error": [-3, "Method 'resume' not found for service 'mining'", null], "id": 1, "result": null}

Server should close connection only when the payload is corrupted (thus there's no way to identify messages and respond with standard error).

Thanks. However what if it's the first command after the socket is opened?

[slush]

Thanks. However what if it's the first command after the socket is opened?

Why there should be any exception for first message?

If the miner knows any recent session id, it should send mining.resume(id) firstly. If miner don't have any session id, it should send mining.subscribe(). When the pool doesn't understand mining.resume message, it should respond with standard error, not by closing the connection.

Is there really any pool which closes the connection on unknown message?