New stratum/getwork proxy with Web-based GUI and pools management

[Delivereath]

Great work ! Thanks.

How does the getwork protocol look like ? Hard to implement ?

[Pfool]

Lack of spare time...

The share submission does not yet work (but it should soon). Then I will implement the long-polling support.

[jedimstr]

Trying to call the proxy's API.. the API details don't list what the actual syntax of the get and post requests should be.  You just list things like "pool/list" and "user/list" which don't follow JSON based GET or POST requests like the Mining programs (cgminer, bfgminer, etc) use which is something along the lines of {command:"whatYou'reRequesting"}

Do you have an example?

[Pfool]

I will try to improve the API description on the project main page.

The proxy API does not use the same format as the Mining programs one (even if both are JSON formatted).

For GET requests (like pool/list, user/list and connection/list), there is no parameter. For example, just use the URL http://localhost:8888/proxy/pool/list in your browser. The response is a JSON formatted response like:

[{"name":"stratum.nicehash.com:3336","host":"stratum+tcp://stratum.nicehash.com:3336","username":"13whEvPgLWmybVhizY91DtFqSrq2wznriv","password":"p=0.65;
d=0.02","isActive":false,"isEnabled":true,"isStable":false,"isActiveSince":"10-06-14 14:18:57 +0200","difficulty":"0.02",
"extranonce1":"5bfff2543b","extranonce2Size":3,"workerExtranonce2Size":2,"numberOfWorkerConnections":0,"priority":0,"acceptedDifficulty":40.92064000000084,
"rejectedDifficulty":0.067813,"isExtranonceSubscribeEnabled":true,"acceptedHashesPerSeconds":0,"rejectedHashesPerSeconds":0}]

If you use the browser, you can add this kind of plugin to format automatically the JSON response in a more human-readable way: http://jsonview.com/

For POST requests (like pool/priority), the expected parameter is a JSON formatted string. You will have to install a plugin if you want to use your browser. For firefox, you can use this one: https://addons.mozilla.org/en-US/firefox/addon/poster/

The URL to use is http://localhost:8888/proxy/pool/priority. The Content-Type is application/json and the JSON content to POST looks like:
{"poolName": "stratum.nicehash.com:3336", "priority": 0}

[Pfool]

Yet another release: the getwork protocol for miners is now supported.
Getwork has been tested with Scrypt and X11 coins only. If some people want to test some others algorithms, they are welcome  .

In the same way, the scrypt hashrate estimation is now fixed with the --scrypt option.

Binaries: https://github.com/Stratehm/stratum-proxy/releases/tag/0.3.0

[jedimstr]

I will try to improve the API description on the project main page.

The proxy API does not use the same format as the Mining programs one (even if both are JSON formatted).

For GET requests (like pool/list, user/list and connection/list), there is no parameter. For example, just use the URL http://localhost:8888/proxy/pool/list in your browser. The response is a JSON formatted response like:

[{"name":"stratum.nicehash.com:3336","host":"stratum+tcp://stratum.nicehash.com:3336","username":"13whEvPgLWmybVhizY91DtFqSrq2wznriv","password":"p=0.65;
d=0.02","isActive":false,"isEnabled":true,"isStable":false,"isActiveSince":"10-06-14 14:18:57 +0200","difficulty":"0.02",
"extranonce1":"5bfff2543b","extranonce2Size":3,"workerExtranonce2Size":2,"numberOfWorkerConnections":0,"priority":0,"acceptedDifficulty":40.92064000000084,
"rejectedDifficulty":0.067813,"isExtranonceSubscribeEnabled":true,"acceptedHashesPerSeconds":0,"rejectedHashesPerSeconds":0}]

If you use the browser, you can add this kind of plugin to format automatically the JSON response in a more human-readable way: http://jsonview.com/

For POST requests (like pool/priority), the expected parameter is a JSON formatted string. You will have to install a plugin if you want to use your browser. For firefox, you can use this one: https://addons.mozilla.org/en-US/firefox/addon/poster/

The URL to use is http://localhost:8888/proxy/pool/priority. The Content-Type is application/json and the JSON content to POST looks like:
{"poolName": "stratum.nicehash.com:3336", "priority": 0}

Thanks, that helps.  I have a Cocoa REST testing app on my dev Mac that makes developing for web services much easier, so I'll be using that for now to get status from my proxy until you get a chance to put a nice UI on it (if that's still the plan). Thanks!

[Delivereath]

Yet another release: the getwork protocol for miners is now supported.
Getwork has been tested with Scrypt and X11 coins only. If some people want to test some others algorithms, they are welcome  .

In the same way, the scrypt hashrate estimation is now fixed with the --scrypt option.

Binaries: https://github.com/Stratehm/stratum-proxy/releases/tag/0.3.0

Great ! I will try that soon !

Have you tried the proxy with coins that send part of the reward to a defined address (dev or foundation) ? I've seen that some of those coins (summercoin if I remember correctly) needed a stratum modification on the pool side. So I'm wondering if your proxy would work with them.

[Pfool]

Just a precision (to avoid misunderstanding): the proxy is not ready for solo mining. The Getwork implementation is only on the miner side: the proxy is connected on a stratum pool and both stratum and getwork miners are connected on the proxy.

The solo mining will be implemented later (with the getBlockTemplate protocol). It is a really long task to add the getBlockTemplate support. I have not implemented the getwork support on the Pool side since I haven't found solution to do so (I think it is not possible to build stratum jobs from getwork jobs).

I haven't heard about this kind of coins. So it is not tested...(summercoin doesn't seem to send rewards to another address)

[Delivereath]

Ah, I really thought that the getwork feature would allow solo mining. Still, thanks for the work.

[jedimstr]

Another feature request...

Can we get an option to allow HTTP connections (still using stratum and/or getwork LP) versus requiring stratum+tcp://  from specified ports?

Usecase: Allow mining access to the stratum-proxy by workers behind a restrictive firewall/proxy network.  The old scrypt python stratum proxy allowed this through the getwork requests, but if I want to use it for X11, I can't, so this would be a great feature in those cases.  Otherwise these workers are stuck on Scrypt and the old python stratum-proxy.

[Pfool]

Ouch, I don't understand your request...

What does your restrictive firewall/proxy not allow ? Raw TCP connections (stratum+tcp) ? HTTP requests on a non standard port (8332 for Getwork requests) ?

You may change the listening port of Getwork requests with the --getwork-listen-port option (use the port 80 to trick the firewall/proxy). If the long-polling does not work due to some kind of timeout on the firewall/proxy, you can try to disable the long-polling support on your mining software (with a counterpart: a little more rejects).

[jedimstr]

Ouch, I don't understand your request...

What does your restrictive firewall/proxy not allow ? Raw TCP connections (stratum+tcp) ? HTTP requests on a non standard port (8332 for Getwork requests) ?

You may change the listening port of Getwork requests with the --getwork-listen-port option (use the port 80 to trick the firewall/proxy). If the long-polling does not work due to some kind of timeout on the firewall/proxy, you can try to disable the long-polling support on your mining software (with a counterpart: a little more rejects).

I think the getwork listen port option you specified could do the trick...  but here's an example:

Using the old stratum-proxy I'm able to access it from a restrictive network by specifying http://proxyDNSname.com:xxxx (with xxxx being whatever getwork port I set.)

When I tried the same thing with your Java based proxy using X11 and stratum by specifying stratum+tcp://proxyDNSname.com:yyyy (with yyyy being the stratum port), it fails over this network's http proxy (using miner software's proxy settings).  Directly specifying http:// instead with the same port also fails.  With some Pools, I've seen them able to support stratum protocol on http:// urls (regardless of port 80 or something else).  But it seems your proxy only supports stratum+tcp:// for stratum handshake.

The restrictive network allows various specific TCP traffic over port 80 to whatever destination port you want (so things like SSH through the proxy work).  But for some reason specifying stratum+tcp:// always fails to connect on this network.

So I guess I'll have to use getwork and http:// for these specific workers?  I haven't done a git pull in awhile (predating getwork capability in your jar), so I'll try it out later when I get home.

Thanks!

UPDATE:  Yes, doing things through getwork bypasses the "stratum over http proxy" issues I was having.  Thanks!!!

Are there plans on putting a TUI in front of this (sorta like bfgminer/cgminer/sgminer), or just a separate application making requests via the API?

Maybe a MobileApp or Front-end app for the UI?  I've been looking for a side project to use for learning Apple Swift, so maybe a monitoring app pointing to your REST API would be a good one.

[cayars]

I've just started playing with this and really like it a lot.

I have however found it doesn't seem to work with any CryptoNight based stratum servers.  I'm sure the problem is on the server side and not doing proper protocols but it might be something you could code around to make it work.  Here a part of the log

2014-06-16 17:40:42,854 INFO  [main]: [Launcher]    LogLevel not set, using INFO.
2014-06-16 17:40:42,880 INFO  [main]: [Launcher]    Using pools: [Pool [name=mro.extremehash.com:9999, host=mro.extremehash.com:9999, uri=null, username=47VvxGyL9D9a33PULnPSVvVqVvTfox1fS3E6uBbFX2NRQWb48nBq8caDK1FsMa4Sr69EeETZP6W7BifUEgWZ8FQgGpz4TLn, password=password, difficulty=null, extranonce1=null, extranonce2Size=null, activeSince=null, isActive=false, isEnabled=true, isExtranonceSubscribeEnabled=false, numberOfSubmit=1, priority=0, acceptedDifficulty=0.0, rejectedDifficulty=0.0]].
2014-06-16 17:40:43,279 INFO  [main]: [StratumProxyManager]    ServerSocket opened on /192.168.1.6:6005.
2014-06-16 17:40:48,186 WARN  [TimerExecutorThread-0]: [Pool]    Subscribe response timeout. Stopping the pool
2014-06-16 17:40:48,187 WARN  [TimerExecutorThread-0]: [StratumProxyManager]    Pool mro.extremehash.com:9999 is DOWN. Moving connections to another one.
2014-06-16 17:40:48,196 INFO  [TimerExecutorThread-0]: [Pool]    Pool mro.extremehash.com:9999 stopped.
2014-06-16 17:40:48,198 INFO  [TimerExecutorThread-0]: [Pool]    Trying reconnect of pool mro.extremehash.com:9999 in 5.0 seconds.
2014-06-16 17:40:53,199 INFO  [TimerExecutorThread-0]: [Pool]    Trying reconnect of pool mro.extremehash.com:9999...
2014-06-16 17:40:53,312 ERROR [Pool-mro.extremehash.com:9999-Thread]: [StratumConnection]    JSON-RPC Parsing error with line: {"id":1,"jsonrpc":"2.0","error":{"code":-1,"message":"invalid method"}}
com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.util.ArrayList out of START_OBJECT token
 at [Source: java.io.StringReader@8bbe03; line: 1, column: 24] (through reference chain: strat.mining.stratum.proxy.json.JsonRpcResponse["error"])
   at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:164)
   at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:691)
   at com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:685)
   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.handleNonArray(CollectionDeserializer.java:256)
   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:214)
   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:204)
   at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:23)
   at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:525)
   at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:99)
   at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:242)
   at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:118)
   at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2993)
   at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2098)
   at strat.mining.stratum.proxy.network.StratumConnection.onLineRead(StratumConnection.java:220)
   at strat.mining.stratum.proxy.network.StratumConnection.access$200(StratumConnection.java:55)
   at strat.mining.stratum.proxy.network.StratumConnection$1.run(StratumConnection.java:152)
2014-06-16 17:40:58,221 WARN  [TimerExecutorThread-0]: [Pool]    Subscribe response timeout. Stopping the pool
2014-06-16 17:40:58,221 WARN  [TimerExecutorThread-0]: [StratumProxyManager]    Pool mro.extremehash.com:9999 is DOWN. Moving connections to another one.

[Pfool]

Indeed, this proxy does not support CryptoNight based stratum servers. To work properly with CryptNight coins, it seems that the stratum protocol has been modified. Unfortunately, I do not know where to find the specifications of this new protocol.

If you can provide me the specifications (or at least more information), I may add the CryptoNight pool mining feature.

[cayars]

Indeed, this proxy does not support CryptoNight based stratum servers. To work properly with CryptNight coins, it seems that the stratum protocol has been modified. Unfortunately, I do not know where to find the specifications of this new protocol.

If you can provide me the specifications (or at least more information), I may add the CryptoNight pool mining feature.

Here is the Github
https://github.com/zone117x/node-cryptonote-pool

[Pfool]

I have checked the code of the node-cryptonode-pool and even if the protocol is called stratum, it is not the stratum protocol that is used. All JSON RPC methods are different from the "real" stratum protocol.

So, heavy developments are needed to add compatibility with cryptonote pools. I put it in the TODO list, but I cannot tell you when it will be done.

[Boffinboy]

Just wanted to say thanks for this, it's a great help when mining on Nvidia. Do you have a donation address (BTC one in sig?). I would like to buy you a beer!

[Pfool]

Just wanted to say thanks for this, it's a great help when mining on Nvidia. Do you have a donation address (BTC one in sig?). I would like to buy you a beer!

Yeah! Thanks for the beer! I am going to need it for this evening (I am going to start working on a GUI for the proxy).

Cheers! 

(You can use the address in sig)

[cayars]

Just wanted to say thanks for this, it's a great help when mining on Nvidia. Do you have a donation address (BTC one in sig?). I would like to buy you a beer!

Yeah! Thanks for the beer! I am going to need it for this evening (I am going to start working on a GUI for the proxy).

Cheers! 

(You can use the address in sig)

What language are you going to use?  Windows, Linux or web based?

[Pfool]

Just wanted to say thanks for this, it's a great help when mining on Nvidia. Do you have a donation address (BTC one in sig?). I would like to buy you a beer!

Yeah! Thanks for the beer! I am going to need it for this evening (I am going to start working on a GUI for the proxy).

Cheers! 

(You can use the address in sig)

What language are you going to use?  Windows, Linux or web based?

It will be web based, directly embedded in the proxy (so no other application to deploy. I am considering two frameworks: GWT (with the GXT library) since I already know it, or maybe Twitter Bootstrap (with JQuery or AngularJS).