[ANN] aiostratum-proxy (v1.1): fast, multi-coin mining proxy for 1000s of rigs

[wetblanket]

aiostratum-proxy is a new, open source stratum protocol (and altcoin derivatives) mining proxy. Its focus on being extensible/modular is one of its greatest strengths: users can create new aiostratum-proxy coin/algos 'plug-ins' or 'modules' without having to modify aiostratum-proxy itself.

Originally built as a learning exercise to better understand stratum protocol(s) and the mining ecosystem, I now think that others will benefit by making aiostratum-proxy available for everyone to use.

The goal for aiostratum-proxy is to make it the easiest solution for any coin/algorithm pool proxying needs, via the creation of JSONRPC (the stratum base 'language') modules.

More information available at the aiostratum-proxy Github repository.

Current Status:

As mentioned, aiostratum-proxy is new, and is looking for some testing beyond what I've thrown at it. Current coin/algo support is Equihash-only (more to come, given community interest). Constructive feedback is needed and appreciated.

Donations:

There is no dev fee. However, I eat food and need to support those around me - devs need to make money too. Consider it incentive to maintain and continue developing aiostratum-proxy. So if you use this proxy, donate using the addresses below:

• BTC: 1BS4QYAFiya4tsjyvHeY945biKnDj6bRA4
• LTC: LTN1LPGnJjHMKq4DcuQYifQgLfT4Phmn9d
• ETH: 0x4B005e68D323bdABD8eeD1D415117Ff1B57b3EC5
• BTCP: b1CACK65UTwzmHGw2VvyozdPsRLMb8utGLg
• ZCL: t1eoqTqyatJzL2rErW83waZLQHZLKuipMbi

Don't have any of the above? I'll look into getting an address in your choice currency.

Features:

The intention for aiostratum-proxy was to be feature-comparable with existing mining proxies, yet extensible without forking to add support for other coins.

• Configure multiple proxies to run at the same time (ie. mine same/different coins on same/different pools)
• Each proxy configuration supports multiple fallback pools
• Each proxy supports up to 65536 miner connections, each mining a separate nonce space (dependent on coin & miner support)
• Supports plaintext and secure connections (TLS/SSL) for both miner and pool connections
• Easily implement new coin/algorithm JSONRPC 'stratum-like' protocols as dynamically-loaded (via config file), external Python3 modules

Supported Coins/Algos:

• Equihash
• ... it's a bit lonely here, let's add some more maybe?

How you can help:

• Test!
• Create coin/algo modules (submit pull requests)
• Find bugs (File them on Github)
• Donate (see above!)
• Spread the word to coin dev teams, pool operators, other miners
• ... share your ideas for improvement

Requirements & Compatibility:

aiostratum-proxy is written using Python3's non-blocking async/await syntax - so Python 3.5 or greater is required.

It runs on Linux and macOS, and should work on Windows (send in reports, please).

Installation:

Read all of the Installation and Usage sections before proceeding with installing.

It's best to always refer to the aiostratum-proxy Github repository for source code and updated installation and usage instructions - but for the eager miners out there... installation is pretty simple:

Code:

pip install aiostratum-proxy

... but you probably really want to use Python virtual environments to contain the installation, rather than installing globally:

Code:

# this will create a directory 'containing' the Python3 virtual environment
python3 -m venv aiostratum_proxy

cd aiostratum_proxy

# this will install the aiostratum-proxy package
bin/pip install aiostratum-proxy

# verify the installation by checking the package version
bin/aiostratum-proxy --version

# view the built-in command-line help
bin/aiostratum-proxy --help

As you can see, installation creates a new command-line shortcut called aiostratum-proxy.

Usage:

A YAML config file is needed for the proxy to run (learn more about YAML); you can generate one:

Code:

bin/aiostratum-proxy --generate-config > proxy-config.yaml

Open and edit the generated proxy-config.yaml in a text editor. To run aiostratum-proxy, pass it your edited config file:

Code:

bin/aiostratum-proxy --config proxy-config.yaml

While you probably should always generate and edit your own config using the above instructions, here's a couple of sample configs for mining Bitcoin Private (BTCP) - to show you how simple they can be. Remember to modify addresses, usernames, passwords to use your own:

Code:

proxies:
- worker_class: aiostratum_proxy.protocols.equihash.EquihashWorkerProtocol
  pool_class: aiostratum_proxy.protocols.equihash.EquihashPoolProtocol

  listen:
  - host: ''
    port: 10666

  pools:
  - host: pool2.btcprivate.org
    port: 3032
    account_name: b1CACK65UTwzmHGw2VvyozdPsRLMb8utGLg
    account_password: x

The above uses the included Equihash support, defines a single unsecured port for miners to connect to and a single pool to proxy to. Simple, but not very secure or robust. Let's try again, adding in secure connections and a fallback pool:

Code:

proxies:
- worker_class: aiostratum_proxy.protocols.equihash.EquihashWorkerProtocol
  pool_class: aiostratum_proxy.protocols.equihash.EquihashPoolProtocol

  listen:
  - host: ''
    port: 10666
  - host: ''
    port: 10667
    ssl: true
    ssl_cert_file: <full path to your self-signed SSL/TLS cert file>
    ssl_cert_key_file: <full path to your self-signed SSL/TLS cert KEY file>

  pools:
  - host: pool2.btcprivate.org
    port: 3032
    account_name: b1CACK65UTwzmHGw2VvyozdPsRLMb8utGLg
    account_password: x
  - host: btcp.suprnova.cc
    port: 6825
    ssl: true
    account_name: wetblanket
    account_password: mypw

The above uses the included Equihash support, defines 2 ports for miners to connect to (1 of them being SSL/TLS secured) and 2 pools to proxy to (the second being the fallback).

Authorization Setting Notes:

Handily, aiostratum-proxy will automatically extract the worker name from each miner's authorization (when it connects) and appends it to the account_name config setting before authorizing with the pool. This allows you to continue per-miner hash/share rate tracking at the pool level. (Thanks Doftorul for the feedback!)

Step by step explanation:

• Miner connects to aiostratum-proxy with randomletters.worker1
• aiostratum-proxy extracts worker1 from the miner's auth
• aiostratum-proxy appends worker1 to the config file pool account_name setting giving us b1CACK65UTwzmHGw2VvyozdPsRLMb8utGLg.worker1
• aiostratum-proxy requests authorization from the pool for b1CACK65UTwzmHGw2VvyozdPsRLMb8utGLg.worker1
• Miner is authorized, stats are tracked at the pool as desired

Note that you CAN optionally put a worker name in the config file pool account_name setting (ie. b1CACK65UTwzmHGw2VvyozdPsRLMb8utGLg.worker1, but this will cause miner-specific worker names to be ignored. You might want this in a larger farm setup, perhaps.

Release Notes:

1.0.2 2018/07/13

• Fixing keepalive issue
• Added debug statement to help detect miner/proxy misconfiguration

1.0.1 2018/07/12

• Adding in worker socket keepalive configuration

1.0.0 2018/03/22

• Initial release

Future Ideas/Thoughts:

This section contains some ideas that may not get implemented - it depends on community feedback.

• Improve documentation
• Consider immediate reply to miner share submissions instead of waiting for pool response
• Complete mining.set_extranonce support
• Consider additional authentication improvements (currently miners aren't authenticated)
  
  • authenticate miners locally within proxy via config
  • authenticate miners via passthru to pool; would require per-pool mappings of username/password for fallback pools in config?
• Add HAProxy PROXY protocol support
• Collect stats (for console display/API purposes)
• Create stats JSON API endpoint
• More coin/algo support
  • Complete Bitcoin/Litecoin/etc stratum module (aiostratum_proxy.protocols.stratum.StratumPoolProtocol & aiostratum_proxy.protocols.stratum.StratumWorkerProtocol)
  • Ethereum (ETH) protocol(s) support?
• ... share your thoughts/ideas

[wetblanket]

reserved for version/release notes

[Keko Fdez]

Really nice to see new stuff coming out specially open source. I have some time tomorrow so might give it a go. Any ideas on how fast it could be on Equihash algo mining ZEC?

[wetblanket]

Really nice to see new stuff coming out specially open source. I have some time tomorrow so might give it a go. Any ideas on how fast it could be on Equihash algo mining ZEC?

Just to make sure we're on the same page here - this is a mining proxy, not the GPU/CPU mining software itself. Its primary purpose is have multiple mining rigs (each rig running mining software like ewbf, dstm, bminer, etc) proxy pool network communication to a) reduce network requirements and b) stabilize share rates.

Given that it is non-blocking by nature and virtually all operations are networking (ie. I/O-related), it should handle a decent number of rig/miner connections without impact, and overall help both miner share rates and reduce pool resources. There have been claims in the past of 20% improvement in shares just by using a pool proxy like this - but keep in mind that there have been a lot of improvements in protocols, miners, and pools in the past couple of years.

Edit: I would say there are benefits to running a proxy like this in a single rig environment as well - when your miner software crashes or restarts, the proxy will maintain the connection to the pool, difficulty, etc. The pool doesn't see your miner drop its connection, which (can?) affects your sharerate stats, whether you're considered a pool jumper, etc.

I'd appreciate testing and feedback!

[Keko Fdez]

Really nice to see new stuff coming out specially open source. I have some time tomorrow so might give it a go. Any ideas on how fast it could be on Equihash algo mining ZEC?

Just to make sure we're on the same page here - this is a mining proxy, not the GPU/CPU mining software itself. Its primary purpose is have multiple mining rigs (each rig running mining software like ewbf, dstm, bminer, etc) proxy pool network communication to a) reduce network requirements and b) stabilize share rates.

Given that it is non-blocking by nature and virtually all operations are networking (ie. I/O-related), it should handle a decent number of rig/miner connections without impact, and overall help both miner share rates and reduce pool resources. There have been claims in the past of 20% improvement in shares just by using a pool proxy like this - but keep in mind that there have been a lot of improvements in protocols, miners, and pools in the past couple of years.

Edit: I would say there are benefits to running a proxy like this in a single rig environment as well - when your miner software crashes or restarts, the proxy will maintain the connection to the pool, difficulty, etc. The pool doesn't see your miner drop its connection, which (can?) affects your sharerate stats, whether you're considered a pool jumper, etc.

I'd appreciate testing and feedback!

yes we are on the same page jeje There have been claims in the past of 20% improvement in shares just by using a pool proxy like this That was the answer I was looking for but my question didn´t come across as obvious as it seemed in my head

[Doftorul]

Great to see a new OSS project and a brave dev making it open source.

In order to get more traction it would be useful to handle worker names transparently, that is presenting to the upstream pool each and every worker name behind the proxy. Sometimes, when you have 20 workers, you'd need to somehow check their hash stats with the pool.

[wetblanket]

Great to see a new OSS project and a brave dev making it open source.

Thanks. Not sure I'll be considered the brightest dev for making it open source, but I'll take 'brave' for now. 🤔

In order to get more traction it would be useful to handle worker names transparently, that is presenting to the upstream pool each and every worker name behind the proxy. Sometimes, when you have 20 workers, you'd need to somehow check their hash stats with the pool.

Good news! Let's get ready for traction! Although not currently documented (my bad, sorry), it already is able to do this. It just requires a minor change in the config. This line of code extracts the worker name from the miner's authorization and appends it to the config file's pool account name, but ONLY IF the config's pool account name doesn't already have a worker name set.

My examples in the original post have ".rig1" in the account_name settings (see the following):

Code:

proxies:
- worker_class: aiostratum_proxy.protocols.equihash.EquihashWorkerProtocol
  pool_class: aiostratum_proxy.protocols.equihash.EquihashPoolProtocol

  listen:
  - host: ''
    port: 10666

  pools:
  - host: pool2.btcprivate.org
    port: 3032
    account_name: b1CACK65UTwzmHGw2VvyozdPsRLMb8utGLg.rig1
    account_password: x

If ".rig1" is removed from that line, aiostratum-proxy will pull the worker name from the miner's authorization - so, like this instead:

Code:

proxies:
- worker_class: aiostratum_proxy.protocols.equihash.EquihashWorkerProtocol
  pool_class: aiostratum_proxy.protocols.equihash.EquihashPoolProtocol

  listen:
  - host: ''
    port: 10666

  pools:
  - host: pool2.btcprivate.org
    port: 3032
    account_name: b1CACK65UTwzmHGw2VvyozdPsRLMb8utGLg
    account_password: x

At least, it should!

I will update the original post to describe this Updated! Thank you for pointing it out.

[wetblanket]

A small update (version 1.0.1) has been released today, to add in in worker connection TCP socket keep-alives.

Thanks to everyone who has donated, to keep the project moving!

[wetblanket]

Version 1.0.2 has just been released: keep-alive fix for non-linux platforms, plus a little help in detecting miner/proxy/pool misconfiguration (via an additional debug statement).

Keep sending me product feedback (via PM or via this thread)!

[wariner]

Great proxy!

works very fine

[xpulse]

Wetblanket,

Could you please help with installation your proxy on CentOs, I added 2 requests in github.

Thanks.

[wetblanket]

Could you please help with installation your proxy on CentOs, I added 2 requests in github.

Hi, thanks for the message - replied to the 'easier' of your two issues already; unfortunately the answer was that it appears you're trying to use Python 3.4, when only Python 3.5 and above is supported.

I will give the ZEC nicehash pool support issue some thought, but I'm pretty sure that using any equihash proxy with nicehash isn't possible due to the way they both take the same approach to supporting multiple miners.

[cat9life]

It's a great proxy! thank you! I am looking for a long time. Gonna test it

[sanjaykumar49]

Hi,

Is this proxy support AutoSwitching to most profitable coin on an equihash  pool, if so how we can configure it?

[wetblanket]

Is this proxy support AutoSwitching to most profitable coin on an equihash pool, if so how we can configure it?

While I haven't tried it, I imagine it would work if you use https://miningpoolhub.com for instance.

[map_ua]

Hi wetblanket! Check your inbox please!

[wetblanket]

New v1.1 release to address a nonce size calculation bug.

As always, product feedback is always appreciated!

[Kirill_83]

Hello
I use antminer s9 with asicboost.
Tell me please. Does aiostratum-proxy support asicboost?
I have installed aiostratum-proxy and asic connected and disconnected

aiostratum-proxy --config proxy-config.yaml
2018-12-12 16:24:04,248     INFO - * Proxy 1 proxy starting
2018-12-12 16:24:04,248     INFO - W:Proxy 1: defaulting to None max workers
2018-12-12 16:24:04,248     INFO - W:Proxy 1: up to 256 workers supported (distinct nonce spaces)
2018-12-12 16:24:04,249     INFO - W:Proxy 1: accepting plaintext connections on 192.168.0.100|10666
2018-12-12 16:24:04,249     INFO - * Proxy 1 proxy started, waiting for worker connections
2018-12-12 16:24:05,404     INFO - P:Proxy 1: connection established to eu.ss.btc.com|1800
2018-12-12 16:24:05,481     INFO - W:Proxy 1: peer connected ('192.168.2.170', 53497)
2018-12-12 16:24:05,482     INFO - W:Proxy 1: peer connected ('192.168.2.170', 53498)
2018-12-12 16:24:05,641     INFO - W:Proxy 1: peer disconnected ('192.168.2.170', 53498)
2018-12-12 16:24:10,642     INFO - W:Proxy 1: peer connected ('192.168.2.170', 53502)
etc

Maybe need to add some parameters to the config?
Thanks.

[chrysophylax]

WetBlanket ...

Would you be interested in continuing the development of the proxy you started?

#crysx