Bitcoin Forum

Bitcoin => Mining support => Topic started by: riverdip on May 15, 2019, 07:46:52 PM



Title: Custom Mining Proxy ?
Post by: riverdip on May 15, 2019, 07:46:52 PM
hey i have lots of miners and sometimes it just paid to manage them all.

i was thinking if i could buy my own domain and run proxy that routes all mining to any server of my choice.

LIKE THIS:

mydomain.com -> kanopool
mydomain.com -> slushpool

so if i have to change mining pool i just chance forward url on my domain settings and it should be done for all machines.


UPDATE - 29 MAY - Too many talkies providing alternative solution, but no working solution yet !

something like this -> https://github.com/pooler/rollproxy

but with modern protocols and conditional pool change features


Title: Re: Custom Mining Proxy ?
Post by: HagssFIN on May 15, 2019, 07:53:47 PM
Have you considered using the Awesome Miner -software for managing your miners?

https://bitcointalk.org/index.php?topic=676942


Title: Re: Custom Mining Proxy ?
Post by: riverdip on May 15, 2019, 10:40:46 PM
Have you considered using the Awesome Miner -software for managing your miners?

https://bitcointalk.org/index.php?topic=676942

no, but i heard of it, i have mixed miners. bitmain ebang avalon, some tools work some don't this is why i seek a universal solution.


Title: Re: Custom Mining Proxy ?
Post by: kano on May 15, 2019, 11:32:14 PM
...
i was thinking if i could buy my own domain and run proxy that routes all mining to any server of my choice.

LIKE THIS:

mydomain.com -> kanopool
mydomain.com -> slushpool

so if i have to change mining pool i just chance forward url on my domain settings and it should be done for all machines.
The miners wont change where they are mining due to a DNS change, without a reconnect.

Instead use a standard mining proxy - and the proxy gets work from the pools you tell it to. When you change the proxy settings, they all (have to) reconnect.


Title: Re: Custom Mining Proxy ?
Post by: riverdip on May 16, 2019, 01:34:34 AM
The miners wont change where they are mining due to a DNS change, without a reconnect.

Instead use a standard mining proxy - and the proxy gets work from the pools you tell it to. When you change the proxy settings, they all (have to) reconnect.

do you have some reference links ?


Title: Re: Custom Mining Proxy ?
Post by: Artemis3 on May 17, 2019, 12:57:19 AM
do you have some reference links ?

Last time i asked this i got no answer. So I'm going to guess: ckpool proxy (https://bitbucket.org/ckolivas/ckpool).


Title: Re: Custom Mining Proxy ?
Post by: kano on May 17, 2019, 11:08:18 PM
Last time i asked this i got no answer. So I'm going to guess: ckpool proxy (https://bitbucket.org/ckolivas/ckpool).

That one has various issues and if you do find problems with it you wont get them fixed. Looks elsewhere.


Title: Re: Custom Mining Proxy ?
Post by: HagssFIN on May 18, 2019, 08:03:51 AM
no, but i heard of it, i have mixed miners. bitmain ebang avalon, some tools work some don't this is why i seek a universal solution.

If you will look at it more closely, you will notice that it has support for many different miner models.


Title: Re: Custom Mining Proxy ?
Post by: riverdip on May 18, 2019, 11:40:05 AM
If you will look at it more closely, you will notice that it has support for many different miner models.

not ebang.


Title: Re: Custom Mining Proxy ?
Post by: HagssFIN on May 18, 2019, 03:03:43 PM
Ebangs are shit anyway, so you would feel better if you sell them.


Title: Re: Custom Mining Proxy ?
Post by: tim-bc on May 21, 2019, 10:57:45 AM
Why don't you just set up an internal DNS server? Or, maybe you could buy a domain and define a different subdomain for each pool.


Title: Re: Custom Mining Proxy ?
Post by: Artemis3 on May 21, 2019, 03:16:42 PM
Why don't you just set up an internal DNS server? Or, maybe you could buy a domain and define a different subdomain for each pool.

You could do that with your own dns server (even a simple caching server would do), no need to purchase anything. But i don't think DNS traffic is the issue here, but the actual mining traffic.

With a mining proxy you could point all your asics to it, and the pool could see it as a single unit. And since you are pointing all your asics to the proxy, you only need to set the pool configuration in the proxy.

Too bad i can't find a list of recommended mining proxies, particularly free open source.


Title: Re: Custom Mining Proxy ?
Post by: tim-bc on May 21, 2019, 05:06:38 PM
I get the idea behind mining proxies, but I don't know of any compelling reasons to use them. Generally, the network traffic will be minimal as long as you have internal dhcp / dns / ntp, even with a large number of miners connecting directly to pools.

Plus, there isn't really any advantage to having the pool see your miners as a single unit when most pools will provide detailed stats that are broken down by worker ID.

In this case, I would suggest pointing all of your miners to an internal DNS server, so that way you only have to configure each pool once on the server.


Title: Re: Custom Mining Proxy ?
Post by: kano on May 21, 2019, 05:19:26 PM
Why don't you just set up an internal DNS server? Or, maybe you could buy a domain and define a different subdomain for each pool.
As I already said, it wont work coz the miners wont see a DNS change unless you force a reconnect on them all.


Title: Re: Custom Mining Proxy ?
Post by: Artemis3 on May 21, 2019, 05:54:00 PM
I don't bother with any of them, maybe the dns cache, maybe. Dhcp? LOL, the only good asic miner is one configured manually. Ntp i wouldn't even bother with either, and its common for that port to be closed in most LANs (and even ISPs) anyway. No, the major traffic is getting work and sending work from/to the pools, that is the whole point, the rest is meh. You could even use the pool ip instead of pool names fwiw, but you risk the small chance of the pool operator switching IPs (which does happen from time to time). Again, a dns cache would be more than sufficient. Just how much traffic do you think resolving all of three possible pool names would ever take anyway? Maybe a few more if you are into pool hopping (which no one should but whatever).

DNS is no issue, but set up a dns cache if you are so worried, you could even get fancy with dnscrypt-proxy (https://github.com/jedisct1/dnscrypt-proxy), so that even your queries get some extra level of privacy (and it caches too).

Dhcp should not be used, you would be introducing an unnecessary point of failure.

Ntp is meh, never seen a pool mind the miner clock.


Title: Re: Custom Mining Proxy ?
Post by: riverdip on May 28, 2019, 03:33:44 PM
all i want is conditional mining proxy

BUMP !!!!!!!


Title: Re: Custom Mining Proxy ?
Post by: tim-bc on May 30, 2019, 07:19:31 PM
I don't bother with any of them, maybe the dns cache, maybe. Dhcp? LOL, the only good asic miner is one configured manually.

Dhcp should not be used, you would be introducing an unnecessary point of failure.

DHCP is way more convenient, and it essentially becomes a necessity when you're running tens of thousands of miners.

I can understand a small farm setting static IPs but it just seems like a waste of effort for larger farms.

As I already said, it wont work coz the miners wont see a DNS change unless you force a reconnect on them all.

You can reboot bmminer on all of the miners fairly easily, through SSH or even using the api if you have privileged access set up.


Title: Re: Custom Mining Proxy ?
Post by: vh on May 30, 2019, 08:45:09 PM
Nginx in stream mode should be able to do what you need.

https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/

Edit:

The generic tcp proxy works just fine.

Seems like adding and removing backends dynamically only affects new incoming connections. You'll need a creative way to kill existing.

A shell script to rotate config, then stopping and starting nginx seems to do the job. Not awesome though.

Here's a short load balance example on Ubuntu 18.04 to get you started.

Code:
apt-get install -y nginx-extras

cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.orig

cat <<EOF >>/etc/nginx/nginx.conf

stream {
    server {
        listen 3333;
        proxy_pass mining_backend;
    }

    upstream mining_backend {
        server stratum.kano.is:3333;
        server stratum.slushpool.com:3333;
    }
}
EOF


nginx -t
nginx -s reload

Edit Part 2:

By installing nginx-extras instead of plain nginx, you have access to lua for the creative reset component. Use the examples in the link above to dynamically add/remove your desired pool backend(s), then force the miners to reconnect with the lua block below.

Code:
    location /drop_connections {
        default_type 'text/plain';
        content_by_lua_block {
            os.execute("(sleep 1; ps aux | grep 'nginx: worker process' | grep -v $$ | awk '{print $2}' | xargs kill -9) &")
            ngx.say('dropped')
        }
    }

Edit Part 3:

While putting it all together, I discovered the backend api are part of nginx+ and not the community version.

With lua available, we can quickly skip that and make it happen with a little rearranging:

First update /etc/nginx/nginx.conf.

Code:
stream {
    server {
        listen 3333;
        proxy_pass mining_backend;
    }

    upstream mining_backend {
        include /etc/nginx/mining.conf;
    }
}

Then grant the nginx child processes the ability to tweak nginx related configuration.

Code:
touch /etc/nginx/mining.conf
chown www-data:www-data /etc/nginx/mining.conf

echo "server stratum.kano.is:3333;" > /etc/nginx/mining.conf

cp /etc/sudoers /etc/sudoers.orig
echo "www-data ALL = NOPASSWD:/usr/sbin/nginx" >> /etc/sudoers

Finally, in the server directive of /etc/nginx/sites-enabled/default:

Code:
    location /switch_to_kano {
        default_type 'text/plain';
        content_by_lua_block {
            os.execute("(sleep 1; echo 'server stratum.kano.is:3333;' > /etc/nginx/mining.conf) &")
            os.execute("(sleep 2; sudo /usr/sbin/nginx -s reload) &")
            os.execute("(sleep 3; ps aux | grep 'nginx: worker process' | grep -v $$ | awk '{print $2}' | xargs kill -9) &")
            ngx.header["Content-type"] = "text/plain"
            ngx.say('switching to kano.is')
        }
    }
    
    location /switch_to_slush {
        default_type 'text/plain';
        content_by_lua_block {
            os.execute("(sleep 1; echo 'server stratum.slushpool.com:3333;' > /etc/nginx/mining.conf) &")
            os.execute("(sleep 2; sudo /usr/sbin/nginx -s reload) &")
            os.execute("(sleep 3; ps aux | grep 'nginx: worker process' | grep -v $$ | awk '{print $2}' | xargs kill -9) &")
            ngx.header["Content-type"] = "text/plain"
            ngx.say('switching to slushpool')
        }
    }

My local end results of this setup:

Miner mined when connected to stratum+tcp://[myip]:3333, and changes to the targeted pool when my browser hits http://[myip]/switch_to_kano & http://[myip]/switch_to_slush.

That was interesting in theory and neat to see working. 8)


Title: Re: Custom Mining Proxy ?
Post by: Artemis3 on June 01, 2019, 03:13:40 AM
DHCP is way more convenient, and it essentially becomes a necessity when you're running tens of thousands of miners.

I can understand a small farm setting static IPs but it just seems like a waste of effort for larger farms.

And then you introduce a new element of failure in your network. One reason people don't bother with it is not having to operate and maintain a proper dhcp server, or deal with endless annoyances from those provided by embedded devices. Then of course you introduce the wonderful element of IPs changing on their own. Sure you could fix them to mac address and that is as much fun as... setting them with static addresses in the first place.

Oh, did you know your asic miner dhcp client isn't exactly industrial grade?

I'm sure miners don't bother with dhcp for various other reasons.


Title: Re: Custom Mining Proxy ?
Post by: riverdip on July 18, 2019, 03:38:33 AM
UPDATE :-> 


http://gobetween.io/index.html


this provides more efficient way to manage proxies, but still no way to manage local workers


Title: Re: Custom Mining Proxy ?
Post by: cfbtcman on August 05, 2019, 01:39:59 AM
Riverdip made a simple question, no matters the need to reboot, of course if he wants to change the pools he can reboot all the machines, is much less work than do it one by one or use softwares to do it brand by brand.

We just need to have a domain or a DNS service like dyndns, someone have tried it with Dyndns and post con figurations?


Title: Re: Custom Mining Proxy ?
Post by: riverdip on August 06, 2019, 09:55:07 PM
Riverdip made a simple question, no matters the need to reboot, of course if he wants to change the pools he can reboot all the machines, is much less work than do it one by one or use softwares to do it brand by brand.

We just need to have a domain or a DNS service like dyndns, someone have tried it with Dyndns and post con figurations?

i found it working by ckpool -p proxy mode, but after some time it generates rejcted shares, so the best way it the DNS on cloudflare 


Title: Re: Custom Mining Proxy ?
Post by: cfbtcman on August 07, 2019, 01:28:58 AM
You are making a IP forward or URL forward?