Bitcoin Forum
March 29, 2024, 04:45:24 AM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 »  All
  Print  
Author Topic: Custom Mining Proxy ?  (Read 609 times)
riverdip (OP)
Jr. Member
*
Offline Offline

Activity: 199
Merit: 2


View Profile
May 15, 2019, 07:46:52 PM
Last edit: May 28, 2019, 03:33:30 PM by riverdip
 #1

hey i have lots of miners and sometimes it just paid to manage them all.

i was thinking if i could buy my own domain and run proxy that routes all mining to any server of my choice.

LIKE THIS:

mydomain.com -> kanopool
mydomain.com -> slushpool

so if i have to change mining pool i just chance forward url on my domain settings and it should be done for all machines.


UPDATE - 29 MAY - Too many talkies providing alternative solution, but no working solution yet !

something like this -> https://github.com/pooler/rollproxy

but with modern protocols and conditional pool change features

https://payhua.com -
RECURRING PAYMENTS ON THE BLOCKCHAIN
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
HagssFIN
Legendary
*
Offline Offline

Activity: 2422
Merit: 1704


Electrical engineer. Mining since 2014.


View Profile WWW
May 15, 2019, 07:53:47 PM
 #2

Have you considered using the Awesome Miner -software for managing your miners?

https://bitcointalk.org/index.php?topic=676942

riverdip (OP)
Jr. Member
*
Offline Offline

Activity: 199
Merit: 2


View Profile
May 15, 2019, 10:40:46 PM
Last edit: May 16, 2019, 12:22:07 AM by frodocooper
 #3

Have you considered using the Awesome Miner -software for managing your miners?

https://bitcointalk.org/index.php?topic=676942

no, but i heard of it, i have mixed miners. bitmain ebang avalon, some tools work some don't this is why i seek a universal solution.

https://payhua.com -
RECURRING PAYMENTS ON THE BLOCKCHAIN
kano
Legendary
*
Offline Offline

Activity: 4452
Merit: 1798


Linux since 1997 RedHat 4


View Profile
May 15, 2019, 11:32:14 PM
Last edit: May 16, 2019, 12:46:08 AM by kano
 #4

...
i was thinking if i could buy my own domain and run proxy that routes all mining to any server of my choice.

LIKE THIS:

mydomain.com -> kanopool
mydomain.com -> slushpool

so if i have to change mining pool i just chance forward url on my domain settings and it should be done for all machines.
The miners wont change where they are mining due to a DNS change, without a reconnect.

Instead use a standard mining proxy - and the proxy gets work from the pools you tell it to. When you change the proxy settings, they all (have to) reconnect.

Pool: https://kano.is - low 0.5% fee PPLNS 3 Days - Most reliable Solo with ONLY 0.5% fee   Bitcointalk thread: Forum
Discord support invite at https://kano.is/ Majority developer of the ckpool code - k for kano
The ONLY active original developer of cgminer. Original master git: https://github.com/kanoi/cgminer
riverdip (OP)
Jr. Member
*
Offline Offline

Activity: 199
Merit: 2


View Profile
May 16, 2019, 01:34:34 AM
Last edit: May 16, 2019, 04:18:22 AM by frodocooper
 #5

The miners wont change where they are mining due to a DNS change, without a reconnect.

Instead use a standard mining proxy - and the proxy gets work from the pools you tell it to. When you change the proxy settings, they all (have to) reconnect.

do you have some reference links ?

https://payhua.com -
RECURRING PAYMENTS ON THE BLOCKCHAIN
Artemis3
Legendary
*
Offline Offline

Activity: 1988
Merit: 1556


CLEAN non GPL infringing code made in Rust lang


View Profile WWW
May 17, 2019, 12:57:19 AM
Last edit: May 18, 2019, 02:10:15 AM by frodocooper
 #6

do you have some reference links ?

Last time i asked this i got no answer. So I'm going to guess: ckpool proxy.

██████
███████
███████
████████
BRAIINS OS+|AUTOTUNING
MINING FIRMWARE
|
Increase hashrate on your Bitcoin ASICs,
improve efficiency as much as 25%, and
get 0% pool fees on Braiins Pool
kano
Legendary
*
Offline Offline

Activity: 4452
Merit: 1798


Linux since 1997 RedHat 4


View Profile
May 17, 2019, 11:08:18 PM
Last edit: May 18, 2019, 11:15:06 AM by kano
 #7

Last time i asked this i got no answer. So I'm going to guess: ckpool proxy.

That one has various issues and if you do find problems with it you wont get them fixed. Looks elsewhere.

Pool: https://kano.is - low 0.5% fee PPLNS 3 Days - Most reliable Solo with ONLY 0.5% fee   Bitcointalk thread: Forum
Discord support invite at https://kano.is/ Majority developer of the ckpool code - k for kano
The ONLY active original developer of cgminer. Original master git: https://github.com/kanoi/cgminer
HagssFIN
Legendary
*
Offline Offline

Activity: 2422
Merit: 1704


Electrical engineer. Mining since 2014.


View Profile WWW
May 18, 2019, 08:03:51 AM
Last edit: May 18, 2019, 10:17:42 AM by frodocooper
 #8

no, but i heard of it, i have mixed miners. bitmain ebang avalon, some tools work some don't this is why i seek a universal solution.

If you will look at it more closely, you will notice that it has support for many different miner models.

riverdip (OP)
Jr. Member
*
Offline Offline

Activity: 199
Merit: 2


View Profile
May 18, 2019, 11:40:05 AM
Last edit: May 19, 2019, 11:36:29 PM by frodocooper
 #9

If you will look at it more closely, you will notice that it has support for many different miner models.

not ebang.

https://payhua.com -
RECURRING PAYMENTS ON THE BLOCKCHAIN
HagssFIN
Legendary
*
Offline Offline

Activity: 2422
Merit: 1704


Electrical engineer. Mining since 2014.


View Profile WWW
May 18, 2019, 03:03:43 PM
Merited by NotFuzzyWarm (1), Steamtyme (1)
 #10

Ebangs are shit anyway, so you would feel better if you sell them.

tim-bc
Full Member
***
Offline Offline

Activity: 538
Merit: 175


View Profile
May 21, 2019, 10:57:45 AM
 #11

Why don't you just set up an internal DNS server? Or, maybe you could buy a domain and define a different subdomain for each pool.

Ignore scammers on Skype, Telegram, etc. I will only ever contact you via forum PMs. See profile for fingerprint.
Artemis3
Legendary
*
Offline Offline

Activity: 1988
Merit: 1556


CLEAN non GPL infringing code made in Rust lang


View Profile WWW
May 21, 2019, 03:16:42 PM
 #12

Why don't you just set up an internal DNS server? Or, maybe you could buy a domain and define a different subdomain for each pool.

You could do that with your own dns server (even a simple caching server would do), no need to purchase anything. But i don't think DNS traffic is the issue here, but the actual mining traffic.

With a mining proxy you could point all your asics to it, and the pool could see it as a single unit. And since you are pointing all your asics to the proxy, you only need to set the pool configuration in the proxy.

Too bad i can't find a list of recommended mining proxies, particularly free open source.

██████
███████
███████
████████
BRAIINS OS+|AUTOTUNING
MINING FIRMWARE
|
Increase hashrate on your Bitcoin ASICs,
improve efficiency as much as 25%, and
get 0% pool fees on Braiins Pool
tim-bc
Full Member
***
Offline Offline

Activity: 538
Merit: 175


View Profile
May 21, 2019, 05:06:38 PM
Last edit: May 22, 2019, 05:21:21 AM by frodocooper
 #13

I get the idea behind mining proxies, but I don't know of any compelling reasons to use them. Generally, the network traffic will be minimal as long as you have internal dhcp / dns / ntp, even with a large number of miners connecting directly to pools.

Plus, there isn't really any advantage to having the pool see your miners as a single unit when most pools will provide detailed stats that are broken down by worker ID.

In this case, I would suggest pointing all of your miners to an internal DNS server, so that way you only have to configure each pool once on the server.

Ignore scammers on Skype, Telegram, etc. I will only ever contact you via forum PMs. See profile for fingerprint.
kano
Legendary
*
Offline Offline

Activity: 4452
Merit: 1798


Linux since 1997 RedHat 4


View Profile
May 21, 2019, 05:19:26 PM
 #14

Why don't you just set up an internal DNS server? Or, maybe you could buy a domain and define a different subdomain for each pool.
As I already said, it wont work coz the miners wont see a DNS change unless you force a reconnect on them all.

Pool: https://kano.is - low 0.5% fee PPLNS 3 Days - Most reliable Solo with ONLY 0.5% fee   Bitcointalk thread: Forum
Discord support invite at https://kano.is/ Majority developer of the ckpool code - k for kano
The ONLY active original developer of cgminer. Original master git: https://github.com/kanoi/cgminer
Artemis3
Legendary
*
Offline Offline

Activity: 1988
Merit: 1556


CLEAN non GPL infringing code made in Rust lang


View Profile WWW
May 21, 2019, 05:54:00 PM
Last edit: May 22, 2019, 05:21:52 AM by frodocooper
 #15

I don't bother with any of them, maybe the dns cache, maybe. Dhcp? LOL, the only good asic miner is one configured manually. Ntp i wouldn't even bother with either, and its common for that port to be closed in most LANs (and even ISPs) anyway. No, the major traffic is getting work and sending work from/to the pools, that is the whole point, the rest is meh. You could even use the pool ip instead of pool names fwiw, but you risk the small chance of the pool operator switching IPs (which does happen from time to time). Again, a dns cache would be more than sufficient. Just how much traffic do you think resolving all of three possible pool names would ever take anyway? Maybe a few more if you are into pool hopping (which no one should but whatever).

DNS is no issue, but set up a dns cache if you are so worried, you could even get fancy with dnscrypt-proxy, so that even your queries get some extra level of privacy (and it caches too).

Dhcp should not be used, you would be introducing an unnecessary point of failure.

Ntp is meh, never seen a pool mind the miner clock.

██████
███████
███████
████████
BRAIINS OS+|AUTOTUNING
MINING FIRMWARE
|
Increase hashrate on your Bitcoin ASICs,
improve efficiency as much as 25%, and
get 0% pool fees on Braiins Pool
riverdip (OP)
Jr. Member
*
Offline Offline

Activity: 199
Merit: 2


View Profile
May 28, 2019, 03:33:44 PM
Last edit: May 29, 2019, 11:22:00 AM by frodocooper
 #16

all i want is conditional mining proxy

BUMP !!!!!!!

https://payhua.com -
RECURRING PAYMENTS ON THE BLOCKCHAIN
tim-bc
Full Member
***
Offline Offline

Activity: 538
Merit: 175


View Profile
May 30, 2019, 07:19:31 PM
Last edit: May 31, 2019, 10:24:22 AM by frodocooper
 #17

I don't bother with any of them, maybe the dns cache, maybe. Dhcp? LOL, the only good asic miner is one configured manually.

Dhcp should not be used, you would be introducing an unnecessary point of failure.

DHCP is way more convenient, and it essentially becomes a necessity when you're running tens of thousands of miners.

I can understand a small farm setting static IPs but it just seems like a waste of effort for larger farms.

As I already said, it wont work coz the miners wont see a DNS change unless you force a reconnect on them all.

You can reboot bmminer on all of the miners fairly easily, through SSH or even using the api if you have privileged access set up.

Ignore scammers on Skype, Telegram, etc. I will only ever contact you via forum PMs. See profile for fingerprint.
vh
Hero Member
*****
Offline Offline

Activity: 699
Merit: 662


View Profile
May 30, 2019, 08:45:09 PM
Last edit: June 01, 2019, 10:01:33 AM by frodocooper
Merited by frodocooper (5), DaCryptoRaccoon (1)
 #18

Nginx in stream mode should be able to do what you need.

https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/

Edit:

The generic tcp proxy works just fine.

Seems like adding and removing backends dynamically only affects new incoming connections. You'll need a creative way to kill existing.

A shell script to rotate config, then stopping and starting nginx seems to do the job. Not awesome though.

Here's a short load balance example on Ubuntu 18.04 to get you started.

Code:
apt-get install -y nginx-extras

cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.orig

cat <<EOF >>/etc/nginx/nginx.conf

stream {
    server {
        listen 3333;
        proxy_pass mining_backend;
    }

    upstream mining_backend {
        server stratum.kano.is:3333;
        server stratum.slushpool.com:3333;
    }
}
EOF


nginx -t
nginx -s reload

Edit Part 2:

By installing nginx-extras instead of plain nginx, you have access to lua for the creative reset component. Use the examples in the link above to dynamically add/remove your desired pool backend(s), then force the miners to reconnect with the lua block below.

Code:
   location /drop_connections {
        default_type 'text/plain';
        content_by_lua_block {
            os.execute("(sleep 1; ps aux | grep 'nginx: worker process' | grep -v $$ | awk '{print $2}' | xargs kill -9) &")
            ngx.say('dropped')
        }
    }

Edit Part 3:

While putting it all together, I discovered the backend api are part of nginx+ and not the community version.

With lua available, we can quickly skip that and make it happen with a little rearranging:

First update /etc/nginx/nginx.conf.

Code:
stream {
    server {
        listen 3333;
        proxy_pass mining_backend;
    }

    upstream mining_backend {
        include /etc/nginx/mining.conf;
    }
}

Then grant the nginx child processes the ability to tweak nginx related configuration.

Code:
touch /etc/nginx/mining.conf
chown www-data:www-data /etc/nginx/mining.conf

echo "server stratum.kano.is:3333;" > /etc/nginx/mining.conf

cp /etc/sudoers /etc/sudoers.orig
echo "www-data ALL = NOPASSWD:/usr/sbin/nginx" >> /etc/sudoers

Finally, in the server directive of /etc/nginx/sites-enabled/default:

Code:
   location /switch_to_kano {
        default_type 'text/plain';
        content_by_lua_block {
            os.execute("(sleep 1; echo 'server stratum.kano.is:3333;' > /etc/nginx/mining.conf) &")
            os.execute("(sleep 2; sudo /usr/sbin/nginx -s reload) &")
            os.execute("(sleep 3; ps aux | grep 'nginx: worker process' | grep -v $$ | awk '{print $2}' | xargs kill -9) &")
            ngx.header["Content-type"] = "text/plain"
            ngx.say('switching to kano.is')
        }
    }
    
    location /switch_to_slush {
        default_type 'text/plain';
        content_by_lua_block {
            os.execute("(sleep 1; echo 'server stratum.slushpool.com:3333;' > /etc/nginx/mining.conf) &")
            os.execute("(sleep 2; sudo /usr/sbin/nginx -s reload) &")
            os.execute("(sleep 3; ps aux | grep 'nginx: worker process' | grep -v $$ | awk '{print $2}' | xargs kill -9) &")
            ngx.header["Content-type"] = "text/plain"
            ngx.say('switching to slushpool')
        }
    }

My local end results of this setup:

Miner mined when connected to stratum+tcp://[myip]:3333, and changes to the targeted pool when my browser hits http://[myip]/switch_to_kano & http://[myip]/switch_to_slush.

That was interesting in theory and neat to see working. Cool

Artemis3
Legendary
*
Offline Offline

Activity: 1988
Merit: 1556


CLEAN non GPL infringing code made in Rust lang


View Profile WWW
June 01, 2019, 03:13:40 AM
Last edit: June 01, 2019, 10:02:23 AM by frodocooper
 #19

DHCP is way more convenient, and it essentially becomes a necessity when you're running tens of thousands of miners.

I can understand a small farm setting static IPs but it just seems like a waste of effort for larger farms.

And then you introduce a new element of failure in your network. One reason people don't bother with it is not having to operate and maintain a proper dhcp server, or deal with endless annoyances from those provided by embedded devices. Then of course you introduce the wonderful element of IPs changing on their own. Sure you could fix them to mac address and that is as much fun as... setting them with static addresses in the first place.

Oh, did you know your asic miner dhcp client isn't exactly industrial grade?

I'm sure miners don't bother with dhcp for various other reasons.

██████
███████
███████
████████
BRAIINS OS+|AUTOTUNING
MINING FIRMWARE
|
Increase hashrate on your Bitcoin ASICs,
improve efficiency as much as 25%, and
get 0% pool fees on Braiins Pool
riverdip (OP)
Jr. Member
*
Offline Offline

Activity: 199
Merit: 2


View Profile
July 18, 2019, 03:38:33 AM
 #20

UPDATE :-> 


http://gobetween.io/index.html


this provides more efficient way to manage proxies, but still no way to manage local workers

https://payhua.com -
RECURRING PAYMENTS ON THE BLOCKCHAIN
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!