Bitcoin Forum
December 09, 2016, 05:47:48 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: Pool server auto-configuration: /config.json  (Read 3402 times)
jgarzik
Legendary
*
Offline Offline

Activity: 1470


View Profile
April 09, 2011, 10:11:28 PM
 #1


Currently, pool server extensions such as long-polling provide static configuration information -- the long-polling URL -- via a special HTTP header.  This is sub-optimal, because the long-polling URL is unlikely to change, yet many pool servers utilize the simple approach and send the same LP URL to each mining client, over and over again, wasting bandwidth.

A server list, used in mining client fail-over (something I have been pushing for), would be another example of static configuration information communicated from pool server to client.

Therefore, I began updating cpuminer to download $POOL_SERVER/config.json.  This will look something like

Code:
{ "name" : "slush",
    "description" : "slush's super-big pool",
   "longpoll" : "/LP",
   "ntime" : true,
   "server_list" : [ { "url" : "http://srv1.net" }, { "url":"http://srv2.net" } ]
   "xslush" : {
       "my server-specific option #1" : "my value",
       "my server-specific option #2" : "my value"
   }
}

This should give pool servers a lot more flexibility in defining optional features and metadata, without needing to transmit that information redundantly in every single 'getwork' HTTP header.

Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
1481305668
Hero Member
*
Offline Offline

Posts: 1481305668

View Profile Personal Message (Offline)

Ignore
1481305668
Reply with quote  #2

1481305668
Report to moderator
1481305668
Hero Member
*
Offline Offline

Posts: 1481305668

View Profile Personal Message (Offline)

Ignore
1481305668
Reply with quote  #2

1481305668
Report to moderator
1481305668
Hero Member
*
Offline Offline

Posts: 1481305668

View Profile Personal Message (Offline)

Ignore
1481305668
Reply with quote  #2

1481305668
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481305668
Hero Member
*
Offline Offline

Posts: 1481305668

View Profile Personal Message (Offline)

Ignore
1481305668
Reply with quote  #2

1481305668
Report to moderator
martok
Full Member
***
Offline Offline

Activity: 140


View Profile
April 09, 2011, 10:45:18 PM
 #2

I like this idea though what is the extended server info (xslush) for since it can't be supported by the client directly? Perhaps defining the base config well enough such that server specifics aren't necessary.

Also, we need a way for the configs to refresh. Either a refresh : seconds option in said config which causes the client to send a HEAD request for the config to see if it has changed or some notification where a server can push the new config out to clients via say an x-config-refresh header.
jgarzik
Legendary
*
Offline Offline

Activity: 1470


View Profile
April 09, 2011, 11:02:36 PM
 #3

I like this idea though what is the extended server info (xslush) for since it can't be supported by the client directly? Perhaps defining the base config well enough such that server specifics aren't necessary.

"xslush" is an example of a configuration that is site-specific, and not standard.

Anything with the "x" prefix, at the top level, is fair game for pool servers to define whatever they want, while the other items are those standardized by the community of mining clients.

Quote
Also, we need a way for the configs to refresh. Either a refresh : seconds option in said config which causes the client to send a HEAD request for the config to see if it has changed or some notification where a server can push the new config out to clients via say an x-config-refresh header.

Yep...


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
dbitcoin
Hero Member
*****
Offline Offline

Activity: 742

BTCDig - mining pool


View Profile WWW
April 09, 2011, 11:35:40 PM
 #4

I like this idea though what is the extended server info (xslush) for since it can't be supported by the client directly? Perhaps defining the base config well enough such that server specifics aren't necessary.

"xslush" is an example of a configuration that is site-specific, and not standard.

Anything with the "x" prefix, at the top level, is fair game for pool servers to define whatever they want, while the other items are those standardized by the community of mining clients.

Quote
Also, we need a way for the configs to refresh. Either a refresh : seconds option in said config which causes the client to send a HEAD request for the config to see if it has changed or some notification where a server can push the new config out to clients via say an x-config-refresh header.

Yep...



Overall this is good idea, but not for static long polling url.
I use unique urls for each miner.

BTCDig - mining pool (Stratum, VarDiff, DGM, SSL, JSON API)
[Tycho]
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
April 09, 2011, 11:41:06 PM
 #5

  • Adding this config probably will make miner programming more difficult because it would require additional actions to be performed. How many clients will correctly understand headers that define cache control and expiration time ?
  • You'll need to parse headers anyway for "force failover" or "update config" commands from server to be detected and acted upon. So all the clients will have to BOTH read config file AND check headers every time. Keep It Simple principle is broken here.
  • I think it's a solution for non-existent problem. Traffic loss is negligible, it would be strange to care about tens of bytes while still sending all the other headers back and forth, not to mention the text-mode data packet instead of compact binary.
    Let's assume a fast GPU with 10 seconds period to hash all the nonces and 80 bytes additional header for server list. This will be ~28 Kb per hour, not really something to worry about.

Welcome to my bitcoin mining pool: https://deepbit.net - Both payment schemes (including PPS), instant payout, no invalid blocks !
ICBIT Trading platform : USD/BTC futures trading, Bitcoin difficulty futures (NEW!). Third year in bitcoin business.
xf2_org
Member
**
Offline Offline

Activity: 70


View Profile
April 09, 2011, 11:48:28 PM
 #6

Overall this is good idea, but not for static long polling url.
I use unique urls for each miner.

Your server is free to return a per-miner LP URL, when the miner requests /config.json upon startup (or when config.json is updated).

[this is jgarzik, using a new forum nick]

slush
Legendary
*
Offline Offline

Activity: 1358



View Profile WWW
April 10, 2011, 11:19:02 PM
 #7

Must agree with Tycho and dbitcoin that it brings unneeded complexity. With extra headers in http requests the miner and pool code can be much more straightforward and those few extra bytes are not real problem.

xf2_org
Member
**
Offline Offline

Activity: 70


View Profile
April 10, 2011, 11:57:26 PM
 #8

Must agree with Tycho and dbitcoin that it brings unneeded complexity. With extra headers in http requests the miner and pool code can be much more straightforward and those few extra bytes are not real problem.

I disagree; It is not straightforward in the client.

[Tycho]
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
April 11, 2011, 03:22:36 AM
 #9

My protocol extension proposal:

http://deepbit.net/failover.php
i'm going to implement it in my pool soon.

Comments and advices are welcome :)

Welcome to my bitcoin mining pool: https://deepbit.net - Both payment schemes (including PPS), instant payout, no invalid blocks !
ICBIT Trading platform : USD/BTC futures trading, Bitcoin difficulty futures (NEW!). Third year in bitcoin business.
slush
Legendary
*
Offline Offline

Activity: 1358



View Profile WWW
April 11, 2011, 08:09:59 AM
 #10

Tycho, I like the switch feature, I plan something similar for my pool, so unified API is good thing (tm). But I don't feel that list of failover servers is needed in every getwork request. And I think it isn't needed at all (at least for me).

Currently I have implemented load balancer between getwork nodes and I'm planning to add IP failover of this balancer, so I don't need server failover feature in mining clients at all...

xf2_org
Member
**
Offline Offline

Activity: 70


View Profile
April 11, 2011, 08:55:29 AM
 #11

But I don't feel that list of failover servers is needed in every getwork request.

Thank you.

In general, static, unchanging information should not be sent repeatedly to mining clients with every getwork.

slush
Legendary
*
Offline Offline

Activity: 1358



View Profile WWW
April 11, 2011, 09:15:02 AM
 #12

In general, static, unchanging information should not be sent repeatedly to mining clients with every getwork.

Agree. Unfortunately not sending some information in every requests leads to some refreshing and client side cache (because those information are not absolutely static; pool can connect new servers and want to use them in oncoming upgrade). That's why I don't like both solution (failover list in every getwork and static json config).

This is why current "protocol" based on http is screwed. We _can_ implement config updates using LP channel, but all this is pretty ugly and complicated. Better than patching current "protocol" in all pool servers and all miners, I vote for discussion about message based TCP based protocol which can solve all current workarounds (I see LP as workaround, too) with some better 'final' solution.

We already had a discussion about TCP protocol with jgarzik and we didn't found an agreement on details (mainly in message structure itself), but I hope that finding wider compromise (pool operators & miner developers) is the only way how to get things better.

xf2_org
Member
**
Offline Offline

Activity: 70


View Profile
April 11, 2011, 09:44:16 AM
 #13

Agree. Unfortunately not sending some information in every requests leads to some refreshing and client side cache (because those information are not absolutely static; pool can connect new servers and want to use them in oncoming upgrade). That's why I don't like both solution (failover list in every getwork and static json config).

Easy solution:  consolidate all information into a single "X-Config-Last-Modified" header.  When that changes, client refreshes config.json.

Though I agree this is and long-polling are both a hack, and a better protocol is needed.

dbitcoin
Hero Member
*****
Offline Offline

Activity: 742

BTCDig - mining pool


View Profile WWW
April 11, 2011, 11:05:26 AM
 #14

I agree what we need new protocol, but we should use another port for this, and support both protocols for a long time.
I still see some miners use old version of poclbm without LP support with fast gpu.
Despite all of my announcements and mailings about software update.

BTCDig - mining pool (Stratum, VarDiff, DGM, SSL, JSON API)
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!