BTC Guild - Server Update

[eleuthria]

Posting this in a separate thread since the main thread can push important updates out of the way, and the first post isn't always checked for changes.

BTC Guild has recently added a 5th server, and updated all servers to significantly increase their capacity before miner idles become an issue.  It is highly recommended that you now manually pick a server from the list.  Picking a server with too much load will negatively impact your rewards, and it is always preferable to take a low/normal load server over a high/overloaded server, regardless of latency (unless you have actual connection problems to the server).


Please point your miners at one of the following servers (load percentages as of 8:04 AM PDT, June 16th):
  nl.btcguild.com   - 36% load (LOW)
  uk.btcguild.com   - 40% load (LOW)
  useast.btcguild.com   - 45% load (LOW)
  uswest.btcguild.com   - 61% load (Normal)


To view the most current load statistics, you can look at http://www.btcguild.com/pick_a_server.php.  Using the BTC Guild selection in GUI Miner, or using the old "btcguild.com" generic address will point you to the US Central server.  This server is currently overloaded, and it is recommended that all users point to a different one.  For GUI Miner, this is done by selecting 'Other' as the pool, and typing in the address.  For all other miners, it's as simple as changing btcguild.com to one of the servers listed above.

[1714044959]

1714044959

[SteveFL]

This is wonderful, but the loads seem to be changing drastically every time I refresh my status.  I picked USeast when it was low, and I've watched it go from overloaded and back a few times.  We really can't be expected to keep changing our workers around.

I see this was recently added:
You have recently submitted shares to an overloaded server. This is hurting your BTC generation, please point your workers at a different BTC Guild server.

This really isn't going to help the situation if people are changing their workers and we all pick the same lowest server.

[cablepair]

nice work Eleuthira

seems to be working well for me.

[eleuthria]

This is wonderful, but the loads seem to be changing drastically every time I refresh my status.  I picked USeast when it was low, and I've watched it go from overloaded and back a few times.  We really can't be expected to keep changing our workers around.

The server load is not changing quickly.  It would require a swing of over 250 GH/sec to take a server from Low to Overloaded.  Last night, US East and NL were overloaded, because the generic 'btcguild.com' was pointing at both of them using round robin DNS.  Immediately after US Central came online, btcguild.com was changed to point to that server.

Since then, the only server to have a status of overloaded was US Central.  There is a 100 GH/sec window for each status, so it would take a huge migration of workers to move the server from Normal -> Overloaded without spending time in the 'High load' status.

[just_someguy]

When is the switch to difficulty two going to happen, or has it already?

[SteveFL]

This is wonderful, but the loads seem to be changing drastically every time I refresh my status.  I picked USeast when it was low, and I've watched it go from overloaded and back a few times.  We really can't be expected to keep changing our workers around.

The server load is not changing quickly.  It would require a swing of over 250 GH/sec to take a server from Low to Overloaded.  Last night, US East and NL were overloaded, because the generic 'btcguild.com' was pointing at both of them using round robin DNS.  Immediately after US Central came online, btcguild.com was changed to point to that server.

Since then, the only server to have a status of overloaded was US Central.  There is a 100 GH/sec window for each status, so it would take a huge migration of workers to move the server from Normal -> Overloaded without spending time in the 'High load' status.

This would be the exact behavior I observed last night then.  I guess I should have clarified that it was an hour or so between each refresh.  Based solely on the 4 or 5 screens I saw I assumed that everyone was doing a knee-jerk reaction and changing pools, then overloading the previously good one.

[eleuthria]

When is the switch to difficulty two going to happen, or has it already?

Difficulty 2 has been pushed back for now.  That was supposed to fix the MySQL CPU usage wall we were hitting, but we found an index that was missing from the servers when the pool was updated for the master/slave setup.  It cut CPU usage down to a quarter of its original use, so difficulty 2 is not being implemented at this time.

[cablepair]

I think you made the right decision on that one. Even though difficulty 2 should work in theory I don't believe it has been tested thoroughly has it?


My miners have been mining quite nicely with no problems. I think you got everything stabilized well. Congrats.

[phenom]

What's the point in telling people to choose a specific server? Can't you have some sort of load balancing so that if one server goes down you can redirect users to another/backup? I've switched back to deepbit for now as your servers are terrible. Always overloading or going down. The fees from deepbit outweigh the downtime I get from your servers at the moment.

[hart]

What's the point in telling people to choose a specific server? Can't you have some sort of load balancing so that if one server goes down you can redirect users to another/backup? I've switched back to deepbit for now as your servers are terrible. Always overloading or going down. The fees from deepbit outweigh the downtime I get from your servers at the moment.

I think the problems so far is that BTC Guild has relied on Round-Robin DNS to "load balance" before picking servers. Problem is, RR-DNS is NOT load balancing, but load distribution. Just like pools get lucky and unlucky, RR-DNS can overload and underload servers since it's just random.

The current "solution" is to pick a server, which might be nice in theory, but BitCoin mining isn't really suppose to be a full time job. If you're not monitoring the IRC chat and/or your miner's logs, you could miss out on quite a lot of money in no time at all. As was the case when UK's MySQL daemon had connectivity issues with US West. Everyone who pointed to the UK, as the website suggested when one of the other servers got overloaded, probably experienced at least an hour or two of idle miners last night, possibly more if their router/ISP/computer cached the subdomain's DNS longer than normal.

My proposed solution is to handle load balancing in mining clients intelligently by use of data provided by the pool's API. At the moment, the pool's API doesn't give enough true load data for this to happen (but rather, quite meaningless, relatively, hash rates per server, which are estimations, no less).

The API would need to be something along the lines of this... http://pastebin.com/PHNbU0u1
The preferred load function doesn't just reference server load, but also server capacity. An ideal function:

Code:

function server_load() { 
	$load = explode(' ', file_get_contents('/proc/loadavg'));
	$cpus = substr_count(file_get_contents('/proc/cpuinfo'), 'processor');

	return $load[2]/$cpus; 
} 

Between every few blocks, clients should check an improved pool stats API and determine the best server to connect to with weighted odds, as to not have every client jumping on the idlest server. For example, using the ideal json response from above, the ideal odds for hitting each server:

uswest.btcguild.com - 26.12%
useast.btcguild.com - 25.17%
nl.btcguild.com - 25.17%
uscentral.btcguild.com - 23.54%
uk.btcguild.com - 0% (offline)

If every large pool can adopt this same API design, more clients can be aware. And since the chance part of it can be done in the API of the server (the "suggested_server" variable), each API hit can run the calculations and pick a server for the client. The client only needs to check the API between every few blocks and listen.

[Phil21]

I already use a local HAProxy install to auto-failover between servers, it wouldn't be very difficult to add weights to this as well to distribute the load based off of an API call.

I'll look into it in the AM and post if I end up doing it