I don't believe BTC-e has trading for any of the other merged mining altcoins.  I think cryptsy is where people trade them?  I really haven't bothered looking into the sheer volume of junk coins out there, but the new server is being designed in a way it can handle all these chains at once.  IXC/DVC/I0C/GRC will all end up running on a separate server from the pools themselves that way there's no impact *at all* to regular performance.

I've previously been against the near-valueless coins because of the HDD/CPU time spent processing blocks on those chains could negatively impact performance.  While it wouldn't be a huge impact either way, they also add so little that the tradeoff didn't seem worth it.  With the new code, I'm making it so that tradeoff no longer exists.

I've been asked a few times, so here's a breakdown of what this latest round of updates to backend code will result in for the end user:

1) Improved server responsiveness due to load splitting across cores.  This should result in a marginal decrease in stales across the board.  While the majority of stales are due to latency between the pool and the miner (and additionally the hardware), there is always a marginal chance of stales due to increased processing time when new blocks hit.

2) Faster block notification from bitcoind to the pool server.  This will reduce the time the pool spends working on an old block.  The current system polls bitcoind very rapidly to find out about new blocks.  The new system uses bitcoind's blocknotify to send a signal to the pool process.  This should reduce the time from an average of ~15ms between notifications to 1ms or so.  Similar to the drop in stale rate, the actual timing difference is somewhat difficult to measure, though definitely an improvement.

3) Uniform worker names.  BTC Guild uses '_' to separate the username and worker name sections.  Some pools use '.' and some pools don't use anything.  I have had quite a few people coming from pools that use a '.' emailing me/PM'ing me on IRC with issues where the problem was a '.' instead of a '_' in their miner settings.  The new pool server will rewrite this change so that either one will work.  Additionally,  if you fail to specify a worker and only use your pool username, it will automatically default to the '_1' worker.

4) [Future Benefit] Additional merged mining coins will almost definitely be added in the month of January.  This will likely be the full suite of merged-minable coins (NMC, IXC, DVC, I0C, Groupcoin).

5) [Possible Future Expansion] There is a chance with this new server that the frontend will be rewritten in how it interacts with the pool servers, specifically for estimating worker speeds and whether or not a worker is idle.  This possible change will be first tested on Scrypt Guild since there is no existing framework already in place for those two functions for Scrypt Guild.

=== Scrypt Guild Benefits ===
All of the above will apply to ScryptGuild as well, since the core for both systems will be the same.  On top of the above, this new code is being written with the assumption that it is possible to run multiple types of coins in a single system.  This will allow Scrypt Guild to have user-selectable coins via the web interface.  Eventually this can also be expanded to profit switching based on coin profitability.




Just thought I'd clarify the above since a few people have asked what all this means for the end user.  While my focus in previous mentions of this work have all been about server stability/efficiency, it isn't always clear how much, if any, of that will end up being visible for the miners.

The US stratum server (frontend/initial server) is now running the new codebase.  Had an eventful day of bug chasing as little quirks showed up on the live deployment that weren't present when testing it in a development environment.  So far things are looking excellent, and the known issues have been ironed out. CPU usage on the server which is almost always being slammed by botnets is only 1-3% per core, a huge improvement in general efficiency (using all cores) and overall efficiency (less total CPU).

The backend servers will likely be updated to the new codebase over the next two days.  Scrypt Guild will begin development once that deployment is done and confirmed to be functioning as expected.

Website errors like the ones posted above are on Cloudflare's end, not BTC Guild's.  Unfortunately, I can't do anything about them.  The benefits gained from using Cloudflare outweigh the cost of the occasional certificate error message.  Luckily they are (normally) region based, rather than affecting all users at once.

The problem with botnets is that outright banning them turns their zombie PCs into a non-stop DDoS as they spam reconnects with the mining software.  Blocking them upstream isn't exactly viable given how many of them there are.  Not many places will let you setup a deny list with hundreds of thousands of individual IPs, with constant updates required.

The validation server acts as a tarpit, keeping the botnets stuck while letting good miners through.  It's not perfect, and it has ended up making some legitimate mining software not work (GUIMiner).  But the good has outweighed the cost.  The few botnets that get through I'm able to identify after the fact, add them to a block list, and then the next time their zombies connect to the validation server they're stuck there instead of getting passed through like good traffic.

Sadly that doesn't quite work for things like spambots.



UPDATE:  Validation server is back to the old code for the night.  There's a few tweaks to be made for tomorrow.  Probably won't be going live on the backends tomorrow just because I don't want to have it running for only a few hours before I get some sleep tomorrow night.  Would prefer to launch it first thing in the morning Monday so I can watch it the entire day.

Well, it took a few hours of debugging little issues, but the US validation server is now running the new Stratum code, and hasn't crashed in the last 10 minutes.  Not much better you can do in terms of stress testing than putting it up against the server that is constantly being spammed by botnets.


This is going to be left up for the next few hours, then I'll swap it back to the old one when I head to sleep for the night (just to be safe).  Tomorrow's work will be focused on adding in the few pieces that are needed to put this up for the real backend servers.  The new server is showing fairly significant performance increases thanks to the much improved threading model.

If anybody is having problems starting up their miners, let me know!  I've tested it myself with my little USB Block Erupter, but there's a lot of software/hardware variants out there.

Users restarting their miners or new users first connecting might experience intermittent connection drops over the next few hours.  They're spaced far enough apart for any average miner to get a share in easily and get redirected to the appropriate backends, at which point they're safe from the drop.

The new Stratum code is getting its first deployment on the botnet/ddos validation frontend (stratum.btcguild.com).  Since this server is only actually used by users for a few seconds (time to get one share submission), it's the safest place to give this new code a live test against the onslaught of botnet miners always trying to access the pool.


Sorry if anybody here is affected by it, but like I stated above, this server shouldn't actually affect anybody unless they just happened to restart their miners as the server was being swapped between the old and new server processes.




99.7% acceptance rate is pretty good.  My general target for users is 99.5% or better before there may be a problem (and in some cases that problem is just the firmware/hardware they're using and the reject rate will follow them on any pool).

The setting on the dashboard only sets your *minimum* difficulty level, so eventually you'll always end up at 32 if you continue to use 4 blades via 1 proxy.  If you plan to always use 4 through one proxy, you should probably just set it to 32.  If you think you might change that setup later, then you might consider setting it to 8.

None yet.  Ran into a few fun problems in the last 24 hours when starting to do some actual testing of the server against bfgminer & cgminer.  Packet-level junk was interfering with their ability to parse the JSON >_<.  Got that fixed, and now I'm just moving through the code in the same process a miner interacts with the server.

Very close to having the SHA256 side done now.  All that's left is the actual pushing of shares to bitcoind/namecoind when a block is solved, logging the shares to the DB (already tested the function just haven't plugged it in), and then memory cleanup when clients disconnect.

Unfortunately, it looks like Scrypt Guild will probably not be ready until sometime next week.  While I'll probably have a working pool server, I won't have any kind of database/web frontend ready for it to plug into so people can setup accounts, monitor stats, and actually get paid.

Can't call it a roll yet, but we've finally broken the streak of < 100% shifts.

As far as I'm aware, it will not affect it.  The stratum proxy doesn't care about network difficulty.

Yeah, not a very nice couple of days.  Funnily, it's almost a perfect inverse of the luck we had from November 8 to November 12.  4 days in a row of +25 to 30%, and now we're at ~3 days of -25 to 30%.

Splitting into multiple pools doesn't change anything.  The same with allowing miners to redirect cloud mining.  They still control the hash rate.

GBT can be used in that way for solo mining.  Actually, all pools use GBT to build their blocks.  But no pool implementation has been developed that allows miners to pick & choose transactions with GBT, mostly because it would require WAY too much bandwidth since miners would have to submit shares including the transactions used to build their share.

Every TX you include would be 64 bytes of data you have to send to the pool (since current GBT implementation communicates with ASCII, so 32-bytes represented as hex pairs = 64 bytes), plus a little more for markup.  The pool would then have to use a *lot* of extra CPU power to identify all those transactions in its own list and build the merkleroot to then validate your share.

If you go further, and allow miners to pick transactions from their local bitcoind, you need to send the entire raw block worth of data to the pool since the pool might not have the raw tx data for the transactions you're picking, preventing it from matching the data with just the transaction hashes.

BTC Guild is ~25% of the full network.  GHash.io is in the upper 30s of the full network.  BTC Guild's percent of publicly available hashrate (private farms decrease the network share of all public hashrate pools) has not changed in months.

Ghash.io owns the private part.  Buying it on cex.io doesn't mean you own it unless you trade it for physical hardware.  They get the best of both worlds, they own/control the hardware, while people pay more than it will ever mine on cex.io.  But if they weren't selling it for amounts above what it can ever produce, they'd still own that hash rate and have it mining for themselves.

They also charge a "maintenance fee" on the cloud mining that is far greater than the actual cost to maintain it.


That pie chart has always been useless.  BTC Guild is ~25% of network hash rate.  24-hour, hell, even 4-day charts are USELESS because they are heavily influenced by luck.  http://blockorigin.pfoe.be is accurate for BTC Guild's share of the network over the last 2016 blocks.  They're not right on Ghash.io currently because they haven't been able to catch up on all the scraping due to blockexplorer.com being stuck for almost 2 weeks.  Still influenced by luck, but SIGNIFICANTLY more accurate than blockchain.info.

I'm not familiar with antminers personally, but I do know there's a ton of users on the pool using them with nowhere near that reject percentage.  What type of reject is it showing on the dashboard?  Some pools don't report duplicate shares as rejects (they just don't report them at all), which is something I know early ASICs and a *ton* of FPGAs had lots of problems with.

It's their private mining capacity.  It makes sense given their rise in hash rate compared to the overall network growth.  I had estimated last night in IRC that their private farm was in the 2 PH/s range since new mining capacity historically gets spread proportionally among pools (a pool with 20% network share gets ~20% of the hash rate from a company shipping a batch of miners), yet GHash.io's rate of growth was nearly double that of the rest of the network, which would mean ~50% of their hash rate was coming from their private farm.

If you discount GHash.io having the world's largest private mining farm, BTC Guild is nearly 1 PH/s ahead of their POOL.  Basically the only network share BTC Guild has lost has been the share of the network taken by the PRIVATE side of GHash.io.  ~20% of the entire network is privately controlled by GHash.io's private farm based on their post that stated 45% of their pool is their private farm, meaning instead of BTC Guild competing with other pools for roughly 85% of mining capacity (large private farms+solo miners were ~15% of the network before GHash.io), it is competing for roughly 65% of mining capacity.  Essentially 35% of the network pie chart is off limits, and BTC Guild has a little over a third of the network hash rate that can be obtained.

BTC Guild has continued to grow roughly in line with the other pools, which, discounting private farms, would mean it has not lost it's share of the public network.

A good move, but it doesn't actually prevent them from controlling 51%.  It actually makes it so now you won't know how much they really control.  At any point all that leased GH/s could be redirected at their own servers, because they still do CONTROL that hashrate.




So the sole reason they've gained any momentum (it sure isn't their support or uptime), and also the *only* way they're going to make people move to other pools, they have no intention of changing.