Aexoden (OP)
Newbie
Offline
Activity: 53
Merit: 0
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 04:28:18 AM |
|
The past few weeks, I have been slowly working on modifying poclbm to be a full-fledged automatic pool hopper. I think it's finally to the point where it might be usable enough to distribute, but I'm still choosing to label it as EXPERIMENTAL. MotivationMany mining pools are still using the inherently unfair proportional payout system. My primary intent is to encourage pool operators to migrate their pools to fair systems. It seems as if the only way this will happen is if miners themselves see that the proportional system is unfair. (Unfortunately, the fair systems often seem unfair at first glance, making explanations more difficult.) MethodologyOriginally, poclbm-autohop operated similarly to bitHopper, retrieving share counts from the various pools, and calculating a utility for each one. Recently, however, instead of replacing their broken payout systems with provably fair systems, pools have begun to implement various obfuscation measures, such as delayed statistics. It is apparent that another approach is needed. Using a local copy of pident to identify the pools that have claimed each block, I calculate the probability that each pool's last block occurred at a given time index. This probability is currently based on each pool's relative hash rate, but work is progressing on improving these probabilities using a heuristic scoring system. (Limited tests show that it may ultimately be possible to predict the finder of a block with at least 30-40% accuracy. Given that most pools make up less than 10% of the network, this is a marked improvement.) I upload these probabilities as well as estimated hash rates for each pool and it available via a JSON API. (As a quick example, if there's a 10% chance that a share will be worth 10x, and a 90% chance it will be worth 1x, its expected value is 1.9x.) poclbm-autohop itself then downloads these estimated hash rates and probabilities, and uses them to calculate the utility of each pool. This method will never be as accurate as using the share counts directly, but since many pools are obfuscating or delaying this information, this new approach may be the best we can do. Caveats- The estimated hash rates are currently slightly inaccurate for some pools. This will improve within the first week of August, as my local copy of pident collects more data.
- The current code estimates hoppers provide a 30GH/s boost above the average. This is probably a lowball figure, so the utility of pools may be overestimated.
- The probabilities in the API are currently based only on each pool's share of the network.
In addition, while the logical next step for proportional pools would be to replace their broken payout systems, I fully expect pool operators will instead move to no longer report ownership of blocks. Personally, I have to wonder about any pool that would resort to this level of obfuscation rather than simply change their payout model. (Bitclockers already seems to obfuscate their blocks in this way.) Bugs- I don't know if it's something my code did or just a problem with poclbm itself, but occasionally it might go into massive reject sprees. This only happens with some pools, so who knows.
- Sometimes things just stop working properly, causing the miner to miss long polls or such, and I'm probably too lazy to dig into poclbm to fix all its problems. I recommend restarting the program periodically (once every day or two, perhaps).
There are probably others that I'm not thinking of, but it should largely work, in any case. UsageThe code is available via GitHub: https://github.com/aexoden/poclbmIt should run in any environment that poclbm itself will run, but it does have two additional dependencies: scipy and httplib2. Both should be readily available for both Linux and Windows. For detailed information, you should definitely consult the README.mkd file. Basically, however, it accepts the same parameters as standard poclbm, except it no longer takes a list of servers as a parameter. Instead, you need to create a pools.conf file, which contains a line for each pool. Each line should say the name of the pool, your worker username, and your worker password, separated by whitespace (I recommend tabs). You may list the pools in your order of preference. If two pools have the same utility, the first pool listed will be preferred. Only pools listed in your pools.conf file will be used. For the list of supported pools, please consult the README.mkd file. ConclusionComments, suggestions, and questions are welcomed. I can only hope this software will be obsolete very shortly.
|
|
|
|
Sukrim
Legendary
Offline
Activity: 2618
Merit: 1007
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 04:44:27 AM |
|
Have you tried to connect to all pools in the list, timing LongPollings and (as an easy start) assuming the pool from which the first LP comes has found the block? This could also be factored in additionally with your scoring approach and should probably give some better results.
|
|
|
|
Aexoden (OP)
Newbie
Offline
Activity: 53
Merit: 0
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 01:44:49 PM |
|
Have you tried to connect to all pools in the list, timing LongPollings and (as an easy start) assuming the pool from which the first LP comes has found the block? This could also be factored in additionally with your scoring approach and should probably give some better results.
I meant to write something to test that strategy, but I haven't gotten around to it yet. Though, the data from https://forum.bitcoin.org/index.php?topic=32664.0 suggests it would at best be another piece of the puzzle, rather than a complete solution. One of the problems is that you're still trusting pools to not obfuscate things. (At least one pool was talking about delaying long polls to known hoppers--I'm amazed at the effort being put into fighting pool hopping with methods other than changing reward systems.) Then again, we're still trusting pools to report which blocks they've solved accurately.
|
|
|
|
hawks5999
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 04:04:13 PM |
|
The hopping monster is getting to a Gojira size. Nice.
|
■ ▄▄▄ ■ ███ ■ ■ ■ LEDGER WALLET ████ ■■■ ORDER NOW! ■■■ LEDGER WALLET Smartcard security for your BTCitcoins ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ Decentralized. Open. Secure.
|
|
|
m3ta
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 05:21:24 PM |
|
I'm amazed at the effort being put into fighting pool hopping with methods other than changing reward systems. +1....
|
|
|
|
iopq
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 06:58:32 PM |
|
I'm amazed at the effort being put into fighting pool hopping with methods other than changing reward systems. +1.... +ONE MIRRION PPLNS is 0 variance for the pool operator and rewards every share equally for the miners
|
|
|
|
1bitc0inplz
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 112
Merit: 10
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 07:22:32 PM |
|
I'm amazed at the effort being put into fighting pool hopping with methods other than changing reward systems. +1.... +ONE MIRRION PPLNS is 0 variance for the pool operator and rewards every share equally for the miners Not to be picky, but PPLNS does *not* reward *every* share equally. Shares that fall out of the PPLNS window are worth 0. Just sayin' ![Wink](https://bitcointalk.org/Smileys/default/wink.gif) Though, I agree, Prop is broken. Miners need a fair reward system (*cough* ESMPPS *cough*), and fighting pool hoppers by hiding stats, faking stats, or anything other than just moving away from Prop is not solving the problem.
|
|
|
|
hawks5999
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 07:30:43 PM |
|
Agree with bitp op, PPLNS is probably the least fair even including Prop. With prop every share is worth 'something'. With PPLNS a lot of work can end up completely wasted and worth 0.
Bitp is a great backup pool by the way.
|
■ ▄▄▄ ■ ███ ■ ■ ■ LEDGER WALLET ████ ■■■ ORDER NOW! ■■■ LEDGER WALLET Smartcard security for your BTCitcoins ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ Decentralized. Open. Secure.
|
|
|
bitcoindaddy
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:15:25 PM |
|
I'm getting the following errors. I tried a full pools.conf at first, then reduced it down to a single mtred xxxx xxxx line and it still doesn't work for me.
./poclbm.py -d 3 -v -f 1 -w 128 Traceback (most recent call last): File "./poclbm.py", line 68, in <module> miner = BitcoinMiner(devices[options.device], options, VERSION, HttpTransport.HttpTransport) File "/home/user/Downloads/poclbm-aexoden/BitcoinMiner.py", line 37, in __init__ self.transport = transport(self) File "/home/user/Downloads/poclbm-aexoden/HttpTransport.py", line 21, in __init__ super(HttpTransport, self).__init__(miner) File "/home/user/Downloads/poclbm-aexoden/Transport.py", line 37, in __init__ say_line('Switching to {} with utility {:.3f}'.format(self.best_pools[0].name, self.best_pools[0].utility), show_server=False) ValueError: zero length field name in format
sudo apt-get install python-scipy python-httplib2 Reading package lists... Done Building dependency tree Reading state information... Done python-httplib2 is already the newest version. python-scipy is already the newest version. 0 upgraded, 0 newly installed, 0 to remove and 10 not upgraded.
|
|
|
|
Aexoden (OP)
Newbie
Offline
Activity: 53
Merit: 0
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:18:46 PM |
|
Not to be picky, but PPLNS does *not* reward *every* share equally. Shares that fall out of the PPLNS window are worth 0. Just sayin' ![Wink](https://bitcointalk.org/Smileys/default/wink.gif) Sure, any given share may not be paid, but the expected value for every share is (for the moment) 50 / difficulty. (In the case of mineco.in, it's actually a little greater, since they return transaction fees.) Remember, if (for example), N is set to difficulty / 2 (as at mineco.in), shares will be paid out with the following probabilities: - No payout: ~60.65%
- 1 payout: ~30.33%
- 2 payouts: ~7.58%
- 3 payouts: ~1.26%
- 4 or more payouts: ~0.18%
The weighted average here is 0.4999 payouts per share. (If I had bothered calculating X number of payouts up to infinity, the sum would be 0.50.) Since each share will be paid out double the expected 50 / difficulty reward (because N is half the difficulty), the expected value per share is 1.0. Some work may seem wasted, but over time, you'll get exactly what you expect to get. Ultimately, PPLNS definitely has higher variance than *SMPPS, but it's not statistically any more unfair. With a large pool like Deepbit, Slush, or BTC Guild, the variance would be quite low, even for intermittent miners. I also don't think it's vulnerable to the whole "unlucky streak killing the pool" problem. In any case, I currently treat PPLNS, Geometric, and SMPPS pools as "fair" in the hopper. Of course, if any SMPPS pool finds itself deep in a hole, I may have to adjust the utility of that pool.
|
|
|
|
Aexoden (OP)
Newbie
Offline
Activity: 53
Merit: 0
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:19:23 PM |
|
I'm getting the following errors. I tried a full pools.conf at first, then reduced it down to a single mtred xxxx xxxx line and it still doesn't work for me.
./poclbm.py -d 3 -v -f 1 -w 128 Traceback (most recent call last): File "./poclbm.py", line 68, in <module> miner = BitcoinMiner(devices[options.device], options, VERSION, HttpTransport.HttpTransport) File "/home/user/Downloads/poclbm-aexoden/BitcoinMiner.py", line 37, in __init__ self.transport = transport(self) File "/home/user/Downloads/poclbm-aexoden/HttpTransport.py", line 21, in __init__ super(HttpTransport, self).__init__(miner) File "/home/user/Downloads/poclbm-aexoden/Transport.py", line 37, in __init__ say_line('Switching to {} with utility {:.3f}'.format(self.best_pools[0].name, self.best_pools[0].utility), show_server=False) ValueError: zero length field name in format
Before I ask anything else, what version of Python are you using?
|
|
|
|
|
Aexoden (OP)
Newbie
Offline
Activity: 53
Merit: 0
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:29:34 PM |
|
2.6.6-2ubuntu2
I've pushed a fix that may solve your problem (due to using Python 2.6). I use Python3 for my personal projects almost exclusively, and only use python 2.7.x when I'm forced to use Python2. If the fix doesn't work, let me know, and I'll try to get a 2.6.x testing environment.
|
|
|
|
c00w
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:30:44 PM |
|
Um if by geometric you mean slush's algorithm it is hoppable.
|
1HEmzeuVEKxBQkEenysV1yM8oAddQ4o2TX
|
|
|
Aexoden (OP)
Newbie
Offline
Activity: 53
Merit: 0
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:31:50 PM |
|
Um if by geometric you mean slush's algorithm it is hoppable.
The Geometric method refers to Meni's scoring system.
|
|
|
|
Sukrim
Legendary
Offline
Activity: 2618
Merit: 1007
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:36:22 PM |
|
*SMPPS is broken/high risk of not returning 100% of expected value per share as soon as the pool hits negative, which should happen ~50% of the time.
PPLNS can choose arbitrarily large Ns (the highest useful N is 5 000 000 000, awarding each share with 1 Satoshi when blocks are found) if people really feel left out or are not able to get that they have a chance to get paid multiple times, equalizing the chance of not getting paid at all. All that is needed is a proper interface, that shows percentages instead of "expected payout", as PPLNS looks forward to define rewards, not so much backwards as prop.
That said, as soon as pool operators start delaying LongPolls they really start to hurt their own profit + their own miners. Just think of the fact, that every millisecond in delay lets the miners compute 100 000 hashes per 100 MH/s in vain and will create more stale shares. Also, if you just have to submit a valid solution every 10 minutes or so to count as an "active miner", one might be able to just pull a getwork from there every 10 minutes, solve + submit it, to not get throttled. The shorter this window, the more miners the operators hit + screw over, the longer, the easier it gets to fake a CPU miner.
|
|
|
|
bitcoindaddy
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:37:58 PM |
|
I've pushed a fix that may solve your problem (due to using Python 2.6). I use Python3 for my personal projects almost exclusively, and only use python 2.7.x when I'm forced to use Python2. If the fix doesn't work, let me know, and I'll try to get a 2.6.x testing environment.
Yes, it's working now, thanks
|
|
|
|
Clipse
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:38:29 PM |
|
This is getting interesting, may the most efficient hopper win ![Wink](https://bitcointalk.org/Smileys/default/wink.gif)
|
...In the land of the stale, the man with one share is king... >> ClipseWe pay miners at 130% PPS | Signup here : Bonus PPS Pool (Please read OP to understand the current process)
|
|
|
hawks5999
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 08:46:28 PM |
|
So when does hopping get its own section under mining?
|
■ ▄▄▄ ■ ███ ■ ■ ■ LEDGER WALLET ████ ■■■ ORDER NOW! ■■■ LEDGER WALLET Smartcard security for your BTCitcoins ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ Decentralized. Open. Secure.
|
|
|
bitcoindaddy
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 31, 2011, 10:11:02 PM |
|
Can you explain what the different fields mean on the last line of the display? Am I only getting an effective rate of 294 MHash/sec? ./poclbm.py -d 3 -v -f 1 -w 128 31/07/2011 18:05:09, Switching to bitcoins.lc with utility 1.057 31/07/2011 18:05:09, Current utilities: 31/07/2011 18:05:09, bitcoins.lc : 1.057 31/07/2011 18:05:09, arsbitcoin : 1.000 31/07/2011 18:05:09, bitpit : 1.000 31/07/2011 18:05:09, eligius : 1.000 31/07/2011 18:05:09, triplemining : 0.721 31/07/2011 18:05:09, mtred : 0.705 31/07/2011 18:05:09, btcguild : 0.630 31/07/2011 18:05:09, ozco.in : 0.390 31/07/2011 18:05:09, nofeemining : 0.251 31/07/2011 18:05:09, rfcpool : 0.248 31/07/2011 18:05:09, Setting server (xxxxx @ bitcoins.lc) bitcoins.lc 31/07/2011 18:05:11, LP connected to bitcoins.lc bitcoins.lc [426.863 MH/s (~294 MH/s Eff:0.10)] [Rej: 0/10 (0.00%)] [GW: 104 (Eff:0.10)]
|
|
|
|
|