Bitcoin Forum
December 04, 2016, 10:31:21 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: Mine in multiple pools to reduce variance  (Read 17445 times)
Meni Rosenfeld
Donator
Legendary
*
Offline Offline

Activity: 1890



View Profile WWW
April 25, 2012, 04:08:57 PM
 #1

tl; dr: If you distribute your hashrate among multiple pools in proportion to their respective total hashrate, your variance will be as if you mine in one pool with the total hashrate of all pools.

This is a point I was trying to make in AoBPMRS (section "Proxy mining") but I get the feeling people miss that so I import it as a forum post.

Mining pools are used primarily to reduce variance, and the larger the pool, the more effective it is for this purpose. There is a simple way to decrease the variance further: Mine in multiple pools. For example, if EMC is 450 GH/s and Ozcoin is 700 GH/s you should use 40% of your compute power to mine for EMC and 60% to mine for Ozcoin, and then your variance will be as if you mined for a single 1150 GH/s pool. It doesn't need to be exact, of course; if in this case you do a 50%/50% split, it will be like a 1100 GH/s pool.

This not only helps variance for individual miners, but is healthier for the network. In the current standard usage, there is a "the rich get richer, the poor get poorer" tendency where larger pools are more attractive and thus grow even larger, and all else being equal, the equilibrium is a single huge pool (thankfully, all else is not equal). If miners adopt the proposed strategy, the tendency will be to maintain the status quo distribution, so pools can rise and fall based on their merits. Miners will enjoy the low variance of a single huge pool, without the centralization of power problem.

I'm not an expert on how to technically implement this but it shouldn't be much of a problem. If you have multiple GPUs you can simply point each to a different pool. I think there is local proxy software that allows you to split your hashrate between multiple pools, and if not one can be made.

Miners will probably not want to bother with creating accounts in many pools so will only use 2-3 pools. Ideally, there will be proxy pools that optimally distribute hashrate among as many hopping-proof pools as possible, allowing miners to enjoy low variance with a single pool account.

Appendix: More generally, if you put a portion of p1 of your hashrate at a pool with total hashrate H1, and so on, your variance will be like in a pool with hashrate of 1 / (p1^2/H1 + ... + pn^2/Hn). That the optimal value of H1+...+Hn is obtained when pi = Hi / (H1+...+Hn) is left as an exercise to the reader.

Pools which have other advantages (for example, take less fees or absorb variance with their reward method) should be given a higher weight than their hashrate indicates.

1EofoZNBhWQ3kxfKnvWkhtMns4AivZArhr   |   Who am I?   |   bitcoin-otc WoT
Bitcoil - Exchange bitcoins for ILS (thread)   |   Israel Bitcoin community homepage (thread)
Analysis of Bitcoin Pooled Mining Reward Systems (thread, summary)  |   PureMining - Infinite-term, deterministic mining bond
1480890681
Hero Member
*
Offline Offline

Posts: 1480890681

View Profile Personal Message (Offline)

Ignore
1480890681
Reply with quote  #2

1480890681
Report to moderator
1480890681
Hero Member
*
Offline Offline

Posts: 1480890681

View Profile Personal Message (Offline)

Ignore
1480890681
Reply with quote  #2

1480890681
Report to moderator
1480890681
Hero Member
*
Offline Offline

Posts: 1480890681

View Profile Personal Message (Offline)

Ignore
1480890681
Reply with quote  #2

1480890681
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480890681
Hero Member
*
Offline Offline

Posts: 1480890681

View Profile Personal Message (Offline)

Ignore
1480890681
Reply with quote  #2

1480890681
Report to moderator
1480890681
Hero Member
*
Offline Offline

Posts: 1480890681

View Profile Personal Message (Offline)

Ignore
1480890681
Reply with quote  #2

1480890681
Report to moderator
1480890681
Hero Member
*
Offline Offline

Posts: 1480890681

View Profile Personal Message (Offline)

Ignore
1480890681
Reply with quote  #2

1480890681
Report to moderator
Mousepotato
Hero Member
*****
Offline Offline

Activity: 896


Seal Cub Clubbing Club


View Profile
April 25, 2012, 10:57:21 PM
 #2

In for later...

Mousepotato
Red Emerald
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
April 26, 2012, 06:27:21 PM
 #3

Interesting.  Now we just need a miner that makes this easy to do.

enmaku
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
April 26, 2012, 06:33:27 PM
 #4

Seems to me that something like bitHopper could be tweaked to do this with the timesliced scheduler pretty easily. It's intended for pool hopping but if you disable all of the hoppable pools and just timeslice between the "fair" pools it should have the effect you're after.

Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
April 26, 2012, 06:36:53 PM
 #5

Well, shouldn't be too hard to modify a poolhopping client like Bithopper to hop based on valid shares submitted instead of round time...

In a world where every block is solved by a pool, you would get a payout on every block by mining at every pool in every round. In reality you can at least squash variance low enough to make it seem you're mining PPS every day or so I guess (= payouts per 24h are close to the theoretical value).

@enmaku:
Seems we both have the same idea... Wink

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
April 26, 2012, 07:06:53 PM
 #6

One caveat is cgminer seems to have inefficiencies when using in round robin mode.  Maybe that will be improved in future versions but a proxy server should work fine.
check_status
Full Member
***
Offline Offline

Activity: 196


Web Dev, Db Admin, Computer Technician


View Profile
April 26, 2012, 08:15:27 PM
 #7

If you have 2 rigs, 1 hashing 1500 MH/s and the other hashing at 2400 MH/s, you could point the 1 1500 rig at p2pool and the 2nd 2400 rig at EMC. Each rig has it's own instance of a miner, so just point each at a different pool. This is what I get out of the OP.

Is the proxy method being suggested for single rig miners?
What would be the minimum hashing ability that would benefit from splitting between 2 pools? Like those with one 6850/5830/5850?

For Bitcoin to be a true global currency the value of BTC needs always to rise.
If BTC became the global currency & money supply = 100 Trillion then ⊅1.00 BTC = $4,761,904.76.
P2Pool Server List | How To's and Guides Mega List |  1EndfedSryGUZK9sPrdvxHntYzv2EBexGA
Meni Rosenfeld
Donator
Legendary
*
Offline Offline

Activity: 1890



View Profile WWW
April 26, 2012, 08:30:36 PM
 #8

If you have 2 rigs, 1 hashing 1500 MH/s and the other hashing at 2400 MH/s, you could point the 1 1500 rig at p2pool and the 2nd 2400 rig at EMC. Each rig has it's own instance of a miner, so just point each at a different pool. This is what I get out of the OP.

Is the proxy method being suggested for single rig miners?
These are two means to the same end. The proxy method is more general, more accurate and potentially simpler. Manually pointing each rig to a different pool is more of a workaround if that's unavailable.

What would be the minimum hashing ability that would benefit from splitting between 2 pools? Like those with one 6850/5830/5850?
Assuming the pools use a reward method with moderate variance such as PPLNS with X=1, the miner needs to have a hashrate of at least on the order of H/D (where H is the pool hashrate and D is the difficulty) to see significant gains. So for example if the pool is 1 TH/s and D=1.5M then the miner needs to be more than about 1 MH/s, which is pretty much everyone.

1EofoZNBhWQ3kxfKnvWkhtMns4AivZArhr   |   Who am I?   |   bitcoin-otc WoT
Bitcoil - Exchange bitcoins for ILS (thread)   |   Israel Bitcoin community homepage (thread)
Analysis of Bitcoin Pooled Mining Reward Systems (thread, summary)  |   PureMining - Infinite-term, deterministic mining bond
Red Emerald
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
April 26, 2012, 08:49:42 PM
 #9

So will any changes need to be made to bithopper's code? Or should the user.cfg be enough?

I like this idea, not only for reducing variance, but also for having a bunch of backup pools (and of course p2pool Smiley )

It also looks like someone has done some math for us already.  https://bitcointalk.org/index.php?topic=77000.0

grue
Global Moderator
Legendary
*
Offline Offline

Activity: 1932



View Profile
April 26, 2012, 11:23:42 PM
 #10

brilliant!

It is pitch black. You are likely to be eaten by a grue.

Tired of annoying signature ads? Ad block for signatures
Red Emerald
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
April 27, 2012, 05:37:59 AM
 #11

So bithopper needs some tweaks to work well with p2pool.  I was getting a lot of rejects.  Anyone have any experience with this?

jkminkov
Hero Member
*****
Offline Offline

Activity: 534


View Profile
April 27, 2012, 07:34:46 AM
 #12

edit user.cfg with an entry line for each pool

percent:XX

where XX = 100 / # of pools enabled

for p2pool you should use p2pmining as it is the only node offering difficulty 1 shares, if you don't use diff 1 shares BH will end in a mess as it hops between pools counting shares(diff 1 shares)

Bleutrade
600 dollars in one place talking - Dudes, hooray, Bitcoin against us just one, but we are growing in numbers!
Red Emerald
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
April 27, 2012, 06:03:55 PM
 #13

edit user.cfg with an entry line for each pool

percent:XX

where XX = 100 / # of pools enabled

for p2pool you should use p2pmining as it is the only node offering difficulty 1 shares, if you don't use diff 1 shares BH will end in a mess as it hops between pools counting shares(diff 1 shares)
p2pmining.com would work, but it isn't needed.  You can just add "/1" after your worker's username and that does the same with the current version of p2pool.

However, I did set difficulty 1 shares and I still got a horrible reject rate.  I think its because of the massive amounts of longpolls.

JayCoin
Sr. Member
****
Offline Offline

Activity: 406


Crypt'n Since 2011


View Profile WWW
April 30, 2012, 03:28:02 AM
 #14

edit user.cfg with an entry line for each pool

percent:XX

where XX = 100 / # of pools enabled

for p2pool you should use p2pmining as it is the only node offering difficulty 1 shares, if you don't use diff 1 shares BH will end in a mess as it hops between pools counting shares(diff 1 shares)
p2pmining.com would work, but it isn't needed.  You can just add "/1" after your worker's username and that does the same with the current version of p2pool.

However, I did set difficulty 1 shares and I still got a horrible reject rate.  I think its because of the massive amounts of longpolls.

Use +1 not /1 to the end of your username.

Hello There!
Red Emerald
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
April 30, 2012, 04:42:42 AM
 #15

edit user.cfg with an entry line for each pool

percent:XX

where XX = 100 / # of pools enabled

for p2pool you should use p2pmining as it is the only node offering difficulty 1 shares, if you don't use diff 1 shares BH will end in a mess as it hops between pools counting shares(diff 1 shares)
p2pmining.com would work, but it isn't needed.  You can just add "/1" after your worker's username and that does the same with the current version of p2pool.

However, I did set difficulty 1 shares and I still got a horrible reject rate.  I think its because of the massive amounts of longpolls.

Use +1 not /1 to the end of your username.
Oh nice. Thanks.

Edit: Nope Sad Still > 30% rejects.

DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
April 30, 2012, 02:25:05 PM
 #16

Edit: Nope Sad Still > 30% rejects.

To clarify you are getting low reject rate using p2pool directly but higher reject rate when using the proxy?

If so the proxy may be adding too much latency.  Is the reject rate higher on other pools when also using p2pool?  If so the proxy may be getting "confused" by p2pool LP (~98% of p2pool LP involve no block change).
Red Emerald
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
April 30, 2012, 04:50:31 PM
 #17

Edit: Nope Sad Still > 30% rejects.

To clarify you are getting low reject rate using p2pool directly but higher reject rate when using the proxy?
Correct

Quote
If so the proxy may be adding too much latency.  Is the reject rate higher on other pools when also using p2pool?  If so the proxy may be getting "confused" by p2pool LP (~98% of p2pool LP involve no block change).
I'm pretty sure it has to do with LP since the reject rate is horrible for all the pools when p2pool is enabled. I don't know what to do to BitHopper to make it not confused. I was going to try a different mining proxy, but none of them seemed actively maintained.

enmaku
Hero Member
*****
Offline Offline

Activity: 742



View Profile WWW
May 07, 2012, 06:57:17 PM
 #18

Aside from the p2pool issue, does anyone have results to report?

organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1946


Poor impulse control.


View Profile WWW
November 17, 2013, 10:27:27 AM
 #19

This is a really important post right now, so I'm bumping this.

GHash.IO's recent hashrate increases mean most small pools are taking a big variance hit. As miners miners decide they can't take the variance on the smaller pools, they will move to the larger pools increasing the larger pools proportion of the network even further.

So now is a good time to remind miners that "load balancing" - mining on multiple pools, even small ones - will significantly reduce your variance.

Edit: Maybe this should also be stickied?

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
Meni Rosenfeld
Donator
Legendary
*
Offline Offline

Activity: 1890



View Profile WWW
November 17, 2013, 11:41:59 AM
 #20

As miners miners decide they can't take the variance the variance on the smaller pools
Duplicate duplicate?

So now is a good time to remind miners that "load balancing" - mining on multiple pools, even small ones - will significantly reduce your variance.
Are there software tools to facilitate this? To use the smaller pools effectively they need a smaller proportion of the miner's hashrate.

Anyway eventually a switch to Multi-PPS will be needed.

1EofoZNBhWQ3kxfKnvWkhtMns4AivZArhr   |   Who am I?   |   bitcoin-otc WoT
Bitcoil - Exchange bitcoins for ILS (thread)   |   Israel Bitcoin community homepage (thread)
Analysis of Bitcoin Pooled Mining Reward Systems (thread, summary)  |   PureMining - Infinite-term, deterministic mining bond
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!