Bitcoin Forum

Bitcoin => Pools => Topic started by: -ck on August 15, 2013, 12:54:37 AM



Title: Suggestion for how to choose a pool difficulty for miners.
Post by: -ck on August 15, 2013, 12:54:37 AM
Most pools are moving to a variable difficulty or minimum variable difficulty setting to avoid the bandwidth flood that occurs with ASIC hardware, but a lot offer the option of setting a fixed target or the minimum yourself per worker. There is an endless stream of questions on almost every separate pool's thread on what difficulty they should set if setting it manually. Note that you CANNOT EARN MORE by tweaking the difficulty.

So to make it easier, I'm going to give you a mathematically suggested strategy - this is all my opinion based on the maths though since there is no one true answer.

Setting the difficulty low for your hardware risks causing floods of bandwidth problems for both the miner and the pool operators.
Setting the difficulty high for your hardware causes no potential for communication problems, but the higher the difficulty, the more variance your payouts will see (i.e. your pay will fluctuate more) but it will average out to the same pay over time.
No bitcoin mining hardware device currently in existence uses the diff internally - they all mine at diff 1 and the mining software filters out results below what your pool has set it to so it makes no difference what hardware you use as to what diff you mine at.

i.e. only hashrate matters.

So we definitely don't want the difficulty too low, but what should we use as the cut off for how high to set the diff?

Organofcorti did a wonderful analysis of this last year:
http://organofcorti.blogspot.com/2012/10/71-variable-pool-difficulty.html

There is no magic endpoint to choose since the graph of share return rate versus variance is a curve - however - above a certain share return rate, 1% variance is small enough that it is much less than the pool's own luck's variance, and unless you were mining PPS there's a good chance you wouldn't be able to distinguish the difference on any one day. You also need a much bigger jump in share return rate before you can get below that 1% variance.

So my suggestion is to aim for a share return rate no higher than the highest difficulty that keeps you at 1% variance if you are setting a "minimum difficulty" or perhaps the value that keeps you under 1.2% for a fixed difficulty. This can be worked out easily if you know your hashrate without even testing it.

So our target is a diff that returns 20-25 shares per minute.
Every 71.6MH/s returns one share per minute - therefore every GH/s returns about 14 shares per minute.

A share return rate of 20 needs a diff of 1 per 1.432GH/s if setting a fixed difficulty.
A share return rate of 25 needs a diff of 1 per 1.79GH/s if setting a minimum difficulty.

Most pools are allowing you to set a fixed diff, so let's keep this final recommendation as simple as possible, and aim for 20 shares per minute. That's a diff of 1 per 1.432GH/s. The accuracy of this maths is not that important since the final endpoint is arbitrary anyway so we can knock off a few significant digits off our final recommendation.

TL:DR
Set your diff to your worker's hashrate in GH / 1.4


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: -ck on August 15, 2013, 01:02:48 AM
So taking it from the other perspective, I believe the pools should by now be forcing a minimum difficulty to prevent an effective DoS from a high hashrate miner. I would suggest they choose the higher value in my range, i.e. 25 shares per minute. The higher value also allows some leeway for the fact that calculating miner hashrate from share return rate is inherently inaccurate. The cyclical problem also exists that the lower the target the pool sets, the less accurately it will predict the miner's hashrate.

TL/DR:
Pools should force a minimum difficulty of miner hashrate in GH / 1.8


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: Graet on August 15, 2013, 01:04:21 AM
nice work
cheers ckolivas :)


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: eleuthria on August 15, 2013, 03:09:29 AM
I posted a similar guideline in Slush's thread when somebody was asking.  My rule of thumb for miners has always been:

Minimum Diffulcty = Closest power of 2 (rounded down) to your GH/s.  1-2 = 1.  2-4 = 2.  4-8 = 4.  8-16 = 8, etc.

This keeps your SPM between 15 and 30 (based on how close you are to the next cutoff).  This happens to be right in line with what almost every pool currently uses for vardiff.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: -ck on August 15, 2013, 05:15:26 AM
I posted a similar guideline in Slush's thread when somebody was asking.  My rule of thumb for miners has always been:

Minimum Diffulcty = Closest power of 2 (rounded down) to your GH/s.  1-2 = 1.  2-4 = 2.  4-8 = 4.  8-16 = 8, etc.

This keeps your SPM between 15 and 30 (based on how close you are to the next cutoff).  This happens to be right in line with what almost every pool currently uses for vardiff.
Yes I saw and since the question comes up at regular intervals I thought I'd post a comprehensive explanation as to why and avoid repeating the discussion over and over.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: organofcorti on August 15, 2013, 07:48:07 AM
I really like your idea, Con. You and eleuthria have both come up with a much simpler way to allow miners to choose a share submission difficulty than big tables of data. Whether a miner chooses the the values you suggest, or goes for the difficulty = ghps to ghps/2 ( I know it's not that exactly, but I wanted to make clear that eleuthria's method includes your values), miners will experience minimal daily variance.

It made me think about an approximation that would allow miners to select SPM based on the amount of daily variation in hashrate they want to experience. It's a bit more complicated than either of your suggestions but might be suitable for miners who want a bit more control.

Code:
SPM = 1/(19 * v)^2

where:
SPM = shares per minute
v = percentage variation from mean, ie +/- 0.01 = +/- 1%

If you want to choose difficulty rather than SPM, here's another approximation:

Code:
d = H*(71 * v)^2

where:
d = share difficulty
H = hashrate in Ghps
v = percentage variation from mean, ie +/- 0.01 = +/- 1%

The results are approximate but reasonably close, and the percentage range is within a 95% confidence interval. The results are less accurate after about +/- 20%, but less than this they are very close to the actual values.

Some examples:
* If you want +/- 1% then SPM = 1/(19 * 0.01)^2 = 27.7 and if your hashrate is 10Ghps then choose d = 10*(71 * 0.01)^2
* If you have a hashrate of 250 Ghps and you're happy with a daily variation in hashrate of +/- 5%, then SPM = 1/(19 * 0.05)^2 = 1.108 and choose d = 250*(71 * 0.05)^2 = 3150.


I know they're not as simple or clean as either of your suggestions, but hard as I try I just can't do simple ;)  As I say it might be useful for experienced miners who know how much daily variation they can take. If I was mining again I'd be ok with a daily variation in hashrate of +/- 2 or 3%.

What do you guys think? Too complicated? Might be ok to use in some kind of website app though.

Edit Also, thanks for explaining things so well. Regardless of how a miner wants to set their difficulty, if they read your post they'll have a nicely intuitive understanding of what changing difficulty actually does.



Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: -ck on August 15, 2013, 08:39:46 AM


Currently getting 35-40 shares per hour on Diff 128 , to average 25 SPH what should I set the difficulty to ?.



A share return rate of 20 needs a diff of 1 per 1.432GH/s if setting a fixed difficulty.
A share return rate of 25 needs a diff of 1 per 1.79GH/s if setting a minimum difficulty.


I didn't think I could make the advice any simpler than that?


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: organofcorti on August 15, 2013, 08:53:06 AM


Currently getting 35-40 shares per hour on Diff 128 , to average 25 SPH what should I set the difficulty to ?.



A share return rate of 20 needs a diff of 1 per 1.432GH/s if setting a fixed difficulty.
A share return rate of 25 needs a diff of 1 per 1.79GH/s if setting a minimum difficulty.


I didn't think I could make the advice any simpler than that?

No, I don't think it's possible to make it any simpler. Maybe he doesn't know his hashrate? ;)

In that case set difficulty to around 35/25*128 to 40/25*128

Edit Hey! Where'd he go?



Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: aneutronic on August 15, 2013, 09:58:25 AM
Great thread guys, this helps a lot.  Thanks!


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: bitcoindaddy on August 15, 2013, 07:05:24 PM
Too bad cgminer doesn't show average shares per minute (hint hint).

My rule of thumb is - if a minute's worth of shares fit in the terminal display, I'm okay.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: Trongersoll on August 15, 2013, 09:05:55 PM
Too bad cgminer doesn't show average shares per minute (hint hint).

My rule of thumb is - if a minute's worth of shares fit in the terminal display, I'm okay.

what if you resize your display? ::)


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: JBT on August 16, 2013, 07:55:49 PM
if my rate is 370MH i should set my diff to like below 100? or am i reading this wrong?


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: zamot on August 16, 2013, 08:04:33 PM
if my rate is 370MH i should set my diff to like below 100? or am i reading this wrong?
You should set it to 1.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: JBT on August 16, 2013, 08:05:46 PM
if my rate is 370MH i should set my diff to like below 100? or am i reading this wrong?
You should set it to 1.

ugh so the suggested setting of 128 for my GPU been waisting my time


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: eleuthria on August 16, 2013, 08:27:27 PM
if my rate is 370MH i should set my diff to like below 100? or am i reading this wrong?
You should set it to 1.

ugh so the suggested setting of 128 for my GPU been waisting my time

Where did you get a 128 difficulty suggestion for a GPU?  Something tells me you mixed up MH with GH.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: JBT on August 17, 2013, 09:03:04 AM
if my rate is 370MH i should set my diff to like below 100? or am i reading this wrong?
You should set it to 1.

ugh so the suggested setting of 128 for my GPU been waisting my time

Where did you get a 128 difficulty suggestion for a GPU?  Something tells me you mixed up MH with GH.

On the bitcoin.cz account page, now with a setting of 2 getting io errors and stale counts.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: JBT on August 17, 2013, 09:38:32 AM
bit of a noob question, if you solo mine at 370MH the chance of actually finding a block before someone else would be  about 0 say running 2-4 months and someone else finds the block all your time would have been wasted right?


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: dbitcoin on August 18, 2013, 12:04:03 AM
bit of a noob question, if you solo mine at 370MH the chance of actually finding a block before someone else would be  about 0 say running 2-4 months and someone else finds the block all your time would have been wasted right?
Your chance is very low (like winning lottery jackpot), so yes, all this time is wasted.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: JBT on August 18, 2013, 08:48:09 AM
bit of a noob question, if you solo mine at 370MH the chance of actually finding a block before someone else would be  about 0 say running 2-4 months and someone else finds the block all your time would have been wasted right?
Your chance is very low (like winning lottery jackpot), so yes, all this time is wasted.

unless you can do 50 to 500GH/s but that boat have sailed.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: faiza1990 on August 18, 2013, 09:03:50 AM
great its works for all cheers ???


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: -ck on August 21, 2013, 01:13:54 PM
SNIP

I know they're not as simple or clean as either of your suggestions, but hard as I try I just can't do simple ;)  As I say it might be useful for experienced miners who know how much daily variation they can take. If I was mining again I'd be ok with a daily variation in hashrate of +/- 2 or 3%.

What do you guys think? Too complicated? Might be ok to use in some kind of website app though.
I like it, but I really doubt that there are many miners who will really go for this level of control over it. I've seen enough very experienced miners get confused with what effect high diff shares have on your mining already...


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: -Redacted- on October 09, 2013, 10:30:53 PM
bit of a noob question, if you solo mine at 370MH the chance of actually finding a block before someone else would be  about 0 say running 2-4 months and someone else finds the block all your time would have been wasted right?

At 370 Mhz you would have a 50% chance of finding a block in about 7.5 years, not counting difficulty changes in the meantime.  So, if the difficulty didn't change,  over a long period of time - measured in hundreds of years, you should be finding one block about every 7.5 years on average.

And it's not THE block you are finding, because that changes every few minutes, and everything starts over again with the same probability - every time the current block changes  (say, because somebody else solved the current block, which should happen about once every 6 minutes) you get to start over trying to solve the new one...


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: gmaxwell on October 18, 2013, 11:29:00 PM
I don't see why people give a shit if their _daily_ income has only a 1% variation. Jesus Christ, do you all have mining hardware setups whos output exactly match the cost of your daily heroin fix or something?  :P

There are very few small to medium size businesses that have variation against expected within 1% even on a timescale of months.

I mean, sure, if the bandwidth isn't a concern and the pool doesn't care to charge people based on their actual load, then by all means, why not lower.

But caring about a daily 1% variation is just further confirmation to me that y'all are crazy. :)


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: -ck on October 19, 2013, 01:54:09 AM
I don't see why people give a shit if their _daily_ income has only a 1% variation. Jesus Christ, do you all have mining hardware setups whos output exactly match the cost of your daily heroin fix or something?  :P

There are very few small to medium size businesses that have variation against expected within 1% even on a timescale of months.

I mean, sure, if the bandwidth isn't a concern and the pool doesn't care to charge people based on their actual load, then by all means, why not lower.

But caring about a daily 1% variation is just further confirmation to me that y'all are crazy. :)
Wait wait wait, you're calling people who spend $5000 on a money making machine that in its entire lifetime will never make $3000 or so crazy? You're crazy.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: organofcorti on October 20, 2013, 05:10:23 AM
I don't see why people give a shit if their _daily_ income has only a 1% variation. Jesus Christ, do you all have mining hardware setups whos output exactly match the cost of your daily heroin electricity fix or something?  :P

<snip>

It's certainly getting there.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: bitminerinthecloud on February 05, 2014, 12:11:57 AM
Hmm, sorry for my ignorance, but increasing the difficult as the OP is saying would decrease bandwith requirements for both the miner and the miner operator by how much percentage? What much bandwith would be needed for 1 GH/s (as with difficult 1 and as with suggested)?


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: Aurum on February 24, 2014, 06:33:15 PM
Set your diff to your worker's hashrate in GH/1.4

So add up the hashrate of all my miners that contribute work to a pool under the same worker name. Is that right ???

E.g., say I have 5 AntMiner S1s named Aurum.Ant then I should specify 1,000/1.4 = 512 (rounded down to the nearest power of 2) as my minimum diff.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: -ck on February 24, 2014, 09:26:34 PM
Set your diff to your worker's hashrate in GH/1.4

So add up the hashrate of all my miners that contribute work to a pool under the same worker name. Is that right ???

E.g., say I have 5 AntMiner S1s named Aurum.Ant then I should specify 1,000/1.4 = 512 (rounded down to the nearest power of 2) as my minimum diff.
Yes I would suggest the total hashrate, not each device's.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: hashfun on April 23, 2014, 02:35:56 PM
Set your diff to your worker's hashrate in GH/1.4

So add up the hashrate of all my miners that contribute work to a pool under the same worker name. Is that right ???

E.g., say I have 5 AntMiner S1s named Aurum.Ant then I should specify 1,000/1.4 = 512 (rounded down to the nearest power of 2) as my minimum diff.
Yes I would suggest the total hashrate, not each device's.

i have 4 Devices Each 180-GHz so my total is 720-GHZ
all my 4 worker in same pool,

but in pool they force me to define Difficulty each worker,
so what u suggest should i select each worker diff : 256


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: -ck on April 23, 2014, 02:38:13 PM
Set your diff to your worker's hashrate in GH/1.4

So add up the hashrate of all my miners that contribute work to a pool under the same worker name. Is that right ???

E.g., say I have 5 AntMiner S1s named Aurum.Ant then I should specify 1,000/1.4 = 512 (rounded down to the nearest power of 2) as my minimum diff.
Yes I would suggest the total hashrate, not each device's.

i have 4 Devices Each 180-GHz so my total is 720-GHZ
all my 4 worker in same pool,

but in pool they force me to define Difficulty each worker,
so what u suggest should i select each worker diff : 256
Combine the total hashrate and give the same diff to all of them, so about the same as the other recommendation of 512


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: hashfun on April 23, 2014, 03:15:37 PM
Set your diff to your worker's hashrate in GH/1.4

So add up the hashrate of all my miners that contribute work to a pool under the same worker name. Is that right ???

E.g., say I have 5 AntMiner S1s named Aurum.Ant then I should specify 1,000/1.4 = 512 (rounded down to the nearest power of 2) as my minimum diff.
Yes I would suggest the total hashrate, not each device's.

i have 4 Devices Each 180-GHz so my total is 720-GHZ
all my 4 worker in same pool,

but in pool they force me to define Difficulty each worker,
so what u suggest should i select each worker diff : 256
Combine the total hashrate and give the same diff to all of them, so about the same as the other recommendation of 512

i test some Result and i want to share here

when i set  512 =  too many stale
when i set  256 =  less stale
when i set  128 =  few stale

is there in my internet problem when increase the diff stale increased.


Title: Re: Suggestion for how to choose a pool difficulty for miners.
Post by: -ck on April 23, 2014, 09:02:40 PM
Stale rate should have absolutely nothing to do with diff setting unless your software is broken. Some hardware came with broken drivers that behaved differently with different diff work but that's a software problem on their part, it is not the diff at fault.