Does pool hopping really work?

[slush]

That's what I meant. "Owned up to" is an American idiom meaning "publicly admitted".

Oh, then excuse my ignorance

[grue]

So if i wanted to pool hop, what kind of pool should i hop to? pay per share? proportional? score-based? I'm really interested in "improving" my efficiency as a miner

[Sukrim]

So if i wanted to pool hop, what kind of pool should i hop to? pay per share? proportional? score-based? I'm really interested in "improving" my efficiency as a miner

Score based ppols claim that they are more or less immune to pool hopping, you would look for share based pols probably.

A possible implementation would be a pool of pools, which moves around different pools depending on their solved blocks (conveniently announced in realtime by the pool via long polling).

One of the most difficult variables to get is the current hash rate of the pool, you would have to operate with older data + estimates. As new pools are currently popping up and miners join and leave, this number might not be too accurate (check out http://eligius.st/~artefact2/ for example, how much the hash rate fluctuates) thus screwing your calculations on how long to stay in each round.

In the end pool hopping is NOT cheating! You do all the work you have to do etc.
To NOT pool hop however harms your income by up to 28%.

As I don't have a server at my hands and I guess a dedicated pool-hopping pool might be popular for miners, but not pool operators (as they would get slammed with a lot of GH/s at the start of a round and then suddenly the miners move on to the next pool) I probably won't implement something like this in the public and would advise other to do the same.

[Meni Rosenfeld]

So if i wanted to pool hop, what kind of pool should i hop to? pay per share? proportional? score-based? I'm really interested in "improving" my efficiency as a miner

You always mine at the proportional pool which has the least shares in the current round. Once all such pools have >43.5% difficulty, you move to a score-based pool like Continuum.

In the end pool hopping is NOT cheating! You do all the work you have to do etc.
To NOT pool hop however harms your income by up to 28%.

Arguing on whether to call it "cheating" or "not cheating" is a red herring.
The facts are:
If one person does it, his payout is greater than his contribution to the pool (or to the Bitcoin network at large), and for others it's less. In this sense he does not do "all the work he has to do".
If everyone does it, all proportional pools freeze and people will mine in score-based / solo anyway.
So why add unfruitful complication by having proportional pools exists in the first place?

[Sukrim]

So why add unfruitful complication by having proportional pools exists in the first place?

Small score based pools get completely unattractive to be joined by anyone, if they haven't very recently found a block.


Score based pools are only working very well for 24/7 miners, if you don't mine all the time, you will loose income when using a score based pool to the 24/7 miners.
If you want to open a new pool, you better make it proportional and after some "critical mass" has been reached, switch to score based to ensure a steady and more fair income if you wish.

I personally wouldn't join a score based small pool where blocks are only found every 3-4 days, as the risk of "buying in" with (depending on my hashrate vs. pool hashrate) quite some BTC in the beginning is too high for me.

[smooth]

So why add unfruitful complication by having proportional pools exists in the first place?

One reason is that people like simple ("fair") pricing schemes even they aren't economically efficient.  This is an empirical result that applies to Homo Sapiens as opposed to Homo Economicus.

Put two pools side by side, one with each payout method and I'm sure a significant number of people will strongly prefer proportional even if they aren't going to pool hop and understand that they will make less as a result.  Think of it as a fee.

[Raulo]

Score based pools are only working very well for 24/7 miners, if you don't mine all the time, you will loose income when using a score based pool to the 24/7 miners.

This is not true. If you mine intermittently on score based pool you only increase your variance but the average income will be proportional to your contributed shares. You will quickly lose score if you stop mining but you quickly gain if you start. If you mine from the start of the round to the half time of the round, you'll get close to zero. But if you join at half time of the round, you'll get almost full payout while contributing only half of the shares. In long term, it'll all average out.

[Sukrim]

This is not true. If you mine intermittently on score based pool you only increase your variance but the average income will be proportional to your contributed shares. You will quickly lose score if you stop mining but you quickly gain if you start. If you mine from the start of the round to the half time of the round, you'll get close to zero. But if you join at half time of the round, you'll get almost full payout while contributing only half of the shares. In long term, it'll all average out.

This is not taking into account difficulty changes. If you're unlucky and miss payouts currently, there might (with increasing difficulty) be no way to make up for that loss via luck.

If a mining pool grows slower than the network, the payout will be less and less. Also there is the risk of people leaving the pool (see slush...) before you have made up for bad luck.


On the other hand, you might be lucky too, join late in an early round and then continue in a growing pool that keeps up with the global hashrate.

In the end ppl. mine in pools to reduce variance as far as possible - introducing variance is often not so cool. Otherwise no sane person would be mining with PPS on deepbit (or any other pool that charges fees).

[Raulo]

This is not taking into account difficulty changes. If you're unlucky and miss payouts currently, there might (with increasing difficulty) be no way to make up for that loss via luck.

And if you are lucky, you will be ahead. It all averages out.

By the way, difficulty changes are completely irrelevant if you mine regularly (it does not have to be constantly). I really don't know why people care about difficulty changes at all. The only thing that matters is what is your hashrate ratio relative to the current global hashrate. This is all continuous. If the current difficulty correspond to, say 3TH/s hashrate but the network is at 4TH/s, you will get the same number of coins per month as if the difficulty was corresponding to 4TH/s and hashrate equal to 4TH/s because in the former case the round will be shorter. 

[Sukrim]

And if you are lucky, you will be ahead. It all averages out. 

Over many people, not over a single miner!

In my opinion, there should be as little "luck" needed in pooled mining as possible.

There is already some variance that comes from pools not having 100% of the mining power of the network, but every further introduction of "luck" (or call it variance) introduces also "bad luck" for others.

Score based mining is imo only useful for pools that have ~5%+ of the network hashrate. This means that on average every ~3 hours a block will be solved, so people jumping in don't get screwed over too much due to being late on a profitable round.

If you have a small pool however where you can expect finished blocks only every few days or so, it might happen that an honest on-off miner might loose out significantly.

Maybe it might be useful to do some calculations which pool is recommendable based on your mining habits ("2 hours/day means you'll need to join a score based pool with at least x% of the network hash rate to minimize initial variance")?

[Meni Rosenfeld]

And if you are lucky, you will be ahead. It all averages out. 

Over many people, not over a single miner!

No. Over a single miner. His expected payout is equal to his contribution. So if he keeps his intermittent mining over a long period of time, his total reward will be close to his total contribution.

Score based mining is imo only useful for pools that have ~5%+ of the network hashrate.

Can you show your calculation why this is so?

so people jumping in don't get screwed over too much due to being late on a profitable round.

I think you're confused about how score-based works. If you join at a time which, in retrospect, is late in the round, you will get more reward. Of course, you don't know beforehand when the round will end, so whenever you join your expectation is the same.
In fact, it is true for proportional pools that if you join late in a round you'll get screwed.

If you have a small pool however where you can expect finished blocks only every few days or so, it might happen that an honest on-off miner might loose out significantly.

Again, it might happens that he has bad luck, but his expectation is equal to his contribution.

Maybe it might be useful to do some calculations which pool is recommendable based on your mining habits ("2 hours/day means you'll need to join a score based pool with at least x% of the network hash rate to minimize initial variance")?

This will depend on greatly on the miner's risk averseness. Because in most cases variance is less important than expectation, and because the variance of score-based is not that high, the best recommendation is to always use score-based.

[smooth]

Because in most cases variance is less important than expectation

It seems hard to make that argument in this context.  If people wanted to maximize expectation, they would just solo-mine and avoid pool fees, pool downtime, stale shares, etc.  In most cases the variance is extremely "large" but in absolute terms the amount of money involved for a hobby miner is small enough that it doesn't really matter.   

I will grant that many hobby-scale miners don't understand this and don't act that way in practice.

and because the variance of score-based is not that high

True.  But still you are ignoring mental accounting costs.  People like equal shares because they are simple and easy to understand.  If you are hoping for pool participants to understand game theoretic indifference, you're going to be disappointed.  The constant push back (even if mathematically incorrect) over score-based pools being "unfair" should be ample illustration of this.

[anon4758]

Look, I'll confess (anonymously), I pool hop and it works.

Pool hopping is not cheating - it is optimizing. It is the responsibility of pool operators to implement algorithms that are fair. I'd think of all people, cypherpunks like bitcoin miners would be the first ones to recognize this fact. If you place all your trust into math, don't suddenly complain about fairness if your math turns out to not be good enough.

Hopping away from a single pool (at 43% of mean round time) earns 30% more. Hopping between a couple pools (I have 4 currently automated) earns me 70% more. If you'd automate a dozen pools, you could earn more than 100% extra.

Slush's pool is not secure because he messed up his algorithm. I earn 20% more on his pool. Deepbit pool is not secure, because they got a large market share. If no other pool solved the block (which I know automatically), it was most likely deepbit, even if their statistics are delayed by an hour.

Pool hopping is not too common. I've analyzed the statistics of the pools I use, and less than 5% of users pool hop, so "unsofisticated" pool miners only loose 2-5% of income. After publishing this post, I expect hopping to become more common.

What I want to happen is for pool miners to implement mathematically-fair payment algorithms. This is the solution best for the community in the end, but not manually enforcing no-hopping rules or withholding information. There is a complicated score-based algorithm published somewhere on this forum (not slush's algorithm) that seems secure, though I didn't check it in too much detail. There is a much simpler fair algorithm though, which I want you to confirm: pay for all shares submitted in the last 1 hour of a round. If a block is found sooner than 1 hour into a round, then rollover the extra time and pay for the unpaid shares of the previous round, or the previous round before that (if the previous round was shorter than an hour as well), and so on. This way, each share has equal expected utility regardless of when it is submitted - early on or late in a round. Late shares in long rounds are subsidized by early shares in short rounds. You can use any time interval shorter than the mean round time. I'd recommend 1/4 of the mean time, e.g. 1 hour for a 130GHash/s pool.

[EpicBacon]

Look, I'll confess (anonymously), I pool hop and it works.

Pool hopping is not cheating - it is optimizing. It is the responsibility of pool operators to implement algorithms that are fair. I'd think of all people, cypherpunks like bitcoin miners would be the first ones to recognize this fact. If you place all your trust into math, don't suddenly complain about fairness if your math turns out to not be good enough.

Hopping away from a single pool (at 43% of mean round time) earns 30% more. Hopping between a couple pools (I have 4 currently automated) earns me 70% more. If you'd automate a dozen pools, you could earn more than 100% extra.

Slush's pool is not secure because he messed up his algorithm. I earn 20% more on his pool. Deepbit pool is not secure, because they got a large market share. If no other pool solved the block (which I know automatically), it was most likely deepbit, even if their statistics are delayed by an hour.

Pool hopping is not too common. I've analyzed the statistics of the pools I use, and less than 5% of users pool hop, so "unsofisticated" pool miners only loose 2-5% of income. After publishing this post, I expect hopping to become more common.

What I want to happen is for pool miners to implement mathematically-fair payment algorithms. This is the solution best for the community in the end, but not manually enforcing no-hopping rules or withholding information. There is a complicated score-based algorithm published somewhere on this forum (not slush's algorithm) that seems secure, though I didn't check it in too much detail. There is a much simpler fair algorithm though, which I want you to confirm: pay for all shares submitted in the last 1 hour of a round. If a block is found sooner than 1 hour into a round, then rollover the extra time and pay for the unpaid shares of the previous round, or the previous round before that (if the previous round was shorter than an hour as well), and so on. This way, each share has equal expected utility regardless of when it is submitted - early on or late in a round. Late shares in long rounds are subsidized by early shares in short rounds. You can use any time interval shorter than the mean round time. I'd recommend 1/4 of the mean time, e.g. 1 hour for a 130GHash/s pool.

[Meni Rosenfeld]

@anon4758 - this is some very useful info, thanks for sharing!

Pool hopping is not cheating - it is optimizing. It is the responsibility of pool operators to implement algorithms that are fair. I'd think of all people, cypherpunks like bitcoin miners would be the first ones to recognize this fact. If you place all your trust into math, don't suddenly complain about fairness if your math turns out to not be good enough.

I generally agree, I mentioned elsewhere that I don't have a beef with hoppers, rather with operators of proportional pools.

Hopping away from a single pool (at 43% of mean round time) earns 30% more. Hopping between a couple pools (I have 4 currently automated) earns me 70% more. If you'd automate a dozen pools, you could earn more than 100% extra.

In the theoretical limit of infinitely many proportional pools and perfect hopping, I think you get log(difficulty) times normal, which currently is X13, or 1200% extra.

Slush's pool is not secure because he messed up his algorithm. I earn 20% more on his pool. Deepbit pool is not secure, because they got a large market share.

I know, slush's has several flaws which is why I devised my method which I named the "geometric method".
Surely you don't get a lot of volume on slush with this profit?

What I want to happen is for pool miners to implement mathematically-fair payment algorithms. This is the solution best for the community in the end, but not manually enforcing no-hopping rules or withholding information. There is a complicated score-based algorithm published somewhere on this forum (not slush's algorithm) that seems secure, though I didn't check it in too much detail.

You're probably referring to the geometric method. But hey, it's not complicated at all. Most of the post is just analysis of its various features.

There is a much simpler fair algorithm though, which I want you to confirm: pay for all shares submitted in the last 1 hour of a round. If a block is found sooner than 1 hour into a round, then rollover the extra time and pay for the unpaid shares of the previous round, or the previous round before that (if the previous round was shorter than an hour as well), and so on. This way, each share has equal expected utility regardless of when it is submitted - early on or late in a round. Late shares in long rounds are subsidized by early shares in short rounds. You can use any time interval shorter than the mean round time. I'd recommend 1/4 of the mean time, e.g. 1 hour for a 130GHash/s pool.

Even if this works (which I'll try to look at at some point) it seems much more complicated.

Because in most cases variance is less important than expectation

It seems hard to make that argument in this context.  If people wanted to maximize expectation, they would just solo-mine and avoid pool fees, pool downtime, stale shares, etc.  In most cases the variance is extremely "large" but in absolute terms the amount of money involved for a hobby miner is small enough that it doesn't really matter.  

I will grant that many hobby-scale miners don't understand this and don't act that way in practice.

The huge variance of solo mining is under the scope of the "other cases". But once you cut down, say, 99% of the variance with a score pool, there's not much point in decreasing it further.

and because the variance of score-based is not that high

True.  But still you are ignoring mental accounting costs.  People like equal shares because they are simple and easy to understand.  If you are hoping for pool participants to understand game theoretic indifference, you're going to be disappointed.  The constant push back (even if mathematically incorrect) over score-based pools being "unfair" should be ample illustration of this.

Right, but if they want to go for "simple and understood", why not PPS? It has the least variance, it's simplest, and it has higher expectation than worst-case presence of hoppers.

[smooth]

Right, but if they want to go for "simple and understood", why not PPS? It has the least variance, it's simplest, and it has higher expectation than worst-case presence of hoppers.

I don't know how popular PPS is.  It seems like it must have a pretty significant appeal if anyone uses it given the fees that are charged for it, and the fact that people still post saying, "I'm using PPS on such-and-such pool, how can I make more monies?"

If it weren't for the block withholding attacks, PPS might be more popular (and less expensive).

Also, given how people start freaking out every time the PPS rate "drops" because the difficulty goes up, PPS is not necessarily as "simple" as proportional.  "We work together to solve blocks and divide them up according to effort" is about as easy to understand (or so people think!) as it gets.

[smooth]

There is a complicated score-based algorithm

it's not complicated at all.

You have to realize that any argument that follows that script is lost, right?

Especially when you are responding to someone who actually demonstrated way above average understanding of pooled mining payout methods.  Think about how it looks from the perspective of miners who don't understand these things half as well as "anon4758," and you will understand why people like proportional.

[slush]

Slush's pool is not secure because he messed up his algorithm.

Can you be more specific? Looks like you found (or you think that you found) some mistake in score implementation, but currently I don't know about any.

[anon4758]

@Meni Rosenfeld

Thank you, yours was the algorithm I was referring to! That is the single most insightful and precise mathematical analysis on these forums, and I did not mean to bash it. I guess I am merely dissatisfied with score-based methods in general, because they make it less clear intuitively how much each share is worth. There is also the problem with large exponential numbers that need to be rescaled for long rounds (or stored as logarithms). Ultimately though, any fair method is better than naive method.

@slush
The decay constant of 300 seconds means that the shares of the past 5 minutes are worth 2.7 times as much as all the previous shares combined, so you are not burdened with others' old shares when mining in a long round. But there is a high variance (you aren't getting anything unless you happen to mine in the last 5 minutes of a round) and you still have the deadweight of the last 5 minutes' worth of shares, such that after about 1/2 mean round time, you are mining at only 90% efficiency (efficiency doesn't keep going down below 90% though, unlike with standard shares). Up to 15% mean time, efficiency is greater than 100%, and it is greater than 200% in the first couple minutes, so I get 20% extra on average (across entire round time) by hopping. Using shorter decay constant would decrease the gain, but would increase the variance even more.

[Meni Rosenfeld]

If it weren't for the block withholding attacks, PPS might be more popular (and less expensive).

Perhaps I should mention that I had suggested a protocol change that would make this attack impossible?

You have to realize that any argument that follows that script is lost, right?

Especially when you are responding to someone who actually demonstrated way above average understanding of pooled mining payout methods.  Think about how it looks from the perspective of miners who don't understand these things half as well as "anon4758," and you will understand why people like proportional.

Point taken, but I'm not sure participants necessarily have to know the details of the scoring method, any more than they need to know what hashing is or how the block chain works. I think also with some charts and graphs we can make it easier to understand.


@slush - The geometric method has two advantages:
1. Score fee - Using this, the pool is at a steady state where the past is always the same. In your pool you do not use a score fee, so very early in the round shares submitted compete with less existing shares, increasing payout. After a while it converges to a steady state. Shortening the time constant speeds up convergence and thus improves hopping-resilience, but adds variance to miners. By the way, the time constant should be proportional to the average time of finding blocks to maintain a specific tradeoff.
2. Share-base decay - Your pool uses time-based decay, which creates several quirks and attack vectors. For example, if the hashrate of the pool fluctuates and is currently more than average, the expected time to round completion (and hence, the expected depreciation of shares submitted now) decreases and it's more profitable to join.
PS. You promised a while back that you would look at my suggested method.

@anon4758 - Can you share with us, do you also post on this forum regularly under a different identity? (Obviously I don't expect you to say what that identity is.)