Deepbit hopping

[P4man]

Recently there have been various 105-115+% PPS schemes propping up.
The most sensible explanation I could see was pool hopping large proportional pools like deepbit.

Deepbit delays its stats by an hour, but I didnt think that would be enough to prevent pool hopping, since deepbit is so large, you might assume every new block that is found, is found by deepbit, and you'd be right 1 times out of 3. Good enough to extract a profit from the non hopping proportional miners.

So, I decided to have a look at deepbit stats, to see if I could find evidence of this theory. I collected the stats for last 5 days and plotted the estimated hashrate against block duration. And sure enough, there is a nice correlation:

edit: this graph is incorrect, see below.


X axis shows minutes it took to find the block, Y axis is estimated hashrate (shares / minutes).
Average is 24.7 minutes.

I discarted sub 2 minute blocks, but even so data for the shortest blocks are not to be trusted, since deepbit only gives a granularity of 1 minute. Moreover, I do not know how the lenght is rounded, so if 1m59s becomes 1 or 2 minutes. This could also skew the graph, but I dont think enough explain the correlation.


edit:
Deepbit rounds 1m59 seconds to 1 minute in the stats, so on average I would have to add 30 seconds to each block. Then we get this graph



X axis shows the minutes it took deepbit to mine the block (average is ~25).
Y axis show estimated speed (shares / time).

Correlation is much less strong, but its still there. Looks like almost 10% hoppers.

Tycho, can you shed some light on this? Or if you can, provide data thats more accurate than 1 minute?

[1714984147]

1714984147

[1714984147]

1714984147

[DeepBit]

I don't think that those 115% schemes gain their profit from hopping deepbit because it would be really obvious for me.
I noticed a couple of times when some user bought a "contract" from GPUMAX and tried to use it for hopping deepbit, but I prevented it.

As for your graph, yes, there is some correlation on it, but I think it's a bit off because randomness should also cause the speed to be lower sometimes for the lucky blocks. Also you should show full range from 0 speed for axis Y :)
EDIT: new graph is better.
Usually they are hopping with some delays, so shortest blocks are not affected after long enough ones.
I know that there are some hoppers and I know that stats delaying is not safe enough to prevent it, but their effect on innocent miners is not so noticeable because they just don't have enough hashing power for this.
I'm working on automatic countermeasures and I have ready detection routines that are undergoing testing at this moment.
Some users already noticed that: https://bitcointalk.org/index.php?topic=3889.msg718043#msg718043
I'm collecting results to ensure that no false positives will ever happen, then automatic punishing can be enabled to protect my users.

Additionally, hop-proof payment method may be implemented later.

[P4man]

I don't think that those 115% schemes gain their profit from hopping deepbit because it would be really obvious for me.
I noticed a couple of times when some user bought a "contract" from GPUMAX and tried to use it for hopping deepbit, but I prevented it.

If no one is hopping deepbit, the above graph would look different.

As for your graph, yes, there is some correlation, but you should show full range from 0 speed for axis Y

Why? Zero would mean a block thats never been found.
Someone better in statistics than me can do the proper math, but it looks to me like ~10% of your hashrate is successful hopping.

Also can you please tell me how the duration in the stats is rounded?

[DeepBit]

I don't think that those 115% schemes gain their profit from hopping deepbit because it would be really obvious for me.
I noticed a couple of times when some user bought a "contract" from GPUMAX and tried to use it for hopping deepbit, but I prevented it.

If no one is hopping deepbit, the above graph would look different.

As I said, there are hoppers and I'm working on this.

As for your graph, yes, there is some correlation, but you should show full range from 0 speed for axis Y

Why? Zero would mean a block thats never been found.

I was talking about the axis range.

Also can you please tell me how the duration in the stats is rounded?

Why don't you just use the first column with timestamps ? It's 1-second accuracy there.

[P4man]

Why don't you just use the first column with timestamps ? It's 1-second accuracy there.

Because Im an idiot
Regardless, from that column its clear you indeed round down, so 59s becomes zero minutes. That means the second chart is correct. Still enough of a correlation to prove your pool is being hopped in a significant manner.

[DeepBit]

Regardless, from that column its clear you indeed round down, so 59s becomes zero minutes. That means the second chart is correct.

Can you post a graph with those data points averaged for each minute ? I mean, with a line instead of dots.
For example, with a Y-range from 0 to 70 000.

[P4man]

Not sure what you mean, but I justed added a 40 point average (and cropped the extremes):

removed incorrect graph.

edit: I see what you mean now. Im no ace at charting in OO, Ill try, but the the above gives a good idea already
Increasing the Y range just adds blank space though

[DeepBit]

Not sure what you mean, but I justed add a 40 point average (and cropped the extremes)

Your Y-axis scale is from 40 000 to 85 000. I asked for 0 to 70 000, to take a look at global scale.

EDIT: BTW, why so many red dots at the zero-time point ?

[P4man]

Like I said, it just adds blank space. Zero means zero hashrate.  It would make more sense to chart the deviations from the average hashrate, then the x axis is at 51K, but the chart will look identical.

[P4man]

EDIT: BTW, why so many red dots at the zero-time point ?

Because Im not adding them together. Each dot is an average for the next/previous 40 datapoints, and there are many of those with sub 1 and sub 2 minutes. The difference in #shares gives different datapoints, even for the same duration (x axis).It looks weird, but its correct. It would look better if I grouped the data per x minute interval, and showed only 1 point with an average hashrate for that interval, but Ill have to do that later.

[Remember remember the 5th of November]

Considering blockchain.info tells me WHICH pool found a block, it is fairly easily(with a 2 minute gap) to find out if deepbit mined a block. Sure it requires additional info, but it could be worth it!

[P4man]

Couldnt find a way to do it with a formula so I manually grouped shorter durations and averaged them. I didnt do that for all durations, so you get some weirdness, but it looks better now:

edit: made a mistake with decimal 30s.. This is one is correct:



Interesting dip on the left side. I originally had a mistake in the spreadsheet adding only 0.4 decimal minutes average to the block length, rather than 0.5 to compensate for the incorrect rounding. Makes a big difference for blocks which take <1 minute obviously. The dip actually makes complete sense, it takes a while for hoppers to notice the new block on the chain and switch to your pool. I think the difference between 0-1 minutes and 0-2 is clearly how many (quick) hoppers you have.

There is no question you are being hopped. Its not massive, but its there.

[DeathAndTaxes]

Linear regression would smooth the trend line out.

If you post the raw data (x,y) in a code block for the scatter plot I will draw one.

My guess about the dip on the left is due to the delayed stats hoppers are using some mechanism to try and confirm blocks before hopping in.   For example if you monitored the top 5 non Deepbit sites and they didn't report a block even delayed or obfuscated the chance it was deepbit increases.  On an unknown block there is a roughly 35% chance it is Deepbit's.  If the top 10 pools don't report it (and you wait 30 seconds or so for blockchain propogation) then the chance it is deepbt's increases to about 70%.  Roughly 50% of the time someone will report a block which guarantees that it isn't Deepbit. That means you are hopping with ~ 85% confidence.

[PatrickHarnett]

Missing the definitions of the different axes, I assume Y = average hash speed and X = block duration.

The point about graphing the Y axis from zero would better show how much (or little) decline there is.  For instance, is deepbit losing 5% or more from hoppers, or less.  It is hard to tell given the way it is presented.

Interesting piece of analysis.

[P4man]

Missing the definitions of the different axes, I assume Y = average hash speed and X = block duration.

Yeah, its in the text.

The point about graphing the Y axis from zero would better show how much (or little) decline there is.  For instance, is deepbit losing 5% or more from hoppers, or less.  It is hard to tell given the way it is presented.

Speed never drops to zero. As i said over and over, its just blank space at the bottom. Or top, why not add a lot at the top?
The chart is fairly clear I think, from the sampled data there seems to be about 10% hopping.

Anyway for those who want to play with the data and make their own charts (incl from zero), here is the spreadsheet
http://www.mediafire.com/?ebamgh0fmkrrl45

Its in ODS format (open office). Charts are on the second sheet.

[PatrickHarnett]

ok, if you take the max at 55,000 and a decline to 50,000, then you get 10%.  I wonder if the noise in the chart over-states it.  Might only be from 53k to 51k.  Either way, it's useful to have a snapshot of data.

[P4man]

ok, if you take the max at 55,000 and a decline to 50,000, then you get 10%.  I wonder if the noise in the chart over-states it.  Might only be from 53k to 51k.  Either way, it's useful to have a snapshot of data.

The data covers just over 100 blocks that where shorter than 10 minutes. Im sure someone will soon calculate 95% probabilities, but I dont think noise is likely to explain it. It could be over- or understating it though, thats for sure.

My main point is they are being hopped, Im fairly confident the data proves that. Even if its just 5%, I wonder if that couldnt match the hashrate of those 115% PPS fake pools. Add to that they all seem to suffer from high stales, and Im getting quite certain thats whats going on. They are probably not just hopping deepbit, but deepbit are obviously the biggest one, and perhaps  the easiest to show it for.

[PatrickHarnett]

ok, if you take the max at 55,000 and a decline to 50,000, then you get 10%.  I wonder if the noise in the chart over-states it.  Might only be from 53k to 51k.  Either way, it's useful to have a snapshot of data.

The data covers just over 100 blocks that where shorter than 10 minutes. Im sure someone will soon calculate 95% probabilities, but I dont think noise is likely to explain it. It could be over- or understating it though, thats for sure.

My main point is they are being hopped, Im fairly confident the data proves that. Even if its just 5%, I wonder if that couldnt match the hashrate of those 115% PPS fake pools. Add to that they all seem to suffer from high stales, and Im getting quite certain thats whats going on. They are probably not just hopping deepbit, but deepbit are obviously the biggest one, and perhaps  the easiest to show it for.

Now that's an interesting observation/comment.  Months go when I was arguing with Meni about hopping I was putting forward proposition that smaller pools might be more vulnerable, and larger pools would be more stable.  Therefore I would expect the smaller pools to show more pronounced changes and trends.

[DeepBit]

I wonder if that couldnt match the hashrate of those 115% PPS fake pools. Add to that they all seem to suffer from high stales, and Im getting quite certain thats whats going on. They are probably not just hopping deepbit, but deepbit are obviously the biggest one, and perhaps  the easiest to show it for.

I don't think that it's related to 115% pools because I'm monitoring all the hoppers (in the process of developing automatic detection) and there were no any changes in their volume when those projects appeared. Also, I did some tests and they didn't reported any problems during this time.

[P4man]

I don't think that it's related to 115% pools because I'm monitoring all the hoppers (in the process of developing automatic detection) and there were no any changes in their volume when those projects appeared.

Well, maybe I misunderstood you, but if some miners moved from deepbit PPS to these 115% proxy pools that are hopping deepbit and others, you wouldnt expect to see a big difference in hashrate. Youd might lose some hashrate on the longer blocks, potentially win some at the beginning of blocks or on short blocks from miners that didnt mine at deepbit before. Not very conclusive IMO.

If you are saying these proxy pools registered at deepbit and you are monitoring their behavior.. then that begs the question why on earth they would pay 115% to redirect the hashrate to a pool that pays out 97%, if its not for hopping?

[DeepBit]

Well, maybe I misunderstood you, but if some miners moved from deepbit PPS to these 115% proxy pools that are hopping deepbit and others, you wouldnt expect to see a big difference in hashrate. Youd might lose some hashrate on the longer blocks, potentially win some at the beginning of blocks or on short blocks from miners that didnt mine at deepbit before. Not very conclusive IMO.

I was talking about monitoring all the hoppers, not the total/average hashrate.

If you are saying these proxy pools registered at deepbit and you are monitoring their behavior.. then that begs the question why on earth they would pay 115% to redirect the hashrate to a pool that pays out 97%, if its not for hopping?

NO, I didn't said that they are registered at deepbit.

[P4man]

Now that's an interesting observation/comment.  Months go when I was arguing with Meni about hopping I was putting forward proposition that smaller pools might be more vulnerable, and larger pools would be more stable.  Therefore I would expect the smaller pools to show more pronounced changes and trends.

I guess what really proves hopping is the % difference in hashrate, regardless of it being a  small or big pool.  But with a small pool, it might be difficult to get enough data points, without spanning such a long time that all sorts of things might factor in.

But hey, I dont mind trying. What other proportional pools do you know that we could look at?

[P4man]

]I was talking about monitoring all the hoppers, not the total/average hashrate.

Okay, but even so, the hoppers might just have "pooled together" now.

[Inaba]

Bitclockers would be a great one.  Their stats are fun to watch.  A new block starts and they will jump to 700 - 900 GH/s, then like clockwork at ~40% their hashrate plummets to 100 - 150 GH/s.  Those poor miners making up the 150 GH/s are getting raped like a two bit whore.

[DeepBit]

The point about graphing the Y axis from zero would better show how much (or little) decline there is.  For instance, is deepbit losing 5% or more from hoppers, or less.  It is hard to tell given the way it is presented.

Speed never drops to zero. As i said over and over, its just blank space at the bottom. Or top, why not add a lot at the top?
The chart is fairly clear I think, from the sampled data there seems to be about 10% hopping.

Do you really not understanding why we are asking this, or you just want the graph to be more impressive ? :)

I was talking about monitoring all the hoppers, not the total/average hashrate.

Okay, but even so, the hoppers might just have "pooled together" now.

Well, you can believe in this if you want to.
I just told what my observations show me, this opinion is based on really good stats collection.

[P4man]

Bitclockers would be a great one.  Their stats are fun to watch.  A new block starts and they will jump to 700 - 900 GH/s, then like clockwork at ~40% their hashrate plummets to 100 - 150 GH/s.  Those poor miners making up the 150 GH/s are getting raped like a two bit whore.

Eeeck. I think this chart on their own website already shows what you mean:



Im having a vague idea when they found blocks looking at that. Ill see if I can plot it against their blocks tomorrow. I got a feeling its indeed not going to be pretty  

[P4man]

Do you really not understanding why we are asking this

No. But if you like lots of blank space, here you go":


I even resisted the urge to add to -100.000
I guess what you really wanted me to show was this:



While what you really should see is this:



I can stretch and zoom it any other way you want, it keeps showing a little under 10% hopping.

[DeepBit]

Oh, so you know what's the difference

[martychubbs]

Pool Ops that offer PPS to their members and hop with their hash power should be exposed. 

[pirateat40]

So yes, GPUMAX is responsible for sending purchases to deepbit on occasion but it's not a popular pool for those that purchase shares.  I think one of the things that you might want to look at is when we're doing updates or have suspended purchases our miners fail-over to their offline pools.  So this could cause a spike in hashes that could make it look like they're hopping the pool when in reality it's just them failing over.  Technically they are hopping but not the traditional way.  I have no problem adding Deepbit to the not supported list for purchases if it's causing that much concern but I can't control where people fail-over to when we're not processing purchases.

[DeepBit]

So yes, GPUMAX is responsible for sending purchases to deepbit on occasion but it's not a popular pool for those that purchase shares.  I think one of the things that you might want to look at is when we're doing updates or have suspended purchases our miners fail-over to their offline pools.  So this could cause a spike in hashes that could make it look like they're hopping the pool when in reality it's just them failing over. I have no problem adding Deepbit to the not supported list for purchases if it's causing that much concern but I can't control where people fail-over to when we're not processing purchases.

Yes, I noticed some users are getting enormous bursts of up to 160+ GH/s sometimes after your service started and this is completely fine for me unless they are hopping, so you can keep Deepbit in your "supported" list. Hoppers would direct their purchase to some proxy anyway.
The hopping incidents were completely different because its very clear to me if their starts and stops are correlated with many of our blocks.

[P4man]

I just analyzed Bitclockers stats and posted the results here:
https://bitcointalk.org/index.php?topic=61985.msg737310#msg737310

No need to for any regression analysis there me thinks lol
Short story: poor (/idiot) constant miners there are paying 10-12% to the hoppers, +2% fees to the pool.

I also tried calculating that for deepbit using second intervals, but either I screwed up somewhere or its statistical noise with pool luck and stales being a bigger factor, as I ended up getting about 0.2% better than expected earnings for constant miners.

[P4man]

Yeah, I highly doubted deepbit was getting heavily hopped.

Non sequitur, depending how you define "heavily".My point still stands.

AFAICT, they are being hopped by something like 5-10% of the hashing power. That I cant prove an effect on the payouts of non stop miners is entirely to be expected since Id need a much bigger sample for that than just the last few days available on their site; there are many reasons you can get slightly above expected payouts (excluding their high fees of course) on such a short span, the most important one being pool luck.  Other factors are stales, since I have no data for them. Since they occur more frequently at the beginning of new blocks, it could chance the result significantly.

[eleuthria]

Pool Ops that offer PPS to their members and hop with their hash power should be exposed. 

Exposed for being savvy businessmen. The miners are getting exactly what they deserve. What's the problem?

1)  They're actively hurting users of other pools.  Not every pool can handle swings of hundreds of GH/s, especially the smaller pools that still use hoppable methods.  It increases stale rates and the frequency of miners idling.  The pool ops can't afford to upgrade to stronger servers just to service a proxy-pool that only shows up for a small time frame.

2)  They're not being transparent, and it can be far more dangerous than the FUD that gets spread about Deepbit.  They should at least have on their site available for all to see what pools they hop, and what pool(s) they are currently hashing on.

[Kohta]

I never understood the purpose to that, my payout is more or less consistent within 1-2%, very rarely (maybe once every other day) i will only get 80/160,000 shares and it'll be higher or lower than normal, but as far as i can see Pool hopping hasn't done anything to my payout if it's happening at all. (Deepbit)

[P4man]

I never understood the purpose to that, my payout is more or less consistent within 1-2%, very rarely (maybe once every other day) i will only get 80/160,000 shares and it'll be higher or lower than normal, but as far as i can see Pool hopping hasn't done anything to my payout if it's happening at all. (Deepbit)

If the hopping were to happen constantly, you wouldnt see it from your payout, it would always be "normal" but x% lower than 100% PPS for the current difficulty. Im not saying I think the impact from hopping is significant though, at least not compared to the fees you are paying anyway .

[P4man]

no no, no one but p4man thinks deepbit is getting hopped.

God you're such an idiot.
"No one but me" and Deepbit themselves

Welcome to my ignore list again. I wonder why I ever let you out.

[Kohta]

Would somebody who is on PPS be effected more than Proportional? Because i'm on Proportional, the avg. payout was higher within a few days time at my current Mhashs.

I think there is a lot of sarcasm around here that i'm taking for fact and getting confused lol.

[P4man]

Would somebody who is on PPS be effected more than Proportional? Because i'm on Proportional, the avg. payout was higher within a few days time at my current Mhashs.

PPS would not be affected.
Of course, that has to be put in perspective, considering deepbit charges, what 10% fee for PPS? That makes hopping a total non issue by comparison, but you still get paid 10% less than on any (hop proof) zero fee pool though.

[PatrickHarnett]

For curiosity I did a graph of the last 321 blocks.  Time per block to the nearest second versus shares per minute.  I too can get a slight decline in hashes returned for the longer blocks.  Average shares/minute = 51,300, Standard Deviation = 2,127.

Better yet, I added a trend-line and calculated the R^2 values.  The result, with R^2 less than 10% tells me lots of noise and any relationship, if there is one, is weak.  The longest block in the sample had a hash rate higher than average - a random result

[deepceleron]

Pool hopping is detectable by the pool op. If a user's per-share reward value over time is consistently higher than average, they are gaming the payout system. In aggregate data like the total per-share reward vs round time, observation by outsiders will be masked by the total pool size and the modest counter-measures, but on a per-user basis, there will be users who earn the average expected per-share reward, and hoppers that stick out like a sore thumb with higher earnings.

Hoppers will also submit a consistent number of shares per round even when they are long (or shares per block if just hopping on new network block information), and by quitting long rounds, are identifiable.

The time scale of the graph in the first post should be logarithmic, so there is an even spread of round quantiles over time.

[P4man]

Pool hopping is detectable by the pool op. If a user's per-share reward value is consistently higher than average, they are gaming the payout system. In aggregate data like the total per-share reward vs round time, observation by outsiders will be masked by the total pool size and the modest counter-measures, but on a per-user basis, there will be users who earn the average expected per-share reward, and hoppers that stick out like a sore thumb.

Agreed. But since I made the post, Deepbit has linked to their own thread were is shown that hoppers are not only detected, but also labeled as such. Just no action is being taken against them - for now.

I hadnt seen that when I made my analysis, I just tried proving deepbit can be and is being hopped. That much is beyond doubt now.

I do agree because of deepbits size, the current impact on non-stop miners by hoppers is trivial and statistically noise, that was never my concern.

[PatrickHarnett]

For interest I added another five days data to try to increase the size of the dataset (a few over 600 blocks) because Deepbit has had a few 2-3 hour solves recently (including a 184 minute monster).

Distribution is still very noisy with an average solve time of 26 minutes (Sd of 27 minutes) and average hash rate 51,440 shares/minute (s.d. 3246).  Testing different regression fits gave less than 5% R^2.

[Jack of Diamonds]

Pool Ops that offer PPS to their members and hop with their hash power should be exposed.  

Hopping proportional pools and exploiting their resources is a natural step in the evolution process of bitcoin pools.

Large proportional pools have been on the decline for ages & will eventually die out completely once the core miners realize they are working for someone else;
They are paying out of their own pocket so that somebody else may profit.

I think Inaba on page 2, the master of Eclipse pool, has put it eloquently in the last sentence:

A new block starts and they will jump to 700 - 900 GH/s, then like clockwork at ~40% their hashrate plummets to 100 - 150 GH/s.  
Those poor miners making up the 150 GH/s are getting raped like a two bit whore.

Of course, those 101+% pools will disappear too, but only once the proportional model disappears first. Right now, non-hopping, perpetual proportional miners are the suckers of the system, the prey. Through either lack of knowledge or sheer ignorance, they distort the balance of pools so as to create an incentive for predators.
True equality in mining rewards will exist, but only when these miners finally understand they are damaging their own earnings.

(This model relies on the rational assumption miners' main motive is profit;
It does not take in account irrational motives such as supporting a pool for ideological reasons, or wanting to give free money to pool hoppers out of generosity etc., so it is still possible there could be proportional miners 3-5 years down the road)