Vladimir's essential self-defence guide for Bitcoin Miners

[Sukrim]

Just for the record, I decided to join a larger pool and proportional payout rather than pay per share or a small pool/solo mining.  I know too much statistics to rely on luck 

If you know statistics well, what is your reason behind not using PPS? It has the lowest possible variance, is not dependent on pool size and can be easily calculated + verified.

Difference in fee structure made the expected payoff for proportional higher than pay per share, plus I can withstand the increased variance over time.

Yeah, on deepbit...

There are pure PPS pools with 0% fee out there too, just sayin'... since PPS is NOT dependent on pool size, you have only little to loose.

[organofcorti]

I only have to reference the polmine miners who are fleeing the pool now that they are stuck on a 9 million block.

Interesting, the system throws random answers (returned shares) at a problem and people attribute long solves to "luck" rather than just the normal probability distribution of solving a block.

If I jumped to a new pool that found the block as soon as I joined, I might get 0 shares out of 100, because the pool was "lucky" to find it right then, but my 100000 shares in the previous pool will contribute to the next block found there anyway.

Willing to be proved wrong, someone might want to supply a link that shows, over a period of a month or longer, for 1000Mhash mining (ie, non trivial) the average return from sitting versus hopping?

There's a bunch of chartporn on the bitHopper thread, or you can try this simulator.
https://github.com/organofcorti/byteHopper-s


The actual Mhps is not important, just the efficiency.

[PatrickHarnett]

I only have to reference the polmine miners who are fleeing the pool now that they are stuck on a 9 million block.

Willing to be proved wrong, someone might want to supply a link that shows, over a period of a month or longer, for 1000Mhash mining (ie, non trivial) the average return from sitting versus hopping?

The actual Mhps is not important, just the efficiency.

I suggested a reasonable hash rate because operating at 1 Mhash there are appreciable probabilities of not completing shares or making a reasonable contribution.  At least when calculating at a reasonable speed and reasonable time span, so of those noise effects drop away.

I there were no pools (everyone was their own pool- solo), or one pool, I have a chance of finding the block for every share I crunch.  If I am in a pool where people come in and out, or I jump around, my chance of finding a block does not change, nor does it change for anyone else.  A bit like playing roulette (which I don't do because of the stacked odds) - one spin or table is much like another.  If hopping worked so well, people would be doing this in casinos.

Great thing random chance.  Full points to the poster mentioning "gambler's fallacy".

[organofcorti]

I only have to reference the polmine miners who are fleeing the pool now that they are stuck on a 9 million block.

Willing to be proved wrong, someone might want to supply a link that shows, over a period of a month or longer, for 1000Mhash mining (ie, non trivial) the average return from sitting versus hopping?

The actual Mhps is not important, just the efficiency.

I suggested a reasonable hash rate because operating at 1 Mhash there are appreciable probabilities of not completing shares or making a reasonable contribution.  At least when calculating at a reasonable speed and reasonable time span, so of those noise effects drop away.

I there were no pools (everyone was their own pool- solo), or one pool, I have a chance of finding the block for every share I crunch.  If I am in a pool where people come in and out, or I jump around, my chance of finding a block does not change, nor does it change for anyone else.  A bit like playing roulette (which I don't do because of the stacked odds) - one spin or table is much like another.  If hopping worked so well, people would be doing this in casinos.

Great thing random chance.  Full points to the poster mentioning "gambler's fallacy".

OK, try this link then:

https://bitcointalk.org/index.php?topic=32814.msg478871#msg478871

[hawks5999]

Wild swings of hashing power of a pool is an indicator of a pool being heavily hopped and a huge red flag for any ethical miner.

Oh, don't worry. We ethical miners have seen the huge red flag and are hopping on Mt Red. That huge swing in hashing power has led to 6 blocks in the last 24 hours. Again, compare to the anti-hopping polmine who hasn't solved a block now in 5 days (corresponding with their full implementation of anti-hopping measures). It's all coincidence, of course.

But if hashing power = work done then the Samuel Goldwyn quote seems apropos:

"The harder I work, the luckier I get"

[PatrickHarnett]

I looked at https://bitcointalk.org/index.php?topic=32814.msg478871#msg478871 and thinking it over while walking to work gave me some entertainment.

It becomes farcical at the point it talks about "Take the extreme example of a pool hitting a 10x difficulty block"  LOL  between difficulty resets, they all have the same difficulty.  Such screwed logic is not backed by the input mechanics or the observed results.

If you want to pick on my strategy, I mine at deepbit because it was easy to set up and the fee was reasonable.  Last 24 hours average time per block is 25 minutes (yesterday 21 minutes).  Roughly 5Ghash or 1/3rd so should be getting a couple of blocks per hour.  It tells me the variance in shares submitted per block is +9.6% at the moment and I get similar payments for a 2 minute or 100 minute block, but on average 20-ish minutes I get a proportional share of what I've contributed.  If I skipped out on a block after an hour and mined somewhere else, my shares still contribute to the next block found (whenever that is, and proportionally), and same with a pool I might jump to.  My averages don't change (assuming fair pools) because my work and the distribution lambda have not changed.

btw, if the pools are set up correctly, pool hopping should make little actual difference over the long term - the notion of ethics is odd unless somehow brand loyalty counts for something - do we get bitcoin-miles for staying with one pool, I haven't looked.

Back to the point of this thread - Vladimir posted up something to help tell if your pool is biased, and added in the variable covering hopping because some people think it makes a difference.  Maybe they're the same ones who like losing money in ponzi's or gambling.

[hawks5999]

Pool hopping, huge pools etc.  From another post read semi-recently, deepbit seems to be nearing the point that it could "take over" the bitcoin network for the immense hashing power it has (is this accurate?)

If this is a possibility are mining pools in actuality a threat to Bitcoin?  Secondly, if so is there anything that could be implemented in the bitcoin network such that no single source could get more than a certain % of total hashing power? i.e some sort of built in throttling mechanism... one would think that could make the whole network safer and while supporting the mining pools, force them to split once in a while for maximum efficiency?

Just a thought....

Yeah it's sad that a p2p, decentralized currency is at risk from an attack against one organization (deepbit). If hopping deepbit can help drive away some of the hashrate currently there, it's just that much more rewarding.

[deepceleron]

I looked at https://bitcointalk.org/index.php?topic=32814.msg478871#msg478871 and thinking it over while walking to work gave me some entertainment.

It becomes farcical at the point it talks about "Take the extreme example of a pool hitting a 10x difficulty block"  LOL  between difficulty resets, they all have the same difficulty.  Such screwed logic is not backed by the input mechanics or the observed results.

That refers to the length of a pool round by comparing the number of round shares to the difficulty. This is a common way of discussing the length of rounds, because when calculating a pool's average round length, it may be a different amount of time depending on pool hashrate, or a different number of shares depending on difficulty, however a round always has a 50% chance of being solved in the same number of user-submitted shares (which are difficulty 1) as the current Bitcoin difficulty. Thus, a 2x difficulty round would be twice as many shares as was expected to find a block solve.

If you want to pick on my strategy, I mine at deepbit because it was easy to set up and the fee was reasonable.

Your strategy is to use the pool with the highest fees? Kudos! That's worthy of being picked on!

[organofcorti]

It becomes farcical at the point it talks about "Take the extreme example of a pool hitting a 10x difficulty block"  LOL  between difficulty resets, they all have the same difficulty.  Such screwed logic is not backed by the input mechanics or the observed results.

 A 10 x difficulty block is one that requires 10 times the share contributions of a normal difficulty block in order to solve it. ATM that means a block of 18 057 000 shares. Can you explain to me where the screwed logic is?

You're suggesting that between difficulty resets every pool solves a block at difficulty shares every time? Surely you've been involved with bitcoin mining to believe that! A quick look at btc-poolwatch.com gives bitclockers total shares at 78% of difficulty, bitpit total shares at 19% difficulty, mineco.in total shares at 51% and triplemining total shares 460% of difficulty. All different totals. If you meant they solve blocks at the same number of shares - difficulty*1 - then that is also just plain wrong.

But I'm glad I was able to give you some enjoyment at work.


BTW - ever get around to running a simulator so you could work out your answers for yourself? Instead of just shooting everyone else's down, I mean.

[PatrickHarnett]

I'm not here to shoot down others without having some basis for my arguments - there are more than enough people who do that in the BTC forums.  As for simulation, I can look at actual results to see outcomes.  I could set something up quickly that would simply demonstrate that across all users and all pools, with a randomly sampled number of shares to solve blocks, that the average number of shares will equal the current difficulty.  If I participate proportionally, I will get a proportional payout.  I think the underlying point that hasn't been made well is that some pools might pay unfairly.  That harks back to the OP, which makes the point if your pool or strategy is consistently delivering poor results, then look for something better.  As I said, I'm churning shares through deepbit and it tells me at the moment it is about 3% longer per block than expected, which is nice because earlier it was at +10% (and average block time fallen from 25 minutes to 23 minutes).

The point I was making was that even for a block that has been running 18 million shares, there is a probability for the next one share to solve the block.  Blocks don't require a set number of shares to be returned before "solving", and it could continue to run for another 18 million shares to.  The distribution isn't perfectly Poisson, but close enough.

Every looked at the distribution of solve times in the block-chain, a nice sequence of values there.  Some long, some short, but as happily reported and adjusted occasionally, resets aiming to preserve 10 blocks per hour.

[organofcorti]

The point I was making was that even for a block that has been running 18 million shares, there is a probability for the next one share to solve the block.  Blocks don't require a set number of shares to be returned before "solving", and it could continue to run for another 18 million shares to.  The distribution isn't perfectly Poisson, but close enough.

I wasn't claiming you could know whether or not the round would be 18000000 shares - just that if you had happened to move to another pool at 1800000 and it had had 9 rounds, then you would have made more btc.

As for the actual reason poolhopping works without knowing the future, well if you know enough about statistic and understand how the number of shares submitted before a pool finds a block follows a *geometric* distribution, then you should be able to figure out the expected value of any given share for yourself. I couldn't, and had to run simulations to check.

[Jack of Diamonds]

How about "don't let one miner take over 80% of the hashpower of one pool" vlad?

Forget that one?  Ooops

Why would I care if Vlad supplies say, 100ghash to a pool I mine in?

If it's not PPS based, then all it means is every member gets paid faster and more frequently.
It also lowers the huge variance that occurs at 1.8 million difficulty.

Really.. This, and pool hopping etc.. Are not advanced math. You can figure these out with common sense.

Some people here seem to oppose everything, for the sake of being against something.

[Sukrim]

Well, he could run a withholding attack (in any pool/payout scheme), for whatever reason - so it's not too desirable to have HUGE miners in your pool as you have to trust them too.

[organofcorti]

it's not too desirable to have HUGE miners in your pool as you have to trust them too.

I'll thank you to keep your size-ist remarks to yourself, Sukrim. Vlad looks quite a healthy weight in his photo.

[PatrickHarnett]

Spending some of my weekend looking at this and went across to https://en.bitcoin.it/wiki/Comparison_of_mining_pools.  The range of different payment systems introduces the advantages and that was probably the argument that should have been put forward earlier.

If the payment methods were logical and consistent, then the hop vs non-hop largely disappears.  However, having systems where, for example,  "Each submitted share is worth more in the function of time t since start of current round." provides the opportunities to maximise returns.  That's more an issue with pool rules and the motivations of operators to attract people and earn fees.

Over time, miners would probably drift to an equilibrium state where consistent payout pools and hop-friendly pools separate, and then converge as the hopped pools then confer reduced advantage because they rely on non-hoppers to extract their advantage.  Simple - if you want to give away work, belong to a pool with exploits.  Again, that was the point Vladimir was trying to make.

[k9quaint]

Vladimir: After looking at your spreadsheet, I found a flaw in your analysis. You need to calculate the Poisson for all blocks found by all pools (and expected), not just the ones found by btcguild. You then need to determine what is the likelyhood that out of all the difficulty periods in all the pools, one will exhibit a run of "bad luck" such as the ones you found in BTCguild. You are failing to account for the selection bias effect.

For instance:

I separate people into many groups and have them all flip coins. I then tally each groups expected number of heads and tails. I see that one group has more heads than tails, and I calculate the Poisson distribution of that group as if those were the only coin flips thrown. Of course, this will show that groups efforts as unlikely to have occurred because I have masked out the efforts of the other groups.

Edit:

Also, you are comparing a real world empirical measurement to a perfect world theoretical average. Since there is no magic fairy dust that would result in a computer solving blocks at an average rate consistently faster than this theoretical average, we can ignore positive biases. If there was an operational friction that was not accounted for, it would exert a negative bias to all results. The way to account is to examine the poisson of all blocks found vs expected, btcguild blocks found vs expected, and all blocks without btcguild vs expected. That may produce a baseline would could illuminate any unseen frictions. It might be interesting to do that exercise for each pool, if deepbit had less friction its size may mask higher rates of friction in other pools.

[k9quaint]

Vladimir: After looking at your spreadsheet, I found a flaw in your analysis. You need to calculate the Poisson for all blocks found by all pools (and expected), not just the ones found by btcguild. You then need to determine what is the likelyhood that out of all the difficulty periods in all the pools, one will exhibit a run of "bad luck" such as the ones you found in BTCguild. You are failing to account for the selection bias effect.

For instance:

I separate people into many groups and have them all flip coins. I then tally each groups expected number of heads and tails. I see that one group has more heads than tails, and I calculate the Poisson distribution of that group as if those were the only coin flips thrown. Of course, this will show that groups efforts as unlikely to have occurred because I have masked out the efforts of the other groups.

I disagree. This would have some sense if I was claiming that all bitcoin pools have "bad luck" and brought up calculations with BTC guild data as proof. This is not the case though. Who cares, how many groups you separate people in. To illustrate this, image one particular "person/pool" with a fair coin which on the first and only try has tossed coin 4865 times and got at most 2275 heads. I have calculated odds of that happening is 0.0675%. Also if there were in existence 1400 more 2 Thps pools in bitcoin network, than yes we would statistically expect one of them being so unlucky, though for that the bitcoin network would have to be not 8 Ghps strong but almost 3 Pthps strong. This would be almost 3 orders of magnitude difference.

What if they flipped a coin made by a "friend" who claims that he made the coin right. It is the first time he has ever made a coin, but he did his best to make sure it was flat and looked good. We should first test the performance of this coin against the universe of coins made by friends, not against the theoretical 50/50. If we need to choose a coin to flip, we should compare then to each other and then choose one.

Also, the point is not that all pools are having bad luck. The point is that the universe of pools was examined and a single unlucky one was chosen. A better analog than coins might be dice. Imagine throwing six dice. One die comes up with 1 and we calculate the odds of that die landing on 1 correctly as 1 in 6. But that is not what we saw, we saw 6 dice land and the odds of any one of those dice showing a 1 is 1 in 1. Now let us suppose that these dice were not bought from the store, but were each made by a different individual. Now we are no longer sure what the correct distribution of die faces is for any of the dice.

Finally, the pools use different infrastructure. There is work to be done by the server when a share is submitted. What if pool A waits until all that work is complete before acknowledging to the client that it has received a submitted share. If clients cannot begin work on a second share until the first is acknowledged, this serializes the share submission with server share collation at the cost of latency. If pool B allows clients to submit their share and then immediately gives them a new one to work on before the collation work is done on the server, the share submission and processing are overlapped. Pool A's method has the effect of a natural governor on the speed that clients can use the server. Pool B's method minimizes latency but relies on the server to never get bogged down. Under maximum sustained load, these two systems will measure the same. But under failure conditions, they will measure differently.

That is it. I do not allege someone is cheating here. It is perfectly possible that it just have happened. Just like if we had someone to toss a coin 4865 times and repeat such a feat 1481 times than we would expect it (at most 2275 heads) to happen only once.

Let me set this str8, I do not allege that BTC Guild pool is cheating. I would have no factual basis for such conclusion at all. My conclusion is that during observed period of time (Summer 2011) for this particular set of data which is publicly provided by BTC Guild it seems (if the math is correct) that the pool is either  extremely (>99.9%) "unlucky" or cheating or having some technical issues or being attacked somehow or otherwise somehow inefficient or some combination of those.


My other conclusion is that I personally (or other people with a modicum of common sense) shall not touch such unlucky or inefficient, for whatever reason, pool and look elsewhere. However, those who unable to understand my reasoning or do their own DD are more than welcome to continue mining with proportional and extemely unlucky pools.

BTCguild did experience a DoS situation in June, I do not have the exact dates. But the scuttlebutt was that it was from a botnet, that it occurred over the course of a couple of days and peace finally "broke out". I am not sure just what the effect this might have on the "efficiency" of a pool, but I suppose it is possible it could skew the data.

Edit:

Also, you are comparing a real world empirical measurement to a perfect world theoretical average. Since there is no magic fairy dust that would result in a computer solving blocks at an average rate consistently faster than this theoretical average, we can ignore positive biases. If there was an operational friction that was not accounted for, it would exert a negative bias to all results. The way to account is to examine the poisson of all blocks found vs expected, btcguild blocks found vs expected, and all blocks without btcguild vs expected. That may produce a baseline would could illuminate any unseen frictions. It might be interesting to do that exercise for each pool, if deepbit had less friction its size may mask higher rates of friction in other pools.

True. However, somehow many other pools do not exhibit such inefficiencies, solo mining too appears to be fairly efficient.

Actually, they do. http://www.l0ss.net/index30.php
As you can see, there is very little if any "good" luck to be found in any of these 6 pools over the last 30 days.
Note btccoins.lc, btcmine, and mtred are all well below the expected luck line. What if there is friction that derives from a common component in the make up of these 3 pools? The luck for these 3 pools is demonstrably worse than btcguild, perhaps you should calculate what the "odds" of their bad luck occurring as well. Then we can calculate the conditional probabilities of those 3 pools + btcguild all having this sort of bad luck all at the same time. 

Also, one side question. How did you account for the invalid blocks?

[PatrickHarnett]

"quote" we saw 6 dice land and the odds of any one of those dice showing a 1 is 1 in 1. "quote"


WHAT?  After the event, looking at the dice, that's allowable, but rolling six dice and getting at least one "1" is [1 - (1 - 1/6)^6] ~= 0.66

[k9quaint]

"quote" we saw 6 dice land and the odds of any one of those dice showing a 1 is 1 in 1. "quote"


WHAT?  After the event, looking at the dice, that's allowable, but rolling six dice and getting at least one "1" is [1 - (1 - 1/6)^6] ~= 0.66

Yep, you are right. one roll of 6 dice is will show a 1 66.51% of the time. /sadface
My point was the probabilities were different. I assure you, my point was not to belabor the fact that I am a moron. 

[PatrickHarnett]

"quote" we saw 6 dice land and the odds of any one of those dice showing a 1 is 1 in 1. "quote"


WHAT?  After the event, looking at the dice, that's allowable, but rolling six dice and getting at least one "1" is [1 - (1 - 1/6)^6] ~= 0.66

Yep, you are right. one roll of 6 dice is will show a 1 66.51% of the time. /sadface
My point was the probabilities were different. I assure you, my point was not to belabor the fact that I am a moron. 

lol - it gave me a laugh earlier today.  I don't think you are a moron, but the expression just jumped off the page at me.