Bitcoin Forum

Hi,

As promised in my earlier thread, I studied some of the strategies that a miner could use to optimize his use of a cooperative mining pool. I have watched slush's pool over a few days and didn't notice any evidence of a massive use of these strategies, so I think they're relatively unknown by the common bitcoin miner. I also provide pointers to mining pool owners who see this as a problem and want to "fix" their pool.

The file is here: http://www.bitcoinservice.co.uk/files/100

Here is the table of contents:
1 - Introduction
2 - A brief overview of pools
  - the creation of bitcoins
  - sharing the work
  - all shares are not equal
3 - Strategies
  - One pool
     * average time to find a block
     * should I join now or wait for the next block ?
     * how long should I work on one block ?
  - Several pools
    * When should I switch pools ?
4 - Consequences for the mining pools
   - Is this the end of pools ?
   - Are some miners currently using these strategies ?
   - Fixing the mining pools
5 - Conclusion

Edit: I believe, for the sake of honesty, that I should note that the pools at no serious risk of being completely broken by those strategies. It's just a question of earning more with the same amount of work.

Edit: due to the rise in BTC exchange rate, I changed the price back to 1 BTC.

Maybe you would be interested in publishing economic articles at The Bitcoin Times, a new fledging magazine.

We already got a few writers lined up and for sure, an advertiser.

I wonder why no big player seems to be using the strategies described in the paper. They seem easy to implement. Could it be that big players rely on brute force and don't care about electricity costs ?

How would you know if a miner was moving back and forth between the pool and working on his own?

It would only be obvious if lots of people were turning off causing very high variance in time to find a block for the pool.

I don't understand this. In the extreme, if everyone left a pool, why would a miner join it? They would get a block no faster than working on their own and they would pay to previous workers who had shares.

I think this could be solved by letting everyone know that shares will begin to decay after some multiple of average time to find a block.

I know it isn't a problem now because there is only one huge pool, but when there are many and switching is easy an unlucky period could easily drive people away and no one will have proper incentive to rejoin until it is cleared.

Interesting article, I brought the subject up back when the first pool switched from current power to the contributed method, but I don't think anyone responded.

Switching back to connected mode would completely solve this problem. I liked connected mode (and puddinpop's pool in general) better, anyway -- you get larger payments less often. Maybe once a contributed-mode pool has stalled, everyone will move to a connected-mode pool.

It is a neat psychology + game theory situation. People seem to prefer the contributed method to connected, but as soon as shares are owed (so instantly) the right move is it switch to a connected pool if it is available. But if the herd all prefers contributed then you will be on the connected server alone and now you don't have the benefit of reduced variance. As some 'serious' miners decide to accept the variance and move to the slightly higher EV pool others may follow and variance will drop at that pool and now there are only psychological reasons to stay at the contributed pool.

As we know from poker though people can persist in low value emotionally pleasing situations for a long long time.

Eh?  What is the difference between the "contributed" and "connected" methods?  I haven't read Ryo's paper, so maybe I'm missing context.

Puddinpop's pool software allows the server to choose between "connected" and "contributed" distribution methods. In contributed mode, it behaves somewhat like slush's pool: every hash you calculate is counted, and the block reward is split among everyone according to how many hashes they calculated. In connected mode, the block reward is spit among currently-connected nodes only, according to the hash/s that they were contributing at the time.

I wish puddinpop's server would become popular again. It's open-source and built in such a way that miners can actually see what the server is doing on the network side (you download the full temporary block). Unfortunately, running the server is CPU-intensive, and probably it could not sustain the number of miners that slush's pool has now.

In the contributed every time you find a hash below some 'easy' target you get a share that is paid when the pool finds their next block, if you stop mining you will still get paid something if you previously earned at least one share provided the pool finds one more block.

In the connected method you get paid if you are connected and mining when the pool finds a block.  You are paid in proportion to how much power you are providing compared to others at that moment (really not a moment more like 2 minutes I think it was). You could think of this as the contributed method with shares that expire really fast. There is plenty of in-between room, make shares decay slowly or disappear after a day or whatever.

In the extreme, yes. But if the powerful miners leave after a few thousand shares, the less powerful miners may find it profitable to stay, because they will get a bigger share of the rest. The problem, for small miners, is that when you're up against people that have 1000x your computing power, they will find blocks before you even contributed a share, and you will gain nothing.

It seems easier for the server to cheat in connected mode. With contributed mode, if they have doubts, miners can log how many shares they contribute, check how much they gain, and decide whether it's fair. The server could "invent" imaginary shares to keep some bitcoins, but at least you have some hard numbers to check. With connected mode, how would you know if you got a fair share ? What if the server pretends you were not connected when the block was found ?

No they won't. The calculation they ought look at is "what is the expected value of a new share here?" this will be less at a pool with lots of shares owed to powerful miners who have left than at a fresh pool.

Perhaps I'm missing something though because I don't see on any margin any way to interpret someone leaving your pool as a positive. All that happens is that you get a larger share less often. If that is what you want then choose a pool with the least people, mine by yourself. If there is some optimal pool size then what is it? When is bigger not better?

Let's take two simple scenarios:

- A pool with three people. Two of them have clusters of 5970s, one of them has an old Pentium IV. For blocks that are found before 10000 shares or so, that third person won't even have submitted one share, and thus will get nothing, not even a small fraction of a bitcoin.

- A pool with three people. Each of them has an old Pentium IV. Blocks will take a long time to be found, but most likely a lot of the members will have submitted at least one share.

If the goal is to not get left out when your pool makes a block then you want the second. But the two have the exact same EV and the first has lower variance.
 

I agree, this was written from the point of view of a miner estimating his expected gain from one specific block (or more precisely, one round), but as long as blocks (or pools) are exchangable, there is no reason to do this. If he has a choice, a small miner should switch to a pool with a "younger" round. I don't belive he should start mining all by himself is there is no other pool, though; his expected gain would drop close to zero, because of the increasing difficulty.

I am no economist, even if might have things to say about game theory and optimization from times to times. I can write about computer science, though, and I already asked in the "bitcoin times" thread if you would be interested in publishing a series of articles about the science behind bitcoin or other crypto-stuff. Writing such articles is time-consuming, and I wouldn't want to do that unless some interest was voiced beforehands.

If by expect gain you mean formal EV then it doesn't drop close to zero it stays close to zero. It's the variance that is reduced by the pool. I don't mean to imply that you don't know this; I'm only saying that the language is not precise.

I do mean expected value, but by saying "gain" I make sure we're not talking about expected time before a share or some other value.

Again, I didn't make calculations, I'm just using intuition (which can be tricky), but I'm pretty sure that expected gain decreases if you mine all by yourself and you're not powerful enough to have a good chance of finding a block before the next increase in block difficulty. At current difficulty, my CPU would take about half a year to have a 50% of finding a block, but before that difficulty will have increased, so my chance of finding a block in half a year is actually lower.

Is it clear that in the period from here to the next difficult change the EV of mining in a fresh contributed pool or a connected pool is identical to the EV of running alone?

How about over the period from the start of new difficult to the new difficulty after that?

If each period is equal then the sum of the EVs must be equal.

That said there is obvious benefit in reducing variance. The benefit is more than psychological; it allows more accurate planning on when you will be able to afford new equipment, etc.

I can envision a future where lottery tickets are 10 minutes with a powerful GPU rig. Or where even a big mining operation cannot be sure to get a block this month so they need to be in a pool to make sure they can pay the electric bill.

It's more difficult, in fact. The server continually tells you what it believes your hash/s is. You can compare this number with other servers to see if it is reasonable. When a block is found, you can calculate your share from the server-reported total hash/s. Over many blocks, you can check that the server's hash/s is accurate by seeing if the actual time to solve blocks matches the projected time at that hash/s.

People will get very angry if they appear to disconnect just before a block is solved...

Since the shares in puddinpop's server are divided within the generation transaction and you download the temporary block, you can also see exactly how the block reward would be distributed at any given time. Verify that you and some other people are listed at an appropriate amount.

Short quote from document:

It's not true, at least for existing pools (puddinpop's and mine), because each instance of pool has own merkle root. So submitting shares from another pool (even based on the same code) won't work. Please correct this in your paper.

About disconnecting from pool after some count of shares; There are much more factors than price of one share and power consumption. For example, rising network difficulty or some fixed costs (costs from owning mining rig, even in idle). If there is only one pool and you are big guy, disconnecting from pool saves your energy, but you have to wait much longer for new round, because pool is missing your hash power. So you are waiting, but new higher difficulty is coming and later it will be even harder to find new block... But I didn't spend time thinking about it, so this is just my intuition, not exact math.

Good point is the switching between pools. But unless there are not two pools with similar hashpower, it's probably not relevant. Any rule which affect main formula for calculating rewards from contributed shares probably change overall pool fairness in some way.

I think your paper is interesting and thank you for working on this. I'm just missing some graph in section 4.2, which shows time (X), hashpower(Y) and vertical lines where new block was found. All of this is public information and it would add more weight to sentence "there was no evidence that people gave up on old blocks".

Ah, also one note regard to 'unfairness of short rounds'. No, it is completely fair, but in longer term, not in one (short) round. Yes, when you are lucky and find 10 shares in 80 shares round, you will receive significant amount of bitcoins from this round. But everyone has the same probability to hit this premium in every short round. More exact, everyone with the same hashpower has the same probability, of course.

This is not about past performance, on the contrary. At any point in time, no matter how long the round has been active, there is the same expected time before finding a block. But what you will gain is not constant, it depends on the number of shares. Because of this, you can optimize.


Thanks, I will correct this.


Even with two identical pool, unless they start their rounds at the same time, you can switch between them and gain a lot.


I have to admit I have been lazy. I intended to provide that graph, but there was a network outage during the time I gathered my data, so there was a gap of a few hours, and I didn't want to start again. If you keep a log of computing power over time, and rounds over time, maybe you could send me the data, so I don't waste your bandwidth again ?

What isn't 'fair' is comparing outcome per/round when the rounds can be vastly different lengths. No one rational cares about payout/round, they care about payout/time.

Think about this. Brand new pool, I start mining, if I find a block I get it all because I'm the only one there. Later someone joins me with exactly my hashing power, we are contributing equally, but I will get more than him under contributed method. Why is he paying me for mining before he got there? I wasn't going to give anything to him if I found a block back then, he gets no benefit from the fact that I previously mined, but he pays for it anyway.

Contributed method is not a rational equilibrium. But neither are a lot of things that persist for a long time. 

I did an extensive analysis of this a couple weeks back, with the aim of potentially working it into a short story.  Basically when you look at expected value per cpu/gpu cycle the beginning cycles of every round have much higher expected value and decrease the longer the round progresses, asymptoting out to the current network rate.  If everyone stays connected all the time at a consistent rate, or disconnect/reconnect with no correlation to the number of shares in the round so far, it all balances out to the same as mining on your own, just with reduced variance.  Whether you actually find shares or not and when varies from round to round but not in the long run, where it averages out (with the one interesting implication that the larger a cooperative effort gets as a percentage of overall network power, the higher the variance becomes for a set amount of hash/sec).

The specifics of the successful attacks are as follows:  if your aim is to maximise your expected value per cpu/gpu cycle you should throw as much power as possible at finding the first share in each round, quitting once it is found.  Your value per cycle goes through the roof for a decent sized pool, but your overall income is negligible because even a person with access to an extraordinary amount of idle processing power will only earn a small income in any given period of time--you only compute during extremely brief periods and have to wait for the next round to do it again.  Your income, however, is disproportionately higher than spending that same amount of cycles mining by yourself, and the income comes from the other pool participants.  And this type of attack would be immediately noticeable to the pool operator no matter how you set up the clients.

Strategies that leave the pool later and later in the round progressively reduce your value per cycle in exchange for being allowed to use more of them, thus increasing your actual income and decreasing your advantage due to the exploitation.  Any strategy that has you leaving the pool after X shares are found without a block gives you a higher value per cycle than working on your own, but only by a negligible amount for high X.  Strategies that use your time out of the pool to hash solo thus give an above average income, but with a higher variance--you choose the tradeoff by setting X, and this assumes that your hash/dollar makes it profitable to mine bitcoin in the first place.  If it didn't you need to not mine on your own and set X at a low enough value to provide you with actual income.  There are edge cases where these types of strategies might make sense, but your maximum gains from them are pretty low, and any reasonably sized value of X will be easily noticed by the pool operator.

My analysis--it would take some pretty severe edge cases and a very oblivious pool operator to make any significant money this way.  And "cyber-genius makes off with $500 in untraceable cash" makes a lousy heist story  :) .  Chances are anyone capable of coding this attack would make more money simply going to work.  But it's good for pool operators to be aware of.  One of those extreme edge cases did turn out to provide interesting plot fodder though, so stay tuned on that one ;) .

Sincerely,
eMansipater

All the pool has to do to prevent this type of abuse is weight the reward of the shares by exp( current shares / difficulty )

Tips can go to 1LyHQFP41e6PtgKaEHHXresnyszE3YrTF6 for anyone who found it useful.  One of the exciting things about bitcoin for me is that the emergence of a successful microcurrency allows us to develop a culture of tipping , akin to what has emerged in restaurants and similar service industries.  Enforced by no one, yet powerful enough to provide meaningful incomes on the internet scale.  If we can train people that when something is valuable to you, you tip that person in some fashion no matter how small the amount, we can start developing meaningful post-advertising models for web content.  We have a little bit of work to do to make it easy and user-friendly, but if we can get people in the habit of tipping at least something, no matter how small, when you want to support a certain type of content, things can really add up!  Giving someone 2 or 3 cents, times a million visitors, is pretty good incentive to keep producing content.

/rabbit trail

sincerely,
eMansipater

Agreed.  Jumping pools doesn't damage/cause any time of fraud or attack against that pool.  That's just someone who is *trying* to play their odds, but the odds of winning in some pools and loosing in others will balance out.  The point being, it's useless to jump pools.

Pick one and stick with it.  Why anyone put any credit into pool jumping I don't know, but if you think it's worth it, then give it a try.....you'll be back to mining in a single pool within a few days.

This is just wrong. If you know a pool has fewer share owed you get a bigger % faster for the same power.

Possible attack against a connected-mode pool: when you find a pool-grade hash, send it immediately to gain credit, but when you find a block-grade hash, hold it until the pool hashrate dips.

This introduces some risk that someone outside the pool will find a block in the meantime, so you can't hold too long, but the optimal wait time is almost certainly greater than zero.

Probably not. I didn't do exact math around it, but though about this possibility, too. You cannot take the share too long, because with new bitcoin block, your share become invalid. And this time should be proportional potential gain of delayed submit... Of course I'll welcome exact calculation about this possibility.