Bitcoin & Tragedy of the Commons

[Technomage]

Block space is limited, as is the computing power to validate the signatures in the transactions.  Right at the moment neither of these is really scarce, as understood by laymen, but they aren't unlimited either.

This is a good point. That is a real issue that actually requires some effort from Bitcoin developers in the next couple of years. Bitcoin still has some room to grow, a lot of room actually, but it doesn't scale very well after a certain point. This is one of the bigger priorities but I'm not too worried about that. Bitcoin has very smart people working on it and it seems that the majority of Bitcoin use will move toward client-server wallets which buys us a good amount of time to come up with real solutions.

Another important issue is how do we actually make the shift to the transaction fee market scenario that I'm talking about. It would probably go smoother if Bitcoin developers co-ordinate with all the big pool operators so it is a smooth experience to Bitcoin users. It will take time for Bitcoin users to get used to the fact that transactions will cost something and that the speed of their transactions will depend on how much they pay. From an UI and usability perspective this needs to be clear and easy to understand. The clients also need to be constantly up-to-date on how long it takes on average for a transaction to be included in a block at a certain fee.

However that issue is probably even less of a priority than the scaling issue. It will still take many years until there is real pressure to raise transaction fees. Even a block reward of 25 BTC is still quite adequate to not worry about this. Scaling could become a real issue before we have to worry about miners' rewards.

[Maged]

There is a whole 'nother dimension to this that you guys are ignoring, and that's time. In addition to only accepting transactions with a certain fee, miners might not even attempt to mine a block at all until it has enough in fees to be profitable. Alone, this may be bad. However, there are several interesting dynamics to consider, even if you assume miners are completely logical:

1) Plenty of people have reasons to get their transaction fully confirmed ASAP. For example, MtGox users. So, users will include a large transaction fee in hopes of convincing more miners to turn on to mine the block. They'd then likely send fee-only transactions in the next 5 blocks to make sure that the mining keeps up.
2) Many people will need to do this several times per day, however they won't know when they'd be able to "freeride" off of someone else until the transaction is on the network. Besides, if you need something to clear right now, you didn't plan ahead in the first place, and it's impossible to guess if someone else might shortly cover the cost of running a large amount of miners for you.
3) Different miners have different profitability points. Even if someone else already paid to have several miners running, there are always several more miners that can run if you pay just a bit more in fees. Currently, all active miners are mining at 100% anyway, so this is an effect that you can't yet see.
4) Businesses will contract out to certain miners to ensure that low priority transactions (such as low-value POS transactions where the person also provided their ID) get included in a semi-timely manner for a monthly rate (businesses love stability). MtGox already does this.
5) A certain amount of people will mine no matter what, likely including every transaction they safely can (remember, the larger a block is, the longer it takes to validate, the more likely it is that someone else will find another block that orphans yours).
6) After a miner successfully mines a (particularly profitable) block, they will either mine at full power for a few blocks to protect their block, send fee-only transactions using stored up funds that will hopefully be reimbursed out of their profits from the block to entice other miners to mine the block (this is where a protocol change would be nice so that this can actually come directly out of the mined funds or at least be dependent on them), or likely a combination of both. This will help smooth block times a bit.
7) Miners will have invested a good deal of money in specialized hardware to mine bitcoins. As a result, any attack that could potentially undermine Bitcoin would likely be met with the FULL force of ALL specialized mining hardware which will have been programmed by their owners to reverse the attack in order to protect their investment. As an added plus, we gain a huge security through obscurity benefit here since it'd be impossible for an attacker to predict how much mining power the whole network actually has until they've tried attacking it.

Altogether, this will come together in such way that Bitcoin will remain secure and block times will be fairly stable. There are so many variables in play here that it's really quite brilliant. At this point, the only thing I would be concerned about is a 51% miner that rejects everyone else's blocks. Any other attack, economic or not, will fail miserably.

[cunicula]

Altogether, this will come together in such way that Bitcoin will remain secure and block times will be fairly stable. There are so many variables in play here that it's really quite brilliant. At this point, the only thing I would be concerned about is a 51% miner that rejects everyone else's blocks. Any other attack, economic or not, will fail miserably.

This is a completely ridiculous argument. Are you braindead or just fucking with us?

Let me paraphrase you:

Sally and Harry might be convinced to mine some.
That proves that people can be convinced.
Therefore, it will be possible to convince enough people to mine.

[phelix]

[...]'nother dimension[...]

+1, good post

really hope it will work like this

edit: this will cause quite some thermal stress for the mining hardware

[...]I will pull some numbers out of my ass to illustrate this[...]

[...]
Pool x has a fee threshold of 0.1, meaning that they only accept transactions that have at least 0.1 bitcoins as a fee. This is the "high speed, high prices, low volumes" pool.

Pool y has a fee threshold of 0.05, which is "if you want decent speed but don't want to pay a premium" type pool.

Pool z has a fee threshold of 0.001, this is the "cheaper than the competitors, but slow as fuck" type pool.
[...]

this is hard to imagine working as most of the time it will be more profitable for pool x to also include all the low fees.


I can see mining guilds forming that ddos pools accepting low fees.

[Meni Rosenfeld]

Block space is limited, as is the computing power to validate the signatures in the transactions.  Right at the moment neither of these is really scarce, as understood by laymen, but they aren't unlimited either.

Block size currently has an arbitrary limit of 1MB. It is generally understood that when there are too many transactions it will be lifted or relaxed, so actually there is no real limit. (If a sufficiently tight permanent limit is set, we will not have this problem. But this of course could hurt scalability). The cost of validating signatures is negligible.

To put it simply what will happen is this:

We will have a large number of pools and most likely many different p2pools as well. I will pull some numbers out of my ass to illustrate this, in reality the market will decide these numbers.

Pool x has a fee threshold of 0.1, meaning that they only accept transactions that have at least 0.1 bitcoins as a fee. This is the "high speed, high prices, low volumes" pool.

Pool y has a fee threshold of 0.05, which is "if you want decent speed but don't want to pay a premium" type pool.

Pool z has a fee threshold of 0.001, this is the "cheaper than the competitors, but slow as fuck" type pool.

Pool z will include all transactions with a fee above 0.001, which includes all transactions included by pool y, which in turn includes all transactions included by pool x. The total block reward in a block found by pool z will be higher than in pool x. The expected payout for mining at pool z is higher than in pool x. Nobody will mine at pool x.

Once again - including transactions is cheap. You can include all of them (that is, those with fee higher than the tiny cost) and increase your profits. Everybody will do that, unless he is big enough to individually (or as part of a cartel) have a real bargaining power, which itself would be a problem.

[cypherdoc]

I think you guys totally underestimate the desire of the marketplace to disintermediate the banks:

 http://www.telegraph.co.uk/finance/economics/9143581/Technology-could-take-the-bankers-out-of-banking-says-BoE-policmaker-Andy-Haldane.html

[Meni Rosenfeld]

In addition to only accepting transactions with a certain fee, miners might not even attempt to mine a block at all until it has enough in fees to be profitable.

Unless something else can be done with the hardware, it's quite possible miners will keep mining to avoid thermal cycles. Whether this will happen is TBD.

1) Plenty of people have reasons to get their transaction fully confirmed ASAP. For example, MtGox users. So, users will include a large transaction fee in hopes of convincing more miners to turn on to mine the block.

Tragedy of the commons, the contribution of a sender to confirmation time is everyone's, the cost of the included transaction fee is all his own. ie, the personal effect on paying a fee for the confirmation of your own transaction is negligible.

They'd then likely send fee-only transactions in the next 5 blocks to make sure that the mining keeps up.

Even more tragedy of the commons, since they don't even get any preferential treatment for fees paid this way. But it would be interesting to find an incentivization scheme which does incentivezes the miners of the next blocks.

4) Businesses will contract out to certain miners to ensure that low priority transactions (such as low-value POS transactions where the person also provided their ID) get included in a semi-timely manner for a monthly rate (businesses love stability). MtGox already does this.

Explanation required.

7) Miners will have invested a good deal of money in specialized hardware to mine bitcoins. As a result, any attack that could potentially undermine Bitcoin would likely be met with the FULL force of ALL specialized mining hardware which will have been programmed by their owners to reverse the attack in order to protect their investment. As an added plus, we gain a huge security through obscurity benefit here since it'd be impossible for an attacker to predict how much mining power the whole network actually has until they've tried attacking it.

See also this thread.

[Technomage]

this is hard to imagine working as most of the time it will be more profitable for pool x to also include all the low fees.

Care to explain this further? The miners who decide to accept only high fee transactions have incentive to not include any low fee transactions. If they did, users wouldn't have any incentive to pay for higher fees. For higher fees to work, a significant amount of hashing power need to set a higher fee threshold. If they just accept whatever, then users have no incentive to ever pay more than the minimum. This is the way it is now but I can guarantee it won't be like this forever.

I can see mining guilds forming that ddos pools accepting low fees.

This is a valid point but it's not that big of a problem. I imagine p2pool type solutions will become more common in the years to come and they are very ddos resistant.

[Technomage]

Pool z will include all transactions with a fee above 0.001, which includes all transactions included by pool y, which in turn includes all transactions included by pool x. The total block reward in a block find by pool z will be higher than in pool x. The expected payout for mining at pool z is higher than in pool x. Nobody will mine at pool x.

It's not that simple. The calculations for this gets a little complex at this point but first it's important to understand that a scenario where there is only pool z is already discarded.

Everyone mining at pool z, which basically represents the way mining works now, is not profitable anymore because traditional block rewards have all but disappeared. This is the scenario which we start with and start to work on. I have to add that it would of course always be profitable to a certain amount of people, difficulty would adjust. But a network based only on the actual cost of sending and verifying transactions is out of the question, Bitcoin will require more than that.

What happens is that major mining businesses who don't want to just quit, start raising their fee thresholds. This eventually creates a situation where you might only have 10% of total hashing power in pool z and everyone else has moved on to either the high fee pools or the medium fee pools. It's true that pool z gets the fees for everything but they only find 1/10 of the blocks. All users that appreciate speedy transactions will pay more than the minimum and 90% of those blocks do not benefit pool z in any way.

I find it perfectly possible that all of the different pool classes can be profitable, except perhaps the one that only accepts minimum transactions. I'd imagine that at this stage a very large majority of users want to pay more than the minimum to make sure their transaction is at least in the next couple of blocks. Pool z gets a small share of that action.

There would of course be constant changes in the hash power of different fee classes. Miners would change based on what users are paying and users would change what they're willing to pay for certain speed. I feel this really could work and it would create a very interesting market overall.

Once again - including transactions is cheap. You can include all of them (that is, those with fee higher than the tiny cost) and increase your profits. Everybody will do that, unless he is big enough to individually (or as part of a cartel) have a real bargaining power, which itself would be a problem.

Including transactions is cheap, that is true. What is also true is that in the future miners have more bargaining power than they do now. Mining pools will start raising their fee thresholds at some point, I see this as pretty much inevitable. It makes a lot sense and I hope Bitcoin devs are aware of this. It's not a priority issue right now but eventually we will be in a situation where the fees must be higher to sustain a mining network that secures Bitcoin. The best way to do it is enabling a market between miners and users, similarly to what I've been explaining.

[Technomage]

I would like to add that it's difficult for a cartel/monopoly to happen. Bitcoin mining might get a little more centralized than it is now but it will still be very distributed. Even if someone controlled 20% of hashing power, he simply could not force anything on users. He can demand high fees but if users are not willing to pay, then he gets pretty much nothing. Users would have slightly slower transactions but by not paying the fees they would force the 20% player to lower his fee threshold because he is asking too high of a price. This is text book stuff btw.

So basically if the miners start to get rich, Bitcoin userbase has ultimate power to stop that. They only need to stop paying high fees for a while and the pools that demand higher fees are forced to correct their prices. Especially with Bitcoin price probably still being fairly volatile at that stage, it's important for users to stay awake. If Bitcoin price doubles and the pools still demand similar fees, then something is wrong.

[Technomage]

It's easy to think that everyone would just mine at the pool that accepts all transactions, because their blocks have the biggest rewards. Fact is that it just doesn't work like that. If everyone did that, we would be back to square one where mining is unprofitable. This is why there will be a sort of profitability index or "difficulty" for each pool, not just mining in general. If too many miners go to a low fee pool, suddenly users are not paying high fees anymore because there is sufficient hashing power in the low fee pools. Thus the profitability of all mining goes down. This is not in the interest of miners so it simply won't happen.

[phelix]

Pool z will include all transactions with a fee above 0.001, which includes all transactions included by pool y, which in turn includes all transactions included by pool x. The total block reward in a block find by pool z will be higher than in pool x. The expected payout for mining at pool z is higher than in pool x. Nobody will mine at pool x.

[...]
What happens is that major mining businesses who don't want to just quit, start raising their fee thresholds.
[...]

to become even less profitable?

bad behavior is incentivized, good behavior is not. that's why it is hard to imagine it could work (similar to communism).

there might be a solution to get it right simpler than proof of stake by changing the fee mechanisms.

Examples (just brainstorming):
* limit the tx fee spread within a block   (factor 10: 0.1 highest --> 0.01 lowest fee alowed)
   spread could be hard coded or voted on by the miners
* have the miners vote on future minimum fees / spread by destroying coins

[Technomage]

to become even less profitable?

bad behavior is incentivized, good behavior is not. that's why it is hard to imagine it could work (similar to communism).

Not less profitable, more profitable. Imagine if 80% of hashing power simply decided that they've had enough of these free transactions. Their mining hardware costs money, it requires work and they want some profit. Traditional block rewards are a fraction of what they were back in the day. They want to keep mining and supporting the network but it's simply unprofitable.

As an attempt to make mining profitable again, the 80% change their fee threshold from 0.001 to 0.01, effectively increasing the fees by a factor of 10. Now users have a choice, they can either send transactions with a 0.001 fee and wait on average for 50 minutes to be included in a block. Or they can add a 0.01 fee, which is still quite cheap, and make sure that their transaction is in the next block.

What do you guys think users will choose? I believe a lot of people would choose paying 0.01 which would basically increase the overall profitability of mining by a lot. It would increase even for the ones who stay mining 0.001 transactions. Everyone would benefit, except the users of course, but they must understand that mining doesn't run on free hardware and free energy.

[Meni Rosenfeld]

What happens is that major mining businesses who don't want to just quit, start raising their fee thresholds. This eventually creates a situation where you might only have 10% of total hashing power in pool z and everyone else has moved on to either the high fee pools or the medium fee pools.It's true that pool z gets the fees for everything but they only find 1/10 of the blocks. All users that appreciate speedy transactions will pay more than the minimum and 90% of those blocks do not benefit pool z in any way.

Who has moved on? Miners?

You're confused about how mining and mining pools work. If a pool has N miners (suppose for simplicity all pooled miners have the same hashrate), then it finds on average N*a/D blocks per day, where D is the difficulty and a is a global constant. If the average reward per block is B, the average total reward per day is B*N*a/D, which is distributed among N miners, so each gets B*a/D on average. The only thing that changes between pools is B (based on their inclusion policy), so miners will always go to the pool with a higher B (which is the pool that includes all transactions). The fact that this pool is "only 10% of the total hashing power" has no effect on this.

A smaller pool will have more variance, but since the all-inclusive pool pays the best, there's no reason it should be small.

The end result is that pools which only include high-fee transactions will not be competitive, and 100% of the pooled mining hashrate will be in low-fee pools.

And the point remains that although all pools would be better off if all pools accept only high-fee transactions, no pool would initiate a move to higher threshold (without being part of a cartel) because it will decrease its own profitability. Which, once again, is called "tragedy of the commons", or "prisoner's dilemma" in the 2-player case.

I find it perfectly possible that all of the different pool classes can be profitable, except perhaps the one that only accepts minimum transactions. I'd imagine that at this stage a very large majority of users want to pay more than the minimum to make sure their transaction is at least in the next couple of blocks. Pool z gets a small share of that action.

It gets a share of that action proportional to its hashrate, like any pool. And in addition it gets a share of the low-fee action.

Not less profitable, more profitable. Imagine if 80% of hashing power simply decided that they've had enough of these free transactions. Their mining hardware costs money, it requires work and they want some profit. Traditional block rewards are a fraction of what they were back in the day. They want to keep mining and supporting the network but it's simply unprofitable.

Now imagine that 80% changing their fee threshold from 0.001 to 0.01, effectively increasing the fees by 10. Now users have a choice, they can either send transactions with a 0.001 fee and wait on average for 50 minutes to be included in a block. Or they can add a 0.01 fee, which is still quite cheap, and make sure that their transaction is in the next block.

Yes, if 80% of hashrate forms a cartel and make an explicit agreement to play hardball, they will increase their profit. But even after the cartel is formed, members of the cartel are better off if personally they defect from the cartel and start accepting all transactions (unless the cartel enforces it somehow), as well as non-cartel members who will still accept all transactions. The cartel would have to crush competition by rejecting all blocks who do not conform to the cartel. And that is not a scenario I wish to happen.

It's easy to think that everyone would just mine at the pool that accepts all transactions, because their blocks have the biggest rewards. Fact is that it just doesn't work like that. If everyone did that, we would be back to square one where mining is unprofitable. This is why there will be a sort of profitability index or "difficulty" for each pool, not just mining in general. If too many miners go to a low fee pool, suddenly users are not paying high fees anymore because there is sufficient hashing power in the low fee pools. Thus the profitability of all mining goes down. This is not in the interest of miners so it simply won't happen.

Now I'm sure you haven't read Prisoner's dilemma and Tragedy of the commons (the former is clearer, it refers to a 2-player case but the logic is the same). Every miner will act in his own self interest, not in the interest of other miners. Whatever other miners do, the most profitable thing to do is to defect (which in this case means accepting all transactions).

Everyone will do that, and as you correctly state, we will be back to square one where mining is unprofitable. (And it's not necessary that everyone will do that, it's enough that someone will do that).

This can be resolved if we change the rules so that the global interest is aligned with everyone's self interest. But this requires a coordinated effort, it doesn't happen spontaneously.

[FreeMoney]

A miner can try to demand high fees, but if 99% of miners accept low fees, nobody will be willing to pay these high fees.

No, that's my whole point. Anyone who wants >99% chance of being in the next block will pay higher fees. Especially if the fee is like .01 on a 100BTC tx.

Even needing to rely on this pricing power phenomenon seems unlikely to me and it's just there as back up.

Already we're paying miners fees so that non-mining nodes will take us seriously enough to pass the tx on.

There are many other things like this that can happen. Maybe some subset of people will be appalled to receive a tx that didn't have a fee. Uncertainty about who these people are may lead to a .0001 (don't hate me for being a freeloading jerk) standard.

Maybe a lot of miners won't keep a complete chain and will be unable to easily verify small or old inputs. You'll need a fee if you don't want to wait for the one generous soul who keeps the whole thing and charges no fees.

[Technomage]

I know about the tragedy of the commons but let's just say that I've always been very skeptical about it, it really does not apply well to most real life scenarios. I understand what you're saying and maybe I'm the one who is clueless here but I'll try my best now to make sense of all of this.

I don't believe that high fee miners will defect to low fee pools simply because it appears that they would get more by going there. In fact they get significantly less because by doing that users don't have incentive to pay high fees anymore.

To think about this in another way is to imagine that all miners will defect to the low fee pools. What happens then? Mining becomes unprofitable and some miners will again attempt to raise fee thresholds.

Of course it won't happen like this in reality, there will be a constant equilibrium where the increase of miners in low fee pools results in the decrease of profits for everyone which basically results in low fee miners moving back to high fee pools because that's the only way they can keep mining profitably.

So in conclusion I still think that what I'm talking about could work and fairly well actually. But I don't mind if you prove me wrong.

[FreeMoney]

@meni

It's true a tiny miner will simply migrate to a pool that takes all fees. However, in the all tiny miner  situation where no one has pricing power the tiny miner doesn't mine at all. But he has cheap/free electric, nice GPUs etc. So he offers to sell it to some smart guy. Smart guy realizes that buying the power of 4000 tiny miners will give him pricing power and profit. It happens to start raising the level of fees across the board making it harder or more expensive for him to buy more miners power because they can better profit on their own now.

[FreeMoney]

Of course it won't happen like this in reality, there will be a constant equilibrium where the increase of miners in low fee pools results in the decrease of profits for everyone which basically results in low fee miners moving back to high fee pools because that's the only way they can keep mining profitably.

They can sell their power to someone who realizes it will make more coin by working together.

[Technomage]

Good points by FreeMoney. That is certainly a market where more awareness helps. If everyone understands that certain actions will benefit all (including me) and certain actions will be harmful to all (including me) then I don't see how the tragedy of the commons can surface in any significant way. It doesn't require a cartel either, all serious miners simply know that it makes no sense for them to accept only low fees because it will crush profits very quickly (in a matter of hours, once user clients realize that you can make fast transactions with low fees). So the feedback is very quick as well especially if there is a lot of Bitcoin usage / high number of transactions.

There will of course be defections and that's natural. Someone decides to mine at a low fee pool, hoping he can get away with it. Eventually these people notice that the grass wasn't greener on the other side, they only got more profits for a short while. Too many people defected and now they profit less than they did originally at the high fee pool. These people will change back to high fee pools in an attempt to increase the incentive for users to pay higher fees. Thus we have a constant back and forth which results in an equilibrium.

[Meni Rosenfeld]

I know about the tragedy of the commons but let's just say that I've always been very skeptical about it, it really does not apply well to most real life scenarios. I understand what you're saying and maybe I'm the one who is clueless here but I'll try my best now to make sense of all of this.

In some cases tragedy of the commons is overcome when there are additional factors at play. It depends on the ratio between incentive to defect and the stabilizing forces.

But I think it's much more prevalent than you think. Voting is a classic example. AFAIK voting rates in US are very low, and it is generally agreed that the total welfare would increase if everyone voted (the total effort of all people voting is justified by the election of a party which better addresses people desires). But for each person, the effect of his own vote on his own welfare is negligible, so most people pass.

Another example is software. Would it not be better if instead of dealing with DRM, registration keys, closed platforms and so on, companies would allow anyone to download the software, specify a price, and receive payments on the honor system? If this was in place and everyone (who pay under the current system) paid, everyone would be better off. But everyone would be incentivized to spare himself this cost, which eventually results in the breakdown of this model and resorting to DRM. (Some people have offered software on an honor system, somewhat successfully, but mostly because of the novelty factor - it doesn't scale.)

There are no doubt cases where it could have occurred, if society hadn't been able to redefine the rules to dispel it.

People have been suggesting all sorts of factors that would mitigate the tragedy of the commons problem in Bitcoin mining, but I still doubt that they will be enough quantitatively. We need something about which we will be able to be much more confident.

I don't believe that high fee miners will defect to low fee pools simply because it appears that they would get more by going there. In fact they get significantly less because by doing that users don't have incentive to pay high fees anymore.

To think about this in another way is to imagine that all miners will defect to the low fee pools. What happens then? Mining becomes unprofitable and some miners will again attempt to raise fee thresholds.

Again, if I am a small miner, the effect of my mining on the equilibrium is negligible. By going to a high-fee pool, I'm increasing the frequency of high fees by about 0.0001% (which I will also enjoy). If I go to a low-fee pool with twice the total reward, I increase my rewards by 100%. I would much rather that all other miners go to the high-fee pool; but whether they do or don't, I'm still better off at the low-fee pool.

In other words: My personal benefit from defecting is always higher than my share of the global benefit of me cooperating. I lose from other people defecting, not from me defecting. (Unless I'm a sufficiently large miner.)

You need some balancing factors to prevent that from happening. And they need to be stronger than the total incentive to defect.