[If tx limit is removed] Disturbingly low future difficulty equilibrium

[casascius]

 A potentially scary power on one hand, especially if future ASICs raise the barrier of entry to mining so high that only well-connected people can get in...kind of like banking now...

ASIC's wouldn't raise the barriers to entry in the long term.  Only increase the overall security of the system for roughly the same cost.  ASICs continue to exist in the modern world because they are cheap in quantity.

An ASIC by definition is a custom piece of silicon.  It's not something you can go pick up at Radio Shack.  If someone goes and creates an ASIC for mining, and decides to keep the design private, then the only people who can compete with him are those with the resources to go design ASICs as well.  Wikipedia puts it nicely: "the non-recurring engineering cost of an ASIC can run into the millions of dollars".  Compare to an ATI 5970, at a current retail cost of about $650.

[FreeMoney]

 A potentially scary power on one hand, especially if future ASICs raise the barrier of entry to mining so high that only well-connected people can get in...kind of like banking now...

ASIC's wouldn't raise the barriers to entry in the long term.  Only increase the overall security of the system for roughly the same cost.  ASICs continue to exist in the modern world because they are cheap in quantity.

An ASIC by definition is a custom piece of silicon.  It's not something you can go pick up at Radio Shack.  If someone goes and creates an ASIC for mining, and decides to keep the design private, then the only people who can compete with him are those with the resources to go design ASICs as well.  Wikipedia puts it nicely: "the non-recurring engineering cost of an ASIC can run into the millions of dollars".  Compare to an ATI 5970, at a current retail cost of about $650.

If the someone who doesn't share the design is the only one with a design then yes, but that doesn't seem likely.

And just because they can cost millions doesn't mean one that does some hashing will.

[Meni Rosenfeld]

 A potentially scary power on one hand, especially if future ASICs raise the barrier of entry to mining so high that only well-connected people can get in...kind of like banking now...

ASIC's wouldn't raise the barriers to entry in the long term.  Only increase the overall security of the system for roughly the same cost.  ASICs continue to exist in the modern world because they are cheap in quantity.

An ASIC by definition is a custom piece of silicon.  It's not something you can go pick up at Radio Shack.  If someone goes and creates an ASIC for mining, and decides to keep the design private, then the only people who can compete with him are those with the resources to go design ASICs as well.  Wikipedia puts it nicely: "the non-recurring engineering cost of an ASIC can run into the millions of dollars".  Compare to an ATI 5970, at a current retail cost of about $650.

Exactly, designing the GPU for ATI 5970 cost millions of dollars but ATI still sells them to me at $650 because they return their investment many times over by selling to millions of consumers. Similarly, someone will find it profitable to design a mining ASIC and sell it. Also, in the same way that some manufacturers allow their products to be rebranded and sold by other companies, I don't find it at all unlikely that a mining company who designed an ASIC for their own use would eventually sell it as a product to diversify their revenue sources and reduce risk.

[Meni Rosenfeld]

That a "cartel" of miners who collectively do more than 50% of the mining can simply reject blocks that are too generous in terms of low fees.  They could effectively make it impossible for miners to include transactions on terms they didn't like, because if they did, they would find their blocks quickly undone.

Wow, that's a nightmare scenario. We know that someone with >50% capacity can damage Bitcoin by reversing transactions, but I never imagined this hostile agent would be the supposedly "normal" miners. To prevent this from happening it's important to make sure that most of the capacity will belong to pools, who need to answer to their participants and thus less likely to carry such an attack.

[Stefan Thomas]

I'm kind of relieved to see more people agree we do have a problem.

Just to be perfectly clear: I don't think it's a problem. If it ever becomes a problem, client developers will get together and figure out a reasonable block size limit.

There is no point in trying to preempt the problem now, because we can predict neither what the hash rate will be, nor what it needs to be.

Also note the timescales - the problem will be foreseeable well in advance because the transition from minting to fees is gradual.

[FreeMoney]

I'm kind of relieved to see more people agree we do have a problem.

Just to be perfectly clear: I don't think it's a problem. If it ever becomes a problem, client developers will get together and figure out a reasonable block size limit.

There is no point in trying to preempt the problem now, because we can predict neither what the hash rate will be, nor what it needs to be.

Also note the timescales - the problem will be foreseeable well in advance because the transition from minting to fees is gradual.

I agree, but the same applies for difficulty. We don't and can't know what it should be so it self adjusts, would be cool if max block size could do something like that.

[MoonShadow]

would be cool if max block size could do something like that.

There is probably a way to do this.  What if the client were to track the average number of blocks that a fee paying and free transactions were to persist in it's own queue, and increase the blocksize limit by 25% when it adjusts difficulty if the average wait blocks for fee paying transactions were over 6, and the free transaction section limit if free transactions averaged over 30.  And a client wouldn't reject an oversized block if it's own calculations put the averages over half of those numbers. 

I just pulled those numbers out of my rear, so I'm not suggesting they are ideal, just throwing out an idea.  If transaction congestion became a problem, this metric could stretch the blocksize to accommodate without undermining the scarcity value of the blocksize limit, and it would take months to grow significantly.

Likewise, if the average block wait for a fee paying transaction were under 2, or the average wait for free transactions were under 10, those same size metrics could be reduced by the same factor until they returned to the current limits.

[casascius]

There is probably a way to do this.  What if the client were to track the average number of blocks that a fee paying and free transactions were to persist in it's own queue, and increase the blocksize limit by 25% when it adjusts difficulty if the average wait blocks for fee paying transactions were over 6, and the free transaction section limit if free transactions averaged over 30.  And a client wouldn't reject an oversized block if it's own calculations put the averages over half of those numbers. 

I just pulled those numbers out of my rear, so I'm not suggesting they are ideal, just throwing out an idea.  If transaction congestion became a problem, this metric could stretch the blocksize to accommodate without undermining the scarcity value of the blocksize limit, and it would take months to grow significantly.

Likewise, if the average block wait for a fee paying transaction were under 2, or the average wait for free transactions were under 10, those same size metrics could be reduced by the same factor until they returned to the current limits.

How about if hardcoded max block size were removed completely (other than to prevent obvious gigantic spam), but then options were added into the client so that other miners (the human operator) can put in parameters to decide whether to accept new blocks into the chain or attempt to reject them.  That way all of the miners can sort it out amongst each other and adjust these parameters collectively as needed, allowing the 50% rule to enforce a consensus.  Those deviating from the agreed upon norms will find it hard to have their blocks accepted.

Settings to enforce on other miners' blocks might include: maximum block size, maximum free transactions KB, minimum transaction fee %, minimum transaction fee (absolute).

[ByteCoin]

That a "cartel" of miners who collectively do more than 50% of the mining can simply reject blocks that are too generous in terms of low fees.  They could effectively make it impossible for miners to include transactions on terms they didn't like, because if they did, they would find their blocks quickly undone.

Wow, that's a nightmare scenario. We know that someone with >50% capacity can damage Bitcoin by reversing transactions, but I never imagined this hostile agent would be the supposedly "normal" miners. To prevent this from happening it's important to make sure that most of the capacity will belong to pools, who need to answer to their participants and thus less likely to carry such an attack.

It's actually worse than you think. See my posts in this thread http://bitcointalk.org/index.php?topic=2227.0

I did a lot of simulation and I didn't post the results because they were complex and required a lot of explanation. The basic result is that a miner or group of miners with >40% of the hashing power can mine more than their fair share of blocks under very reasonable expectations.
Someone with 46% of the hashing power can win 51% of the blocks.

I seem to remember that by tweaking the strategy, "hostile mining" can be profitable at 33% of the hashing power. In all these cases the overall difficulty goes down considerably as many blocks (half?) are wasted both by the hostile miners and the normal miners they are attacking.

I'm unhappy with the current effectively GPU-only mining scheme and the popularity of mining pools as I believe that they provide a suitable environment for "hostile mining" to start. Fortunately, chronic hostile mining seems difficult to conceal.

If there were occasional transactions with huge fees then there would be an incentive for miners to not accept a competitor's block which claimed the fees but to try and generate their own block which claims the fees for themselves and then generate a subsequent block that ensures that their chain is the longest. This is "hostile mining" in a nutshell.

ByteCoin

[Meni Rosenfeld]

I did a lot of simulation and I didn't post the results because they were complex and required a lot of explanation. The basic result is that a miner or group of miners with >40% of the hashing power can mine more than their fair share of blocks under very reasonable expectations.
Someone with 46% of the hashing power can win 51% of the blocks.

Interesting. But the fact that 46%->51% is not nearly as disastrous as the fact that 51%->100%.

I'm unhappy with the current effectively GPU-only mining scheme and the popularity of mining pools as I believe that they provide a suitable environment for "hostile mining" to start.

Lots of people have GPUs, that's not the problem. The problem is that whatever the mining substrate used, a single at-home miner can't hope to generate a block solo when there are so many other miners in the world (where mining companies are counted by multiplicity). I don't think mining pools are a problem, they're not very prone to forming a cartel.

[caveden]

I haven't managed to read the entire topic, so sorry if I'm repeating somebody else. But I saw some people worried with the need to pay higher fees in order to have more security.
Please understand that now, with this high inflation phase bitcoin is going through, every bitcoin holder is already paying a lot in the form of inflation-fee to have the network secured. The thing is that, as politicians have long found out, inflation is not easily perceived. But you do pay for it.

So, even if the fees go up, I hardly think they will ever be comparable to something like 50% annual inflation. Individually, we are probably paying much more expensively right now for network security than we will pay in the future when more people join, even if we have to pay more transactions fees.

Regarding the psychological effect - people being less inclined to join bitcoin due to the visible transaction fees annoying them more than the "invisible" inflation-fees we have now -, it could be avoided by letting the transaction fees to be payed mostly by the banks. Actually, that's how I think it will work.

[MoonShadow]

Individually, we are probably paying much more expensively right now for network security than we will pay in the future when more people join, even if we have to pay more transactions fees.

Yes, the transaction fees will be relatively light on the average consumer, because the costs of maintaining the security of the network will be spread across many more users.  But we have a ways to grow, and a couple of years of high inflation, before we get to a reasonable level of costs.

Regarding the psychological effect - people being less inclined to join bitcoin due to the visible transaction fees annoying them more than the "invisible" inflation-fees we have now -, it could be avoided by letting the transaction fees to be payed mostly by the banks. Actually, that's how I think it will work.

Well, banks and individuals transfering large sums.

[ByteCoin]

The problem is that whatever the mining substrate used, a single at-home miner can't hope to generate a block solo ...

The long term returns from mining solo with a GPU are equal to and often better than those of joining a pool. A pool gives a more constant payout; that is all.

I don't think mining pools are a problem, they're not very prone to forming a cartel.

Mining pools concentrate the administration of a lot of mining power in a few hands - those of the people running the pool. If everyone were mining individually, to form a cartel, they would have to all unite and arrange a way of communicating which blocks they would accept and which they would shun. They would also have to distribute the profits of their collusion among the participants who did not win blocks. With a mining pool, both these problems are solved. I can't imagine a plausible mining arrangement more prone to cartel formation.

I'm confident that when the reward for block generation becomes small in comparison to the fees in some blocks that we will see cartels being formed and evidence of hostile mining activity.

ByteCoin

[Meni Rosenfeld]

The problem is that whatever the mining substrate used, a single at-home miner can't hope to generate a block solo ...

The long term returns from mining solo with a GPU are equal to and often better than those of joining a pool. A pool gives a more constant payout; that is all.

You're preaching to the choir. You might be interested in my thoughts about this here.

But you don't seem to appreciate the huge variance solo mining will have in a Bitcoin world. The standard deviation of the monthly payout can easily be 1000 times the expectation. The "long term" you speak of could be hundreds of years. For all intents and purposes at-home solo mining will be futile.

I don't think mining pools are a problem, they're not very prone to forming a cartel.

Mining pools concentrate the administration of a lot of mining power in a few hands - those of the people running the pool. If everyone were mining individually, to form a cartel, they would have to all unite and arrange a way of communicating which blocks they would accept and which they would shun. They would also have to distribute the profits of their collusion among the participants who did not win blocks. With a mining pool, both these problems are solved. I can't imagine a plausible mining arrangement more prone to cartel formation.

Mining pools can be required to be transparent about the blocks they build on, so they can only do hostile mining if their participants condone it.
If the participants are mining corporations then sure, they can use a pool as an administrative tool to organize the cartel. But if the participants are individual miners, they will for the most part not condone hostility.

I'm confident that when the reward for block generation becomes small in comparison to the fees in some blocks that we will see cartels being formed and evidence of hostile mining activity.

Are you confident that this will happen if we don't find a solution, or that we won't find a solution?

[ByteCoin]

. You might be interested in my thoughts about this here.

Spotting the non-intuitive fact that fixed weight pool payouts are unfair was a good catch. Good job for working out how to price the work fairly (I haven't checked it yet!)

I agree about the large variance and your analysis. Whether or not home mining makes sense depends on the motivations of the miner though. I can imagine a home miner motivated by supporting the bitcoin network, not really bothered by the reward but perhaps wanting the very occasional pleasant surprise. Admittedly, most are driven by profit and want to see an immediate return on their "investment".

To clarify, in my previous article I was objecting to you saying "can't hope to generate a block solo". Lottery players can hope to win the lottery even though the odds are even more unlikely than solo mining.
Only if you don't participate can you not justifiably hope to win.

Mining pools can be required to be transparent about the blocks they build on, so they can only do hostile mining if their participants condone it.

Agreed, and when mining pools were being developed I posted a framework which would have forbidden certain types of fraud. I almost certain nobody has adopted it or a secure alternative.  From this apparent indifference, I surmise that pool contributors will tacitly condone hostile mining (at least initially) as it doesn't harm them and indeed possibly raises their return.

I hope you are right and people will object to behaviour that benefits them personally but is generally harmful. Unfortunately, I see this very rarely.

Are you confident that this will happen if we don't find a solution, or that we won't find a solution?

Good question. I can expound (on request) in more detail what I see are the necessary preconditions to hostile mining by cartels and what might happen incidentally in future to preclude it. The recent sharp increase in the national currency value of Bitcoins is a powerful new force behind cartel formation and attendant misbehaviour and that makes me concerned.

I'd like to answer your question the following way which is more general.
A solution is unlikely to be found until it has already become a problem and caused some damage. The solution found, although it solves the problem, is not the best solution and causes further problems in future and so on ad infinitum.

To digress:
I would argue that various Bitcoin design decisions which I have noted in many of my previous posts have caused, are causing and will cause many problems because they are successful attempts to solve other problems but they solved them in the wrong way.

Sometimes the best way to improve things is to undo certain bad or suboptimal decisions and redo it the right way rather than patching holes indefinitely. I'm concerned that the community seems to have an irrationally negative reaction to talking about beneficial incompatible changes.

ByteCoin

[Meni Rosenfeld]

But you don't seem to appreciate the huge variance solo mining will have in a post-Bitcoin world.

"Post-bitcoin" world seems to mean "after bitcoin"? Is that what you meant? Edit that bit and I will delete this bit.

Yeah, I meant "a world after the Bitcoin revolution has completed, in which Bitcoin is ubiquitous." I edited to "a Bitcoin world", though that sounds to me like "a world where Bitcoin exists", which is now.

Are you confident that this will happen if we don't find a solution, or that we won't find a solution?

Good question. I can expound (on request) in more detail what I see are the necessary preconditions to hostile mining by cartels and what might happen incidentally in future to preclude it. The recent sharp increase in the national currency value of Bitcoins is a powerful new force behind cartel formation and attendant misbehaviour and that makes me concerned.

I'd like to answer your question the following way which is more general.
A solution is unlikely to be found until it has already become a problem and caused some damage. The solution found, although it solves the problem, is not the best solution and causes further problems in future and so on ad infinitum.

To digress:
I would argue that various Bitcoin design decisions which I have noted in many of my previous posts have caused, are causing and will cause many problems because they are successful attempts to solve other problems but they solved them in the wrong way.

Sometimes the best way to improve things is to undo certain bad or suboptimal decisions and redo it the right way rather than patching holes indefinitely. I'm concerned that the community seems to have an irrationally negative reaction to talking about beneficial incompatible changes.

I can only hope that as time passes and issues relating to the present reality of Bitcoin are solved, the community will be more receptive to preventing possible future attacks.

[Dusty]

I found the thread very interesting and I hope that it will go on.

But maybe due to the limitation on my knowledge of the subject or my limits in the understanding of the English language I humbly ask to elaborate this sentence because I can't get its meaning:

with $5k, one could profit substantially on the exchanges from the "inside information" one has that the attack is going to be mounted and publicized

Thanks in advance!

[Vandroiy]

Those who agree with me, skip this post, it's ugly meta-arguing, but may be useful.

There's something I'd like to point out now. In this thread, what is the consensus? Apparently, if there'd be a simple yes/no vote, I'd be wrong. But look one step deeper; what are the exact statements and models? I'll go through the thread again, and things said in the follow-up thread.


The side denying a problem (I re-write arguments as good as I can in short form. Frankly, I see them all nonsensical, but I'm doing my best. Sorry if some are oversimplified, I have to keep them short. This list is being edited a lot due to the expected complaints about exact statements.)

• People will add fees to accelerate mining temporarily (theymos)
• Global transaction cost will be spread, generating minimum fees (gavinandresen)
• Limits will not be removed (epii, I agree this would work, though it's ugly)
• Companies will start mining due to orders from customers to secure their current transactions ([mike])
• Many miners will not accept low fees to ensure the price. [This is saying Tragedy of the Commons doesn't exist] (Bitlotto) (revoked)
• The miners will from some sort of self-regulating cartel to make sure that the fees don't go to 0 (da2ce7, BitterTea, vladimir (slightly different wording))
• Merchants will come and fix the system because they need it (gavinandresen, stillfire)
• Inclusion of previous unprocessed transactions will make inclusion expensive enough for miners to live on margins [debunked by own side] (nextnonce)
• Once minting ends it will be at a level dramatically below where it is now. (...) Either that won't be a problem, or if it does become a problem, it will be solvable. (justmoon, FreeMoney, I partially agree but would rather like to be on the safe side.)

There are more, [mike] wrote a lot of them, some of which I think are wrong and some I'd say go into the right direction... but I think the list is long enough now.


The side seeing a problem (if transfer limits are removed, thus epii is on both sides now.)

• Without limits on fees or block size, difficulty falls due to a Tragedy of the Commons. (Vandroiy, Holy-Fire, casascius, db, asdf, gim, probably epii?)

caveden agrees but also stated he doesn't see it as a big problem, I'm waiting for an answer from him to put back into the list. Sorry I added you with a wrong statement.


I know there may be something wrong in the details and I may have unconsciously twisted things to fit. Let me assure you, I seriously tried to be fair in handling both sides and merging opinions. Even if I messed with things a lot, it shouldn't be this extreme. Am I really imagining things here, failing to see the common ground of others? How good are an open source system's dynamics probably if nobody even knows them? If you start reading the actual arguments, things get worse -- because in my eyes, all but three of the upper models have been debunked, two of those three are considered ugly and the remaining means to postpone the problem.

I know I'm not using an actual argument in this post. Feels strange to do that, too. But I think it's safe to say that a solid consensus should not produce this outcome. There is no consensus. There is no generally accepted operating model for Bitcoin after minting becomes negligible. We better find one.

[bitlotto]

Ya, you can take me off the list.  I'm starting to understand what your getting at. I'm not too sure what to think now...

[BitterTea]

The tragedy of the commons is a dilemma arising from the situation in which multiple individuals, acting independently and rationally consulting their own self-interest, will ultimately deplete a shared limited resource even when it is clear that it is not in anyone's long-term interest for this to happen.

What is the shared limited resource to be depleted in the case of mining?

But I think it's safe to say that a solid consensus should not produce this outcome. There is no consensus. There is no generally accepted operating model for Bitcoin after minting becomes negligible. We better find one.

What are you talking about? This is a free market, without regulation or central planning. Bitcoin cannot work, post generation subsidy, unless it is centrally planned?