Miners that refuse to include transactions are becoming a problem

[DeathAndTaxes]

This seems to solve the problem by setting a reasonable limit on miners' ability to extort the network for fees.  We really should consider a somewhat drastic solution.  Even though the mystery miner is likely a botnet, within a year or so we could easily see this same scenario play out with a large FPGA cluster that doesn't want to include transactions for whatever reason.  It isn't a problem that is going to just go away, or be solved any time soon by falling block rewards.  We're already subsidizing miners with 30% inflation;  it might as well go to those that at least process a few transactions.

If the top 10 transactions are 1 satoshi then that forces miners to include all tx with at least a 1 satoshi fee.  It creates a race to the bottom.  Smart users will keep fees as low as possible and encourage others users to do so.  "Look as long as nobody pays too much the miners are forced to include are freeloader txs".

Your insulting language is counterproductive.  You are aware that passing any breaking change on block validity rules will require support of a majority of us "greedy miners".  Also money supply growth isn't 30% anymore and after the subsidy drop it will be closer to 10%.  By the next subsidy break it will be ~4%.  Users have enjoyed price deflation due to the fact that the demand for currency has grown faster than the money supply.

[1715135455]

1715135455

[1715135455]

1715135455

[1715135455]

1715135455

[benjamindees]

Okay I was only suspicious before, but now I'm nearly convinced that this issue has done much more to harm Bitcoin by bringing to the fore the absolute lunacy of some miners.

Some of you seem completely oblivious to the fact that nearly every Bitcoin you "earn" comes at the expense of the scant few actual producers and investors and participants in the real economy.  Half of you could fall off the face of the earth and the Bitcoin network would be no worse off.  In fact, the only reason this hasn't happened is because you are being subsidized by the non-negotiable inflation built into the protocol.  The abundance of zero-fee transactions should give you a hint as to the marginal market demand for mining services.

"Price deflation," give me a break.  That is entirely paid for by the few businesses willing to accept Bitcoins as payment for real goods and services.  And for the amount of time that Bitcoin has been in widespread public view, that number is dangerously low.  Accepting the premise that miners can earn rewards without processing a single transaction not only doesn't encourage new participants, it screams "stay away."  You are shooting yourselves in the feet by doing so.

For the record, I didn't call you "greedy".  I called you "short-sighted" and "extortionist".  If you were greedy, you would at least act in your own best interests.  Many of you can't even seem to do that.  Take a step back, do a little math and think about how many transaction fees you will be earning a year from now, if the average block still has only 50 transactions in it.  Then ask yourself why you are here arguing for higher transaction fees, which are ultimately insignificant compared to the block reward, and which will only serve to discourage growth in transactions between now and when the block reward goes away.

Did it never occur to you that the uncertainty caused by having 15% of the network not processing transactions is already hurting your profitability more than accepting a solution ever would?

Come on, get real.  What is the possible market for international transfers in an illiquid digital currency nobody uses?  Do you seriously think Bitcoin does anything that Western Union can't?  Do you think anyone will pay a premium to use a payment service that isn't even denominated in their local currency?  How long do you think Bitcoin will last if the only people using it are criminals and drug dealers willing to pay exorbitant transaction fees?

You haven't monopolized the secret directions to El Dorado like you seem to think you have.  You have simply drank the Kool-Aid, and are going to be completely blindsided when this desperate grasping for every last BTC ends up backfiring spectacularly...

[Maged]

So the rule could be this simple and this modest: If there are at least four transactions that are not free and have at least the standard fee that were not included in the previous block even though they could have been, discourage any block with just a coinbase transaction.

You also need to discourage the block if it has only transactions you don't know about, since otherwise the miner could make up transactions.

Actually... I think we should re-think this. Why not just let them make up transactions? It seems that if they include any transactions (maybe needing to add up to a minimum amount) that this specific problem is solved.

To make up a transaction, the following must be true:
1) You hold the private keys for the transaction or have a method of getting a transaction signed. (this would be death to 100% decentralized botnets (centralized botnets would just make a mining pool))
2) You know that the transaction isn't a double-spend. (so a decentralized botnet can't give the same private keys/pre-signed transactions out more than once without each computer holding a blockchain)

Honestly, I can't believe that this didn't occur to anyone else, and that we've just kept saying "the miner could make up transactions" to dismiss such an effective idea. By requiring at least one (reasonably sized) transaction that isn't confined to the block itself, we finally have a link to previous transactions that MUST be verified. If it isn't verified, your chances of creating invalid blocks rises significantly.

[westkybitcoins]

2: He isn't actually contributing to security, since he's only generating blocks off the previous hash without validating anything. That means if an attacker starts throwing down invalid blocks, our friendly botnet might blindly build right on top of them and give them a nice helping hand.

Not true. All the building on top of an invalid block can't hide or change the fact that it's invalid and so would do no harm whatsoever. It would simply be wasted effort.

These botnets do increase the amount of hashing power an attacker would need to launch something like a 51% attack. They do increase the security of the network. And that's what the block reward is for. If the system is broken, it's because the transaction fee and volume aren't sufficient to justify the expense of including transactions in a block.

Another way to look at that: the reduction in the mandatory fee for large transactions was reduced too much, too quickly within the main client. While I personally pay those fees often enough, it hardly amounts to much. At the very least, the mandatory fee shouldn't be reduced again anytime soon (a $100 price for bitcoins would still make the 0.0005btc fee be only worth $0.05. At current prices I pay one-quarter of one cent per fee.)

Similarly, what is the default optional fee that the latest version of the standard client is set to upon install? If it's less than 0.01btc (currently $0.05 per tx,) perhaps raising this default optional fee to 0.01btc in the next release would be a good start at addressing the long-term low-fee problem, since it wouldn't change any aspect of the code or protocol, and could be reduced by users as needed.

Even if it defaults to 0.01btc already, maybe an additional notice could be displayed whenever the user reduces the optional fee, including text such as "Lowering this fee, while possible, is strongly discouraged.")

I can't see any practical objections to at least starting with that.

[DeathAndTaxes]

Come on, get real.  What is the possible market for international transfers in an illiquid digital currency nobody uses?  Do you seriously think Bitcoin does anything that Western Union can't?  Do you think anyone will pay a premium to use a payment service that isn't even denominated in their local currency?  How long do you think Bitcoin will last if the only people using it are criminals and drug dealers willing to pay exorbitant transaction fees?

Where do you get this garbage about exorbitant transaction fees?  A bit penny is exorbitant?  In what universe?

If you think the value and utility of Bitcoin is so useless why are you even here?

You haven't monopolized the secret directions to El Dorado like you seem to think you have.  You have simply drank the Kool-Aid, and are going to be completely blindsided when this desperate grasping for every last BTC ends up backfiring spectacularly...

Srawman aside I never claimed I had.   You seem to have a very deep seated hatred of miners.

[westkybitcoins]

Okay I was only suspicious before, but now I'm nearly convinced that this issue has done much more to harm Bitcoin by bringing to the fore the absolute lunacy of some miners.

<snip>

Come on, get real.  What is the possible market for international transfers in an illiquid digital currency nobody uses?  Do you seriously think Bitcoin does anything that Western Union can't?  Do you think anyone will pay a premium to use a payment service that isn't even denominated in their local currency?  How long do you think Bitcoin will last if the only people using it are criminals and drug dealers willing to pay exorbitant transaction fees?

You haven't monopolized the secret directions to El Dorado like you seem to think you have.  You have simply drank the Kool-Aid, and are going to be completely blindsided when this desperate grasping for every last BTC ends up backfiring spectacularly...

You seriously think, at current prices, that a miner charging a 0.01btc per-tx fee is exorbitant? And in the same context as comparing Bitcoin to Western Union, no less?

While I can see how many would think that's a bit much (as I do,) I hardly find that to be anything close to "lunacy." In fact, such a strong reaction to the idea has me wondering who exactly it is that's desperately grasping for every last BTC they can get at.

[theymos]

Actually... I think we should re-think this. Why not just let them make up transactions? It seems that if they include any transactions (maybe needing to add up to a minimum amount) that this specific problem is solved.

That'd make things a little more difficult for botnet operators, but it can be bypassed. The botnet software could listen for blocks and transactions on the Bitcoin network and include them without checking them or storing them. The resulting blocks will usually be valid, since legitimate nodes don't relay bad transactions. The botnet could even "check" each transaction by seeing if its other peers either already have the new transaction or will accept it.

I just realized that this attack also applies to other proposals requiring some amount of transactions from the memory pool to be in the next block.

[Jon]

They aren't becoming a problem. They are simply making transaction processing more valuable. When the incentive comes for them to process transactions, will be the day a higher fee is set across the board.

I see no issue. If the problem becomes bad enough, people will talk with their pocketbook.

Although, you could always coerce the miners into giving you free transactions. As for the consequences to Bitcoin as a whole in such a case, I'll leave that to you to figure out.

I don't care what the means are. They have the direct ability and financial incentive to influence the dev team to their whims -- and vice-versa. The 30%+ hash rate is a dangerous majority and needs to be lowered to less than 10%.

I WILL NOT REST UNTIL DEEPBIT IS A SUBJECT TO THE WHIMS OF ALL BITCOIN SHAREHOLDERS.

Bitcoin shareholders, your assets are in the hands of this monstrosity. Do you want that to change?

Join me. Join the cause of getting all miners on P2Pool and away from these power-wielding parents we call pools.

Shut up, Atlas.

Spell out the contradiction, friend.

[westkybitcoins]

Come on, get real.  What is the possible market for international transfers in an illiquid digital currency nobody uses?  Do you seriously think Bitcoin does anything that Western Union can't?  Do you think anyone will pay a premium to use a payment service that isn't even denominated in their local currency?  How long do you think Bitcoin will last if the only people using it are criminals and drug dealers willing to pay exorbitant transaction fees?

Where do you get this garbage about exorbitant transaction fees?  A bit penny is exorbitant?  In what universe?

If you think the value and utility of Bitcoin is so useless why are you even here?

I've found myself asking this more than I would have expected lately.

Although to be fair, when you have an entire mining pool willing to use it's resources to attack any new alt-coin that springs up (in similar fashion to this, I note,) then it's not as if people are very free to just create their own alternatives and go run with them... they're pretty much stuck with Bitcoin if they like the idea of a cryptocurrency but want changes to it.

The law of unintended consequences rears its ugly head again.

[DeathAndTaxes]

An alt-coin which uses a different PoW (or PoS) would be immune to that particular attack.

So would any alt-coin that wasn't a dump and pump piece of junk.  You know one that actually submitted a proposal, took in feedback, had a launch date announced well in advance, had a working testnet, got support of multiple pools and miners to have sufficient strength on day one.

Of course no alt-coin has shown any innovation, any progress.  They are wothless pump and dump scams. 

[westkybitcoins]

An alt-coin which uses a different PoW (or PoS) would be immune to that particular attack.

So would any alt-coin that wasn't a dump and pump piece of junk.  You know one that actually submitted a proposal, took in feedback, had a launch date announced well in advance, had a working testnet, got support of multiple pools and miners to have sufficient strength on day one.

Of course no alt-coin has shown any innovation, any progress.  They are wothless pump and dump scams. 

My point still holds. Alt-coins being attacked by legitimate bitcoin pools that take it upon themselves to police the world of cryptocurrencies has a chilling effect on dissatisfied bitcoin users who might otherwise have tried to create a non-pump-and-dump alt-coin.

I personally don't need to be saved from the scammy alt-coins. I'd rather be saved from all those trying to change bitcoin into their vision of an alt-coin.

[Technomage]

Even though benjamindees seems to be on a warpath against the miners, he has some good points. This issue, even though it's not critical, is one more thing that makes Bitcoin suspicious overall. I'm one of the biggest Bitcoin-enthusiasts out there and even I don't like this at all. I can't explain something away to other people if I'm bothered by it as well. In addition I'd like to point out some important aspects to this.

First of all, some of you claim that it's just a matter of finding a tx fee high enough to incentivize this mystery miner to start adding transactions. I think this is entirely false. From what some people have speculated, the benefits he could potentially gain from not adding any transactions, or not verifying transactions at all, can be sort of huge for him. There is a possibility that he simply doesn't care what the tx fees are in the transactions because he doesn't even look at them.

This means that we could have a tx fee of gazillion and a gazillion people paying those tx fees, but he still wouldn't care, because it would possibly lead to many of his miners to be discovered and thus his whole mining operation would be threatened. It could certainly be that Bitcoin needs to become the most expensive money transfer method in the world for him to care, and then it wouldn't even matter anymore. Who would care about Bitcoin if it was that expensive?

So not doing anything is basically ignoring the issue and hoping it goes away or doesn't become too bad. My other point is somewhat related to this, I'm starting to feel that there is an ideological conflict going on right here. We have people who want to desperately hold on to a pure market approach even when we have a clear exploit that allows miners to leech on the network. Then we have people who think that some restrictions are appropriate to stop the leeching.

Then there is this notion that the fixed block rewards are for creating blocks and tx fees are for adding transactions, this has to be the most ridiculous idea I've seen in this thread so far. Who came up with this? I'd like to see Satoshi saying that, I have a hard time imagining it though. Bitcoin is a cryptocurrency and for a currency being able to transact is critical. Ín the case of Bitcoin it's critical to be able to transfer money cheaply, that is one of Bitcoin's major benefits.

Again, I'm not saying that transactions should be free nor that tx fees should be dictated in a socialistic way, but being able to intentionally not work with transactions at all and still get a 50 BTC reward is one hell of an exploit that is likely not going away with the block reward drop.

So we will at least have to put up with this for 5 years, possibly much longer. This is because if Bitcoin continues to grow, bitcoins will certainly be more valuable in the future. That will make sure that nothing really changes as far as mining is concerned, our network just gets a bit less secure relative to the market cap. From the perspective of our mystery miner, nothing changes.

Finally, I think that this issue is not big enough to put a solution in place that either heavily restricts a future tx fee market or has other major side-effects. For example I feel that jgarzik's proposal was too drastic and now it seems, at least according to theymos, that basically all of the recent proposals are problematic. I don't support doing anything hasty about this issue so let's just continue to think it through.

[ribuck]

So we will at least have to put up with this for 5 years.

No. The path forwards is to discover how this is being done, so that people whose computers are compromised can get rid of the malware.

Meanwhile, empty blocks are a minor irritation but do not threaten the foundations of Bitcoin.

Here are Satoshi's thoughts on the matter from the early days of Bitcoin:

There would be many smaller zombie farms that are not big enough to overpower
the network, and they could still make money by generating bitcoins.  The
smaller farms are then the "honest nodes".  (I need a better term than
"honest")  The more smaller farms resort to generating bitcoins, the higher the
bar gets to overpower the network, making larger farms also too small to
overpower it so that they may as well generate bitcoins too.  According to the
"long tail" theory, the small, medium and merely large farms put together
should add up to a lot more than the biggest zombie farm ...

The Bitcoin network might actually reduce spam by diverting zombie farms to
generating bitcoins instead.

[benjamindees]

I really don't understand how some of you simply ignore the fact that transaction fees are irrelevant in comparison to block reward, and will be for the forseeable future.  Even if they weren't, there are plenty of people out there with the incentive to disrupt the Bitcoin network by mining without including transactions, regardless of what the fees are.

Nothing you have suggested so far changes that.

[DeathAndTaxes]

I really don't understand how some of you simply ignore the fact that transaction fees are irrelevant in comparison to block reward, and will be for the forseeable future.  Even if they weren't, there are plenty of people out there with the incentive to disrupt the Bitcoin network by mining without including transactions, regardless of what the fees are.

Nothing you have suggested so far changes that.

Not sure what your definition for "forseable future" is.  As long as tx fees are essentially 0 so no matter what the block reward is ~0/anything is still essentially 0%.

Even a modest rise in fees can provide meaningful change to that dynamic.  Hypothetical scenario.  Over next 12 months tx volume doubles to ~100 per block and block reward falls to 25 BTC.  If miners start enforcing fees and avg fee ends up being just 0.01 BTC per tx that is 1 BTC in fees and 25 BTC reward.  Fees are a small but hardly insignificant 4% of total revenue in just 12 months.

Hopefully on a longer time frame say 3 or 4 years tx volume will quintuple (or more).  At say 250 tx per block and 12.5 BTC subsidy.  An avg fee of just 0.01 BTC per tx results in fees making up 16% of total revenue.

If USD:BTC remains the same then mystery faces falling revenue and will need to adapt (or make less).  If USD:BTC prices then global hashing power will also rise as miners chase larger block rewards (including fees) and "mystery" makes up a smaller portion of the network.

Going forward bitcoin will need to be supported more and more by fees anyways.  Any "problem" myster presents is temporary at best.  Breaking changes to protocol to "fix" a temporary problem seem dubious.

[muyuu]

It's a simple fact that any extra incentive to mining including transactions will make mining more profitable to those who do, and will push difficulty up, directly reducing the power of ANY individual entity.

Simple logic really: more reward to mining including transactions => more proper mining, more difficulty, less baseline reward for everybody.

Whatever else is done, this step sounds trivial and it's also fair. It also reduces uncertainty about the reactions and instability that block reward halving might cause.

[Technomage]

This is off topic, but I honestly don't think that the first block reward drop will change things that much. It will make tx fees slightly more relevant but still not very relevant. If you presume that Bitcoin usage is going to grow in the coming years, then Bitcoin price will most likely more than double as well. Also the smaller inflation will probably increase appreciation in Bitcoin's value. That will basically compensate rewards in a way that the total miner reward in fiat is likely to actually increase significantly per block, which will mean more mining investments, more competition and a more secure network in absolute terms (although weaker in relative terms).

In relative terms the network is bound to get much weaker than it is now but that is actually fine, at least for a long while. The security level in relative terms is overkill right now and in absolute terms it can only get better if Bitcoin continues to grow. Eventually we'll reach a point where tx fees have a significant role and then we have an interesting situation where we basically switch to a tx fee market system that works or if that fails, the only other solution to providing a safe enough network is a PoS/PoW hybrid.

Now back to the topic. I have to agree that it's possible that this problem could get less significant in the future and simply go away eventually but we can't know for sure. It could also get worse if more botnets start to apply similar methods. I think it's healthy to think about possible solutions if this gets worse. It's bound to get better eventually just like you guys are saying but the issue is how bad does it have to go in the short term to be unacceptable.

My problem with this hasn't been the practical issues of it, it doesn't increase average tx confirmation time by much and that angle won't change significantly even if the percentage of no-tx hash power increases. The problem is that the mystery miner entity is partially leeching on Bitcoin and I don't like it, and apparently there are a lot of people like me. But it could be that trying to apply a quick solution to this could be problematic in the long term. The best solution to this would be if we could somehow expose the botnet (if it indeed is a botnet) and stop it that way. That is very difficult as well though.

[benjamindees]

They aren't becoming a problem. They are simply making transaction processing more valuable.

This is the broken window fallacy, by the way, Atlas.

[wogaut]

This is the broken window fallacy, by the way, Atlas.

Agree.

[sturle]

So miners are required to include tx that don't meet their fee requirements?  If they don't they risk having the valid blocks not forwarded?

Perhaps the mystery miner includes transactions which meets his fee requirements as well?  He's just set a high fee requirement.  Try a transaction with 50 BTC fee, and check if he includes it.

The point of the proposed change is to reward responsible miners who actually contribute to the network with more blocks.  Satoshi suggested a part of each block to be free, and miners who believe in Bitcoin as a currency include those free transactions.  If Bitcoin transactions get more expensive than PayPal transaction (which you actually suggest for small transactions later in this thread), then people will use PayPal instead, making Bitcoin less valuable for everyone.  It is not in the interest of most miners.