Miners that refuse to include transactions are becoming a problem

[Technomage]

Currently we have a situation where a major solo miner, most likely a botnet, is mining Bitcoin blocks without adding any transactions to them. Approximately 15% of the network hashing power is currently creating blocks that do not have any transactions in them. There is a large thread about this in the mining forum: https://bitcointalk.org/index.php?topic=67634.0

So far it seems to be unclear why this is done, with most setups it shouldn't matter from a bandwidth/cpu point of view if he adds transactions or not. Clearly he has a reason for this and it could be either that he has found a way to mine less noticeable with infected PC's by not adding transactions to blocks OR he is doing this with malicious intent, perhaps trying to hurt Bitcoin. That could prove profitable to him if he short sells BTC at the same time, especially if he is planning to add more hashing power.

At the moment he has only 15% which is not a real threat to the network but what if it rises to 30%? Then one third of all blocks refuse to include transactions. Developers should take this seriously, it's in my opinion a high priority. We should start solving this before it becomes a bigger problem, and it definitely could become a major issue.

I'm not interested in speculation of why he is doing this. All of the possibilities are bad, some are just less bad than others. I'm interested in finding a solution, some way that we can fight this. Perhaps by making it less profitable to just mine blocks without adding transactions.

[Explodicle]

At first I was thinking we could treat empty blocks as invalid, but the attacker might just add a few pre-made offline transactions to his botnet code and still keep it more secretive than a full bitcoind.

The best  solution I can think of is to slowly lower the subsidy over many years, while developing specialized hardware to outperform botnets. I remember hearing somewhere that Satoshi wanted to upgrade the transaction fee system before he left, does anyone have any further reading?

[Revalin]

I hadn't realized this problem had gotten so bad so quickly.  Clearly including transactions for fees isn't a great enough incentive.  I agree it should be addressed.

I've always thought relay nodes should take a greater role in security.  If they refuse to relay blocks that don't include at least 80% (measured by fees to prevent cheap transaction spam) of transactions they've seen since the last block it might solve it - no-transaction blocks would have very poor propagation and therefore would be highly-orphaned, giving a big economic incentive for them to play nice.

On the other hand I'm not sure how well this would behave in circumstances where there is a major network split with only a few nodes holding it together.  If the two halves of the net aren't seeing most of the other side's transactions and the couple relay nodes between are dropping both side's blocks (due to failures to include), could that worsen the problem?  The couple bridge-nodes would of course be relaying transactions across to both sides, but I'm concerned if it was compounded by other problems causing poor propagation.

[Gavin Andresen]

Not relaying "smelly" blocks is a very interesting idea.

It doesn't have to be a binary relay/don't relay... you could immediately relay good blocks but wait a while before relaying bad blocks, and make the length of time you wait to relay based on how bad you think they are (maximum of maybe 10 minutes before you relay).

But... not relaying new blocks immediately might just encourage the bad guys to try to connect directly to as many nodes as possible, and that could be bad for network health.

[DBordello]

What are the side effects of 15% of "extra" empty blocks?

1)  The difficulty increases.  This is good for the network, but bad for miners
2)  It takes longer for transactions to make it into blocks.

Am I missing something?  Neither one seems like THAT big of a deal.

[Technomage]

What are the side effects of 15% of "extra" empty blocks?

1)  The difficulty increases.  This is good for the network, but bad for miners
2)  It takes longer for transactions to make it into blocks.

Am I missing something?  Neither one seems like THAT big of a deal.

You are missing something. The problem is that 15% is now, it's not necessarily that low forever. To me it seems like not adding any transactions is advantageous to a botnet miner for some reason (which is unknown) and this needs to be countered. Imagine if in a couple of months this percentage is 30, then our network is basically one third slower than it should be. It's a big deal that will make people lose confidence in Bitcoin's reliability.

We need to find a way to either give those who add transactions more carrot for doing that or give those that don't do that more stick. Whatever the solution is, we definitely need one.

[DeathAndTaxes]

People will lose confidence if the avg 1-confirmation time is 13 minutes instead of 10?  Really?  Especially given the 90% confidence interval is 3 to 31 minutes anyways.

[theymos]

I wouldn't consider it a problem for even 75% of miners to exclude all transactions. This would only slow transactions down by 10-20 minutes, and eventually the fee mechanism would sort it out.

I am a little concerned that they're not actually verifying transactions like they should be. This reduces the percentage of network power that an attacker needs in order to execute a "50%" attack. Applying gmaxwell's proposal might be worthwhile.

[ByteCoin]

I am a little concerned that they're not actually verifying transactions like they should be. This reduces the percentage of network power that an attacker needs in order to execute a "50%" attack.

So how does that work then? If they let an invalid transaction into the block then that block will be rejected by the network and they have wasted their hashing effort.

ByteCoin

[theymos]

So how does that work then? If they let an invalid transaction into the block then that block will be rejected by the network and they have wasted their hashing effort.

If these miners are building off of the longest chain without looking at any of the transactions (which may not be the case), then any attacker capable of getting a few blocks in a row can get these dumb miners to work for him. Full clients will reject these invalid chains, of course, but lightweight clients won't.

Now that I think about it more, it's probably more likely that the miners are getting the latest hash from Bitcoin nodes or websites, which might not be quite as bad as using the longest chain. Still weakens the network a lot, though.

[Syke]

I hadn't realized this problem had gotten so bad so quickly.  Clearly including transactions for fees isn't a great enough incentive.  I agree it should be addressed.

It was addressed in the original Bitcoin design. Over time, the block reward will drop to zero, and transaction fees will make up the majority of the value of mining blocks. Nothing more needs to be done.

[DILLIGAF]

I hadn't realized this problem had gotten so bad so quickly.  Clearly including transactions for fees isn't a great enough incentive.  I agree it should be addressed.

It was addressed in the original Bitcoin design. Over time, the block reward will drop to zero, and transaction fees will make up the majority of the value of mining blocks. Nothing more needs to be done.

If it was perfection from the start then why do we need developers at all now?

[DeathAndTaxes]

Design =/= implemented code.

[Technomage]

I simply find it unacceptable that my transactions and everyone elses transactions with Bitcoin can become slower thanks to a leeching botnet that simply profits from the blocks but doesn't do the one thing mining is actually useful for which is adding transactions to the blockchain.

Do we really want to let this leeching continue without doing anything about it? This is not just about inconvenience, if there is a solution we should put a stop to this. I'm quite confident that I'm not the only one who doesn't like this.

Maybe it isn't such a big threat but it does bother me a lot for many reasons. It's the kind of thing that would make Bitcoin users leave Bitcoin and start using a competing cryptocurrency. Of course there are no real cryptocurrency competitors yet but in 5 years Bitcoin might not have this luxury. Issues like this do matter.

[DeathAndTaxes]

Mining isn't just about adding tx to the block chain.  The hashing power still adds to the network security.  The economic value from that hashing power provides the incentive to not be disruptive.  There is no economic incentive to including txs.  People being cheap isn't a flaw.

[Kluge]

I simply find it unacceptable that my transactions and everyone elses transactions with Bitcoin can become slower thanks to a leeching botnet that simply profits from the blocks but doesn't do the one thing mining is actually useful for which is adding transactions to the blockchain.

Do we really want to let this leeching continue without doing anything about it? This is not just about inconvenience, if there is a solution we should put a stop to this. I'm quite confident that I'm not the only one who doesn't like this.

Maybe it isn't such a big threat but it does bother me a lot for many reasons.

If it becomes a big enough hassle, people will stop waiting for confirmations. Once that convenience problem turns into a security problem, the devs will probably start thinking hard about fixes.

[Technomage]

Good points by both DeathAndTaxes and Kluge. I agree but still not happy about it. Perhaps the hash power of these no-tx botnets need to become a bit larger for people to really notice this issue. It could be that it doesn't even happen and we're left with this small issue that actually goes away eventually just as Syke said. But the thing is we don't really know.

[benjamindees]

I bet this company here is testing one of their fpga/asic products: http://www.sevensols.com/

It's located in Granada, Spain. That's where the ip is from.

This could explain why transactions are not included.  If this is a manufacturer doing a burn-in on an fpga cluster, omitting transactions would be a good way to avoid being accused of terrorism or whatever.

Then again the possibility of one manufacturer (and likely gov't contractor) having 15% of the hashing power is not good to think about.

[Nim]

As I understand it (and I probably understand poorly), the miners redo the transaction calculations only every few minutes. So, even if a miner is good intentioned and creating blocks with valid transactions, a transaction in the last part of the 10 minutes might be missed and will have to wait untill the next block. Assuming I have all that correct, could we not have a validation system where the node keeps track of the transactions and if a block is announced by a different node, it would check to see if transactions of a certain age are present and if not, reject the block? For instance, time 0 is solve time of last block and time 10 is solve time for current block. I keep track of transactions and if I see a new block at time 10 that doesn't include a transaction from time 5 (or before), I reject it. Is that doable or would we get into timestamp issues with not everybody agreeing on the time?

[grue]

As I understand it (and I probably understand poorly), the miners redo the transaction calculations only every few minutes. So, even if a miner is good intentioned and creating blocks with valid transactions, a transaction in the last part of the 10 minutes might be missed and will have to wait untill the next block. Assuming I have all that correct, could we not have a validation system where the node keeps track of the transactions and if a block is announced by a different node, it would check to see if transactions of a certain age are present and if not, reject the block? For instance, time 0 is solve time of last block and time 10 is solve time for current block. I keep track of transactions and if I see a new block at time 10 that doesn't include a transaction from time 5 (or before), I reject it. Is that doable or would we get into timestamp issues with not everybody agreeing on the time?

discussed a short while ago, i think it was shot down because it can easily cause forks.