Miners that refuse to include transactions are becoming a problem

[guruvan]

Miners requiring higher fees is a simple strategy to try first.

I'm showing ignorance here, but do all miners need to accept/require the same fees? Or could one pool take lower fees than another? (thus upping it's own TX volume, and potentially earning more in fees)

[1714078523]

1714078523

[1714078523]

1714078523

[1714078523]

1714078523

[1714078523]

1714078523

[bitlotto]

Miners requiring higher fees is a simple strategy to try first.

I'm showing ignorance here, but do all miners need to accept/require the same fees? Or could one pool take lower fees than another? (thus upping it's own TX volume, and potentially earning more in fees)

Miners can decide however they want, what tx's to include.
The client software on the other hand can decide what tx's are broadcast through the network.

[Technomage]

To me it's clear that a solution that makes it impossible, unpractical or at the very least clearly unprofitable to refuse to add any transactions is VERY MUCH preferable to a "solution" that simply hopes it becomes unprofitable enough for the mystery miner to some day start adding transactions.

This whole thing is an absolute outrage. Miners get their 50 BTC reward because they maintain the network and maintaining the Bitcoin network is not just about providing security (which the extra hashing power surely does) but making sure that Bitcoin transactions are actually included in the blocks. Miners who don't do this are partially leeching on Bitcoin and that should NOT BE TOLERATED.

I would accept a solution based on nothing but higher tx fees ONLY if the stick based solutions have severe side-effects. Other than that I would basically accept any stick-solution, whatever the hell it is. The main problem here is less the practical delays that 15% of no-tx blocks cause (which are undoubtedly small), than simply the fact that it's an outrage that it's even possible for someone to mine like this when transactions are clearly such an important part of Bitcoin.

This whole thing should not be confused with the issue of creating a tx fee market in the future. As block rewards go lower, that will certainly be necessary if we continue to use proof of work. Tx fees will start playing a bigger part in miner rewards, that much is obvious. And the tx fee system needs to change from what it is now. It is however a separate issue.

[DeathAndTaxes]

I don't see the outrage.

As far as forcing miners to include all tx does that include free ones? 
Does that include ones with a token fee (say 1 satoshi)?
Why as a miner should I be FORCED to include free loader transactions?

[Haplo]

The solution I proposed doesn't force anyone to accept any given tx, however it does require that they include valid tx data, rather than just ignoring the blockchain. If a mining rig is already accounting for the blockchain, then it would be nonsensical for them to exclude every tx, especially if some of the tx going through their network include substantial fees. There's a big difference between a miner getting few tx because of high fees and one producing token 1tx blocks just trying to look valid. The solution I proposed would make it dead obvious if any miner wasn't using the blockchain, so those blocks would be thrown out completely and the coin they generated would be invalidated. No more carrot, no more problem.

Of course, it would be nice if mystery was actually contributing, but it might be more difficult to implement the means for that to happen, and there's no guarantee that he/they will adopt it even then. Kicking them out would definitely work, although with less potential upside.

So basically, unless validating the tx hashes of blocks is ungodly more complicated than rigging remote serving of the blockchain, then the stick is the more viable and assured option.

[DeathAndTaxes]

A I understand it (possibly wrong) your solution is that miner must include in the block a hash of the tx of the prior block.  The issue is that doesn't accomplish anything.  A few distributed servers (or nodes, or calls to blockchain.info) could provide that data via remote call.  A miner could easily then have nodes process blank blocks with valid signatures.

Of course you aren't the only one with proposed solutions.  At least two people have proposed systems which invalidate blocks that don't contain a certain amount of tx.  Satoshi clearly believed miners should be able to set whatever fee requirement they want.

[Haplo]

A I understand it (possibly wrong) your solution is that miner must include in the block a hash tx of the prior block.  The issue is that doesn't accomplish anything.  A few distributed servers could provide that data via remote call.  A miner could easily then have nodes process blank blocks with valid prior hash.

Of course you aren't the only one with proposed solutions.  At least two people have proposed systems which invalidate blocks that don't contain a certain amount of tx.  Satoshi clearly believed miners should be able to set whatever fee requirement they want.

I'm afraid I don't know that much specifics about how much of the blockchain is actually required in order to validly mine tx. However, gmaxwell's system uses the input data from the previous block's tx, the input data from the tx currently being mined, and the output data, which is added to the current coinbase in order to create a sort of "signature". If the rig isn't validly mining tx, then it wouldn't be able to include valid tx data from the tx it's mining, nor a valid signature.

If they can get around that by using a few servers for distribution, then it seems to me they should also be able to validly mine tx. Problem solved.

However, after re-reading this a few times it seems to me that the primary reason mystery does not do this (or use a mining pool) is because his computing power is probably stolen, and anything that could be used to trace him would put him up shit creek. In that case, the only way to deal with it is to give him the stick. If he tries to get around it, he makes himself traceable, game over.

[DeathAndTaxes]

How would he be traceable.  He can use a few of the nodes to provide validation hashes for the rest of the nodes.  Obviously every block requires the prior blocks hash so he must already be distributing that anyways.  He has to have at least one copy of the bitcoind running somewhere as he needs to be aware of other blocks generated and the current dominant chain.  Requiring "current" tx is nonsensical.  If there are no tx in the block then that adds nothing to what miner needs.  Unless one intends to force miners to include all tx that is.

If the rig isn't validly mining tx, then it wouldn't be able to include valid tx data from the tx it's mining, nor a valid signature.

But he IS validly mining.  Imagine I wanted to run a pool which requires a 1 BTC tx fee.  Currently in last 7 days guess how many tx (other than the coinbase) would be in my pool's blocks.  ..... 0.  

So would I be "invalid mining" (or whatever nonsense term you want to make up)?  The reality is no tx meet my fee requirements.  It doesn't matter than you or others might think my fee conditions are stupid.  Satoshi intended an open fee market.    If I feel my computing time is worth 1 BTC per fee that is my right as a miner.

[theymos]

A I understand it (possibly wrong) your solution is that miner must include in the block a hash of the tx of the prior block.  The issue is that doesn't accomplish anything.  A few distributed servers (or nodes, or calls to blockchain.info) could provide that data via remote call.  A miner could easily then have nodes process blank blocks with valid signatures.

Under gmaxwell's proposal, the miner would need to include exactly the transactions specified by the person who verified that the transactions and the previous block were valid. So if blockchain.info published a set of 10 verified transactions, the miner would have to include all 10 of them.

Satoshi intended an open fee market.    If I feel my computing time is worth 1 BTC per fee that is my right as a miner.

If necessary I can write code to make nodes prefer not to use a block if it doesn't contain enough of the transactions they know about.  A discouraged block would almost always fail to be included in the main chain, but would be accepted if it did get in.  I doubt this will be necessary, since there's no real advantage for nodes not to include all transactions.

[VVS dump]

well, just to add my 2 cents. If it's a botnet, then it'd need a lot of bandwith and controlling processing power. Someone in the thread about  this in the mining section gave 1.8m clients as an average number of estimated clients. If I had a botnet of this size, I'd mod bitcoind to just have the coinbase for getwork, and the clients listen for block changes(could be done via IRC). that way, bots would only "getwork" when a new block comes out, which reduces the amount of resources needed to serve the botnet amssively.

[Haplo]

But he IS validly mining.  Imagine I wanted to run a pool which requires a 1 BTC tx fee.  Currently in last 7 days guess how many tx (other than the coinbase) would be in my pool's blocks.  ..... 0.  

So would I be "invalid mining" (or whatever nonsense term you want to make up)?  The reality is no tx meet my fee requirements.  It doesn't matter than you or others might think my fee conditions are stupid.  Satoshi intended an open fee market.    If I feel my computing time is worth 1 BTC per fee that is my right as a miner.

Well, you wouldn't be doing very good business if you actually had to depend on tx fees, which is mostly the point. Admittedly it's a difficult problem to define an acceptable threshold, but if push comes to shove it could be done. However, unless a lot of miners start purposefully killing their tx throughput that shouldn't be an issue at all. If mystery really is running sans a blockchain, which seems to be the case, then only he would be affected since the stingy miner can still produce the minimum verification, whereas mystery could not.

If mystery finds a way to run with the blockchain with his stolen computing power, then he could continue to do business, but would be running like a normal miner, problem solved. Given his current cushy 16% market share, however, nothing short of chasing him out would motivate him to go to that kind of trouble. Downloading a 2GB file onto thousands millions of computers you don't own without being noticed would be nothing short of a miracle, and not our problem, for that matter.

well, just to add my 2 cents. If it's a botnet, then it'd need a lot of bandwith and controlling processing power. Someone in the thread about  this in the mining section gave 1.8m clients as an average number of estimated clients. If I had a botnet of this size, I'd mod bitcoind to just have the coinbase for getwork, and the clients listen for block changes(could be done via IRC). that way, bots would only "getwork" when a new block comes out, which reduces the amount of resources needed to serve the botnet amssively.

That's a lot of freaking clients. And also, that's probably exactly what's going on, or something very close. I'm glad someone knows how this crap works .

[DeathAndTaxes]

Well, you wouldn't be doing very good business if you actually had to depend on tx fees, which is mostly the point. Admittedly it's a difficult problem to define an acceptable threshold, but if push comes to shove it could be done. However, unless a lot of miners start purposefully killing their tx throughput that shouldn't be an issue at all. If mystery really is running sans a blockchain, which seems to be the case, then only he would be affected since the stingy miner can still produce the minimum verification, whereas mystery could not.

But it is my choice right?  Or is the choice on what is an acceptable fee decided by the cental agency of bitcoin fees? I have a right to charge whatever amount I want for my hashing power.  Do you agree?  As the subsidies decline the market will be more competitive but it is still my choice.

If tx MUST be included then does that include all free tx.  Am I now forced to include all free tx?  What incentive is there for a user to pay any fee then?

Does it include all tx w/ a fee?  If so then why every pay more than 1 satoshi?
Does it include all tx w/ a "minimum defined" fee?  Say 0.0005 BTC.  If so then why would any user pay more?

If someone says you must include 25% of all pending transactions... well what if those transactions are below my processing cost (because the users are cheap bastards)?  I am forced to include them or lose the block?  Really?

[DeathAndTaxes]

Under gmaxwell's proposal, the miner would need to include exactly the transactions specified by the person who verified that the transactions and the previous block were valid. So if blockchain.info published a set of 10 verified transactions, the miner would have to include all 10 of them.

Specified by whom?  The person who solved the last block?  If so why aren't the transactions in their block?

[theymos]

Specified by whom?  The person who solved the last block?  If so why aren't the transactions in their block?

Anyone could publish lists of transactions that should be included in the next block along with the necessary "verification proof". Bitcoin nodes themselves could publish lists using a new network message.

[DeathAndTaxes]

When there are conflicting lists?
When the list contains free tx and I as a miner don't want to include them?

Wouldn't it be in the best interest of free-loader nodes to publish and follow lists which put all free tx into the required list thus ensure free tx on the backs of miners?

It comes down to a fundamental question:
Do miners have a right to set the price for their services and thus indirectly choose which tx are included in the block they are working on?

[theymos]

When there are conflicting lists?

The miner could choose the list to use. Someone could even publish valid lists that are always empty, though rational miners won't use these because including a transaction is almost free.

[DeathAndTaxes]

Why would a rational miner want to allow free tx forever?

Thanks for explaining it theymos but I like it less now with the explanation.  

For example in the last block:

http://blockchain.info/block-index/197728/0000000000000628467e112dd0aca3b6e5abbdd12cbd1b79074c446e85a2689d

there was 0 tx with a fee of 0.01 or higher?

So if majority of miners include a list with these tx I am forced:
a) to mine for nothing
b) to risk my block being considered invalid

Horrible idea.  Mining should be an open free market and if I don't want to include any tx without a sufficient fee I should be able to.  

More centralization for nothing.

If DeepBit and Slush built a shared list they could
a) determine pricing of entire bitcoin economy
b) force other pools to pay for access to the list or risk them building invalid blocks
c) to aid in b they could witholding tx from the list and publish them only to subscribers (if you don't pay you can't mine)

[theymos]

You're misunderstanding the proposal. Anyone who is verifying the chain can publish a list. Lists can use any fee rules. For example, Bitcoin Block Explorer might publish lists at /q/getValidationProof/minFee, which would only list transactions with a fee above minFee. Under this system, dumb miners would be able to choose essentially arbitrary fee policies; they'd just be forced into mining only properly-verified transactions and blocks.

[Haplo]

I think there's some confusion here.

Currently, it is possible to get a tx pushed with no fee, but the commit times are horrible and there's no guarantee that it will even work. Nothing really changes here, and miners can still do whatever they want, however stupid it may be.

Also, while gmaxwell did propose a mechanism for forwarding lists of tx, this was primarily intended to give the bots some pre-verified tx they could use to mine into their own blocks. In other words, only real miners can produce such a list (since only they can verify the previous tx of the blockchain) and the only way a bot's block could be included at all is if it voluntarily included some tx's sent to it. In other words, you've got it backwards; if a miner doesn't want a 0 fee tx, it can dump it on the bots.

However, I suggest against that simply because it might give people an incentive not to include a fee, and still be able to get their tx committed, thus robbing potential fees from miners who actually own their equipment. A stolen mining rig costs nothing, so even if they only processed 0tx fees they could still compete and drive down prices.

(basically what theymos said, but with a bit more detail)

[DeathAndTaxes]

You're misunderstanding the proposal. Anyone who is verifying the chain can publish a list. Lists can use any fee rules. For example, Bitcoin Block Explorer might publish lists at /q/getValidationProof/minFee, which would only list transactions with a fee above minFee. Under this system, dumb miners would be able to choose essentially arbitrary fee policies; they'd just be forced into mining only properly-verified transactions and blocks.

Of course they would.  Whatever list is used by 51% of miners will orphan out any other chain.