Idea: new rules for block validation

[ovidiusoft]

I am not trying to alienate miners, but they have, IMHO, too much power regarding which transactions are included and when.

They have no obligation to include a transaction. They are doing the work. You are the one that wants it included, so you are the one that need to convince them.

Miners are being paid 50 BTC + fees to help the network. In Bitcoin context, helping the network means confirming transactions. I will argue that a miner who doesn't include transactions (I know there are still miners who don't include *any* transaction) is not helping the network, but harming it (see Luke-Jr's attack on a alt coin) so they should not be paid at all. Invalidating their blocks seems to me the right thing to do.

Unfortunately, convincing a miner to include a transaction after it's been broadcast is simply impossible, so I was looking for a solution to always guarantee a tx that got transmitted will be included in a block in a timely fashion. Other, better ideas are welcome, of course.

[2_Thumbs_Up]

They are still doing the job and you are not, so it should be their choice and not yours.

As to miners who are not including any transactions (against their own best financial interest), they are still providing confirmations for previous blocks. Thus, they are providing owners of bitcoins a service by increasing the proof-of-work needed to double spend the coins. Considering that they are only providing this service to savers, it seems fair that only the savers pay for this through inflation. In short, miners get paid for the service they provide, by the people who recieve said service. Savers are actually getting the short end of the straw here since they are actually subsidizing those who want to get their transaction relayed at the moment.

[DeathAndTaxes]

Miners are being paid 50 BTC + fees to help the network. In Bitcoin context, helping the network means confirming transactions. I will argue that a miner who doesn't include transactions (I know there are still miners who don't include *any* transaction) is not helping the network, but harming it (see Luke-Jr's attack on a alt coin) so they should not be paid at all. Invalidating their blocks seems to me the right thing to do.

Your prejudices and misinformation is astounding.

Even a miner which mines empty blocks improves the security of the network.  For an attacker to gain 51% of hashing power they must gain 51% of network hashing power (including miners who mine empty blocks).

Also 1 confirmation is generally not seen as high security.  Miners who mine empty blocks increase the dept of already confirmed transactions and thus improve their security.

Currently there is little economic incentive for miners to process transactions.  Honestly an empty block is worth 50 BTC and on average a block w/ transactions is worth 50.01 BTC.  So the network collectively is saying processing transaction is worth an extra 0.02% revenue. 

Most miners are doing it for economic compensation.  If fees made a larger and more significant portion of revenue it is very unlikely that more than a tiny fraction of the network would exclude them.

[kjj]

Features like those I described will only be necessary years from now, though Gavin has implementing an open market for tx fees high on his todo list. I'm not sure what are his plans for this though.

It is high on my list because I think most miners (and pools) would be happy to include many more free transactions than the current rules allow, and if there is another price spike or somebody rich decides it would be fun to make the block chain a couple of gigabytes bigger it is much easier to react if the fees are not hard-coded.

The rough plan is:

 + Give miners more "knobs" to set fee policy-- let them specify (via command-line switch and maybe bitcoind RPC command) how much (if any) space to set aside in blocks for free transactions, how much to charge per-kilobyte and/or per-ECDSA-signature-validation, and what the priority/size/number-of-signatures thresholds are for considering a transaction for inclusion in the free space.

 + As Meni says, teach clients to look at the recent blockchain history and, for a given transaction, estimate how much of a fee will be required to get it into a block reasonably quickly. Maybe a "createtransaction" RPC call that locks coins for a certain amount of time and returns the how-long-to-confirm estimate along with "commit/aborttransaction" calls....

 + Figure out a reasonable UI for fees. Maybe:  calculate the probability sending the transaction with 0 fee will get into the next, oh, 3 blocks, and if it is greater than, oh, 90% then just send it without a fee.  Otherwise, let the user decide between paying a fee that will get it included (with 90% probability) in the next 3 blocks or letting them know how long it might take if they pay no fee.

Lots of details to be worked out...

The first part is relatively easy.  The second part is much harder.  The third part could turn out to be impossible.

Check out the stratum thread.  An overlay network would probably be a better place to put fee information.  I made a post that described sorta what a fee description message would look like.  It was overly simple, in that it didn't include portions for the fee policy knobs, but those could be added easily enough.

[ovidiusoft]

Your prejudices and misinformation is astounding.

Prejudices, yes. Misinformation, no. I know how the network works, I just don't agree with this specific part.

Also 1 confirmation is generally not seen as high security.  Miners who mine empty blocks increase the dept of already confirmed transactions and thus improve their security.

Correct. I simply believe that for 50 BTC they should also include old tx. Call me an capitalist pig who's trying to exploit poor miners and ask for a little more than just security

Most miners are doing it for economic compensation.  If fees made a larger and more significant portion of revenue it is very unlikely that more than a tiny fraction of the network would exclude them.

No problem here, let's increase fees. I think I said it at least 10 times already. What I'm not comfortable with it the "unlikely" and "tiny fraction" parts. Luke proved that it can happen. How can we make sure it's not possible in the future, to Bitcoin? Everyone disagrees with my idea - which is all right, that's why this is a forum. I don't see a better idea yet, but I'm hoping the great masters of the Bitcoin will find one (most likely annoyed by me, lol).

[DeathAndTaxes]

Call me an capitalist pig who's trying to exploit poor miners and ask for a little more than just security

I won't call you a capitalist.  A capitalist would believe the free market can solve the "issue" (which is really a non-issue as your transaction was likely delay for completely different reasons).

No problem here, let's increase fees. I think I said it at least 10 times already.

Who is stopping you?  Increase you fees.  That is how a free market works.   If you are concerned about this increase your fees 10 fold.  Get a couple thousand other users to do the same thing.   With higher fees there is more of an incentive to include transactions.

What I'm not comfortable with it the "unlikely" and "tiny fraction" parts. Luke proved that it can happen. How can we make sure it's not possible in the future, to Bitcoin? Everyone disagrees with my idea - which is all right, that's why this is a forum.

You are aware that Luke is a small fraction of BITCOIN but was >51% of the altchain which was attacked.  It IS impossible in Bitcoin without 51% of hashing power AND ... DRUMROLL .... if someone has 51% of hashing power they can do a lot worse things than not include your transaction.

I don't see a better idea yet, but I'm hoping the great masters of the Bitcoin will find one (most likely annoyed by me, lol).

Why would there need to be a solution to a non-problem?  You haven't even proven a problem exists. Most miners include transactions.  As block reward continue to decline towards zero and fees rise the economic pressure will make the % of network that excludes transactions fall to nothing.  You don't need 100% of the network working on your transaction for the network to work.

How many empty blocks have you seen in last week?  How many of them were empty simply because there were no transaction during that period of time?

It is entirely possible that some % of network isn't even aware of your transaction.   Even if they are there is no mechanism to guarantee the time the transaction was created.  Even if there was different parts of the network would have different timestamps for the same transaction.   That is even before issues like a network split, intentional attack, abuse by pools (pools could cripple smaller pools by paying a fee to have people run clients which only forward transactions to them).

My prediction.  No "solution" for your non-problem will ever be implemented because a) no problem exists and b) the complexity and risk of unintended consequences would far outweigh the benefit.

[ovidiusoft]

Ah, there's no problem. Ok then, I'm cool

[markm]

The "problem" seems to have been that the client recommended being a cheapskate instead of recommending paying a substantial fee.

Maybe this can be "solved" during install or something by asking the installer whether they want to install as a cheapskate client that is willing to wait days for their transactions to be processed, a normal home user client that won't be upset if some transactions take a day or few to be processed but would prefer most transactions to most likely be processed within a day or so, or a business client that would like a reasonable number of their transactions to most likely be processed within one business day or less.

Basically just multiply all fee estimates, starting with the current low estimate and multiplying by 2 or 3 for home users, 3 4 or 5 or so for business, Maybe two questions: home user or business user and low, medium or high priority. Then tell them also how to increase it even beyond that estimate/guess for very important transactions, like maybe tell them to double it or triple it for such transactions.

Since the only thing actually seemingly wrong is the client recommending paying too little, maybe forcing the user to make their own choice as to how stingy or generous the client's recommendations should be will "fix" it?

-MarkM-

[Killdozer]

Why would there need to be a solution to a non-problem?  You haven't even proven a problem exists. Most miners include transactions.  As block reward continue to decline towards zero and fees rise the economic pressure will make the % of network that excludes transactions fall to nothing.  You don't need 100% of the network working on your transaction for the network to work.

He never said the problem was all so big right now. But do you suppose we just think about today and don't give a shit about what happens in the future?

ovidiusoft, you have rised some valid points for further contemplation. (And got a bit too much childish critique for it instead of a mature discussion from some members if you ask me.)

First of all, I sort of agree with your assumption that the whole network is going to stay whole for such a long time as 10 or even moreso 100 blocks. There will not be transactions that "will not propagate enough".

Because if there will be, this basically means that the network is split, and some serious shit is going on. In that case we will not have to worry about some transaction not being included, we will have to worry about a 100 block long reorg, which will make the recent mtgox hack seem like a fart in a hurricane.

So while this would be most probably technically possible, it is till too early to force an inclusion of a transaction by the network, it seems like it creates more problems than it solves.

Instead I am more for just increasing the fees with time. And the more I think about it, the more it seems that fees should be percentage-based with some minimum limits to still prevent spamming of small transactions. But even that is a bit too early, because it will only solve a real issue when the block reward will decrease and transactions will have to become bigger in order to pay for all those 5770s and keep the bitcoin's contribution to global worming on the level.

For an actual change right now, what somebody else suggested here about a way for a transaction to expire after a certain block number if it has not been included seems reasonable. That way if your transaction doesn't get included, you could resend it with higher fees and not look like a backstabbing double-spending sonofabitch.