Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: dahongfei on October 24, 2013, 09:00:28 AM



Title: Smarter transaction fees will be implemented in bitcoin-qt 0.9
Post by: dahongfei on October 24, 2013, 09:00:28 AM
As Gavin Andresen wrote today posted in Core Development Update #5 https://bitcoinfoundation.org/blog/?p=290

Quote
Smarter transaction fees

Today, transaction fees are hard-coded into the Bitcoin-Qt/bitcoind wallet software, and the rules surrounding those fees are a collection of heuristics that evolved over the last four years. In short, the current fee-handling code is a complicated mess that will stop working as soon as transaction volume doubles a couple of more times.

In theory, it should be simple: transaction fees should to be set by a give-and-take between the people creating transactions (who want to pay as little as possible) and the people validating and storing the transactions (miners, who would like to be rewarded with higher fees).

And the code that miners are using today to select transactions is simple: they fill their blocks with the highest fee-per-kilobyte transaction (and highest priority transactions, if they choose to set aside some space in their blocks for free transactions).

I’ve been working on teaching the wallet code to estimate how low a fee (or priority) a transaction needs, at the moment it is sent, to be accepted by miners and included in the next block or three. The estimates are based on watching transactions as they are broadcast on the network and keeping track of which of those transactions are accepted into blocks.

The danger with estimating transaction fees is miners have an incentive to try to game the estimate to make transaction fees higher. For example, if the estimate was based on the average transaction fee for all transactions in the last N blocks, miners could add very-high-fee pay-to-self transactions to the blocks that they mine to drive up the average. However, by only considering fees for transactions that have been broadcast on the network that threat is eliminated– miners could broadcast very-high-fee pay-to-self transactions, but would end up paying those high transaction fees to other miners. The transaction estimation code also uses median transaction fees, not averages, to make it much harder for a minority of transactions to influence transaction fees.

If most of the miners got together and agreed to all broadcast lots of high transaction fee transactions they could succeed in driving up the average transaction fee… but that isn’t going to happen because it would be easy for miners to “free ride” by breaking the agreement and just collecting the higher transaction fees for themselves.

All of the above will work well for fully-validating software like Bitcoin-Qt/bitcoind. More thinking is needed to figure out how lightweight simplified payment verification software (like Multibit) can better estimate transaction fees, because today they can’t calculate how much transaction fee each transaction is paying.

What do you guys think of this approach?


Title: Re: Smarter transaction fees will be implemented in bitcoin-qt 0.9
Post by: piotr_n on October 24, 2013, 11:03:23 AM
What do you guys think of this approach?
Personally, I don't like a software which assumes that I am so stupid that I cannot calculate a proper fee myself.
But I guess Gavin is doing it for his grandma...

Anyway, I will believe that this solution actually works when I see it.
And by "works" I mean that it can actually calculate the lowest fee possible, to get a tx mined within the assumed time window.

Moreover, like with many other extremely important features, I have no idea what it has to do with a development of the bitcoin core.
One could think of a webpage that would be doing exactly the same; tell a user what fee he shall include in a tx, to get it mined within a given time window.

Maybe Gaving should step down from the core bitcoin dev position and start developing websites for Bitcoin Foundation?
He seems to be much more into this kind of technologies, rather than a virtual currency that begs for more decentralization, anonymity and security.


Title: Re: Smarter transaction fees will be implemented in bitcoin-qt 0.9
Post by: Peter Todd on October 24, 2013, 11:06:06 AM
It's not very well thought out.

The main problem is it calculates fees based purely on blockchain data, so if miners ever accept transactions for reasons other than fees, or because of some contract to mine transactions for someone else, the estimates get pushed down and your transactions will get stuck. If a transaction does get stuck, there's nothing you can do about it other than wait - no different from now.

There are already miners with such contracts, Eligius for example, and the practice is likely to become more widespread in the future because the payment protocol is designed to allow the shifting of the burden of mining a transaction to the merchant. The only way the merchant can get a transaction confirmed if the customer put a too-low fee is to either respend it, making use of Luke-Jr's child-pays-for-parent code, (not yet included in the reference client) or by using a out-of-band transaction mining contract. The latter is cheaper because you don't have to pay for a wasteful second transaction.

The other issue is that fee estimation can only be done by full clients; the SPV clients that more and more people are using don't have the blockchain data or mempool so they can't make good fee estimates. Gavin's solution is to make full nodes provide SPV clients with estimation services, but there's no way for the client to know if the node is lying - it gives miners incentives to setup lots of nodes that lie about fees, saying they are higher than they really are, or griefers incentives to do the opposite and get people's transactions stuck. Bitcoin is supposed to be as zero-trust as possible...


There are better solutions, the main one is transaction replacement. Basically if your first transaction isn't getting mined, you broadcast a second one that spends at least one of the same inputs (so both can't be mined at once) and pays every output that the first did, plus another change output. (so zeroconf transactions are still "safe") This would work best if combined with proof-of-propagation (http://sourceforge.net/mailarchive/message.php?msg_id=31436078), which gives you cryptographic proof that your transaction actually got to miners, although that doesn't have to be implemented immediately.

The main disadvantage with transaction replacement is that it will confuse some wallet software, but that same wallet software can get confused by accidental double-spends, transaction malleability, etc. We're better off if people in the ecosystem write robust software frankly rather than making assumptions that aren't actually true.

Interestingly part of Gavins proposal is to make transactions expire from mempool's relatively quickly, and in that case then yeah, just like in replacement the wallet would rebroadcast a different version with a higher fee! So you still need to implement changing transactions, and you also still need to fix wallets so they handle replacements. Only in his version you have to wait until the transaction expires, which is problematic because you've got stuff like misguided people rebroadcasting transactions that they don't own, preventing the expiration from happening. (making expiration permanent is ugly too - what if you're trying to get a transaction paying you mined, and can't change it?)

Fee estimation with replacement is fine though, as it lets you replace transactions in the event that the estimate was too low. This helps remove some of the incentive for griefers to play games lying to SPV clients, and it lets people be more aggressive in their bargaining with miners by allowing people willing to wait a little bit the ability to make an initial low-ball bid, and raise it if needed.


tl;dr: Why would you want to be at an auction for something you needed where you got exactly one chance and one chance only to make a bid? There's nothing wrong with making an initial estimate for your first bid to save time, but the bottom line is you're going to be able to get a better deal if you can make a low-ball bid first, raising your bid only if required.