[RFC] Finite transaction lifetime (aka "fill or kill")

[jgarzik]

One of my major bitcoin criticisms remains non-deterministic transaction behavior, from the user PoV.  Transactions that are non-standard, or too large for the included fees, will be resent hopelessly, without the user ever having any resolution whatsoever.  I call these "zombie transactions."

In the real world, transactions tend to be more binary:  they either succeed or fail.  They don't sit around in limbo forever.

Therefore, I propose a new network rule:

• Note the current block height, when a TX is received at a node, X.
• When block height increases to X+T, and the TX has not yet been put into a block, remove it from the TX cache.  T could be 150 blocks, around 24 hours, but that's open to discussion.

In the short term, this should have little effect, as clients will continue to resend transactions ad infinitum, until they get into a block.  And TX cache is flushed in any case, when a bitcoin node restarts.

In the long term, this guarantees predictable behavior for each transaction.  With this network rule in place, a user will know that their TX will probably either make it into a block, or disappear from TX caches [and thus be eligible for resubmittal].  This should open up the possibility for users to recover spent coins that never made it into a block (never confirmed).  The current inability for a user to recover unconfirmed-and-never-will-be money is a flaw that should be corrected, though luckily these are rare cases today.

However...   Such unspending is difficult:  a spend potentially creates a change transaction, which itself could then be spent, and so on.  Of course, these zombie coins will never confirm, because their originating transaction is a zombie transaction.  But it is difficult to unroll a chain of spends reliably.

Comments welcome.

[theymos]

An expiration time on memory pool transactions would be a good idea for this and other reasons.

When a transaction is given up on, a new transaction back to the sender needs to be created using the same coins. Otherwise, the old transaction might eventually make it into a block, which would be undesirable. Given this, transaction replacement should be re-enabled so that this problem can be solved as intended: to cancel a transaction, you send a transaction back to yourself using the same inputs with lower sequence numbers, and all nodes update the transaction immediately.

[caveden]

Maybe making the T variable specified by the transaction itself, not a constant.

Also, block that includes a transaction which should have been expired could be considered an invalid block by the network.
This is backward incompatible though, and maybe theymos suggestion of resending the same transaction to self after the expiration would be a good idea - it would be a good idea to send this second transaction with a higher fee, too.

But anyway, reclaiming and resending with a higher fee should be even simpler to implement, shouldn't it? And it's a more generic use case.

[theymos]

Also, block that includes a transaction which should have been expired could be considered an invalid block by the network.

It's bad for a transaction to be valid at one time and then become invalid. If this happens, then network splits of less than 120 blocks in length can cause many transactions to become invalid, breaking the guarantees that Bitcoin normally provides.

For example, say a transaction with an expiration time gets in a block. 6 blocks later, the recipient of this transaction sends these coins, and 6 blocks later the next recipient sends the coins, etc. 80 blocks later, the network recombines from a split. The first transaction is now expired, and all child transactions are also invalid. There could potentially be hundreds or thousands of transactions that are now reversed unexpectedly, even though the network split lasted less than the 120-block limit.

[marcus_of_augustus]

Just considering out into the (far?) future if there will be a time when essentially no transaction will go through without a fee.

Aside: I know that some space in each block is set aside for free transactions but if the queue of free transactions is so long that the time for a free transaction to confirm is crazy long then that mechanism is impractical (1 month, 1 year, etc) and free transactions become basically useless.

So, at this point there will be some minimum fee that is required to get a transaction confirm in a practical time. Is there a way to get the current average minimum fee from the network to be sent to the client, so that when selecting a fee for transaction prioritisation the sender has some idea of what is needed for the trade to go through?

If this was implemented allowing for a minimum fee down to the 8th decimal place it would allow for a natural, continuous growth of the use of fees as the number of transactions on the network grows. I.e start using fees sooner rather than later, even if they are only infinitesimal at present and future-proof the infrastructure for when they become  more significant (forward compatible).

EDIT: On the front-end, the bitcoin client would default to include the current minimum fee for a timely transaction, with a zero fee transaction being a user-selectable choice.

[Thndr]

Well I personally think that in the future, there should be a priority level based on transaction fee and also the time the transaction original was requested.

So basically, there would a minimum number of blocks generated before a 0 fee transaction priority would be shifted to the top by the amount of time since the transaction started. This would ensure no failed transactions, and every transaction is processed.

Even in the even of thousands of 0 fee transactions being processed, the ones with fees would gain priority with time as well, becoming a higher priority faster base on the fee, leveling out the field. Once fees become standard, it should encourage people to use them due to the significant delay on low/no fee transactions.

[Timo Y]

From my understanding, it is in theory possible to "double spend" a transaction in limbo.  If the second attempt with a higher transaction fee makes it into a block, the first attempt will be interpreted as a double spending attempt by miners and simply be ignored.

(correct me if I'm wrong)

[Hal]

From my understanding, it is in theory possible to "double spend" a transaction in limbo.  If the second attempt with a higher transaction fee makes it into a block, the first attempt will be interpreted as a double spending attempt by miners and simply be ignored.

A txn "in limbo" (in the transaction pool) will block double spends from being forwarded by peers, or accepted by (unhacked) miners. The pool is only in memory so gets wiped when the node is restarted, making network behavior somewhat nondeterministic. The wallet currently retransmits unconfirmed txns indefinitely, however the peers will not forward them unless/until the txns are cleared from peer memory due to a restart.