Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: TierNolan on June 26, 2011, 03:55:54 AM



Title: Add option to add a fee to dead transactions
Post by: TierNolan on June 26, 2011, 03:55:54 AM
If a user sends a transaction with insufficient fees, it ends up lost in limbo and never confirms. 

However, there is no downside to allowing a client to increase the fee for a transaction after it has been sent out.

This would resend the transaction except it would reduce the output and/or increase the inputs, so as to include a fee (and recalculate any hashes/signatures).

If the original transaction happens first, then great, you got a free transaction.  Otherwise, the updated transaction is processed as it meets the fee requirement.

This would allow market forces to kick in, rather than requiring the fee rules to be incorporated into the default client.  Users would get a feel for what kind of fees are required to get transactions accepted by the miners/forwarders.


Title: Re: Add option to add a fee to dead transactions
Post by: TiagoTiago on June 26, 2011, 04:08:18 AM
I think the way it is supposed to work with the system the way it currently is, is not that you change an existing pending transaction, but you spend the same coins again with different fees attached and let the network sort out the doublespending weeding out whichever copies of the transaction doesn't get done first.


The  thing is the client currently locks out 'coins already included in an outgoing transaction, even if unconfirmed, and got no means of letting the user try to send those coins again.


I'm not sure what changes would be necessary to allow people to change existing transactions instead of just repeating transactions with different fees.




If i said anything wrong please someone that understands the system better correct me, aight?


Title: Re: Add option to add a fee to dead transactions
Post by: JoelKatz on June 26, 2011, 04:44:33 AM
You'd have to empty the account you were sending from. Otherwise, nothing would prevent both transactions from going through.

Update: THIS IS INCORRECT. See next message. Mea culpa.


Title: Re: Add option to add a fee to dead transactions
Post by: TierNolan on June 26, 2011, 09:19:13 AM
I think the way it is supposed to work with the system the way it currently is, is not that you change an existing pending transaction, but you spend the same coins again with different fees attached and let the network sort out the doublespending weeding out whichever copies of the transaction doesn't get done first.

Yes, that is what I am suggesting.  You just spend the coin a second time and once one of the transactions is accepted, then the other transaction is automatically cancelled.

Quote
The  thing is the client currently locks out 'coins already included in an outgoing transaction, even if unconfirmed, and got no means of letting the user try to send those coins again.

I'm not sure what changes would be necessary to allow people to change existing transactions instead of just repeating transactions with different fees.

As long as at least one coin is present as an input into both transactions, they can't both be added to the chain.  If you do multiple re-sends, the client would have to remember to keep including at least one coin.

The easiest rule would be to just include the first coin in all resends.

You'd have to empty the account you were sending from. Otherwise, nothing would prevent both transactions from going through.

That isn't true.  Your account balance is entirely virtual.  The money is made up of lots of coins (actually outputs from accepted transactions).

Assuming you own coins A, B and C, you can send any number of transactions with those as inputs.

Transaction 1 (fee = 0):
Inputs:
A: 0.4
B: 0.2

Outputs:
D: 0.5
E: 0.1

Transaction 2 (fee = 0.001)
Inputs:
A: 0.4
C: 1.0

Outputs:
D: 0.5
E: 0.899

The E coin is the change coin and it would be set to only be spendable by someone who knows the private key linked to your address.


Title: Re: Add option to add a fee to dead transactions
Post by: prof7bit on June 26, 2011, 09:46:58 AM
The only way to cancel an unconfimed transaction is to make it invalid and the only way to make a valid transaction invalid is to spend one of its inputs and get this confirmed in a block before the unwanted transaction makes its way into a block.

The problem I see here is that as long as the unwanted transaction is already hanging around in the miners' memory pools the new transaction would not even be allowed to enter their memory pools.


Title: Re: Add option to add a fee to dead transactions
Post by: TierNolan on June 26, 2011, 11:37:28 AM
The problem I see here is that as long as the unwanted transaction is already hanging around in the miners' memory pools the new transaction would not even be allowed to enter their memory pools.

My understanding is that if the default client forwards the transaction, it will also mine it?

In all likelihood, the original transaction won't make it very far unless it is acceptable to the default client and if it is acceptable to the default client, then all miners should get it.  It only requires one miner to change his code to replace low fee paying transactions with high fee paying ones if there is a collision.

The recommended network rules are given on the wiki (https://en.bitcoin.it/wiki/Protocol_rules).

For transactions:

"9. Reject if any other tx in the pool uses the same transaction output as one used by this tx."

Maybe that could be changed to

"9. Reject if any other tx in the pool uses the same transaction output as one used by this tx, and has a greater or equal transaction fee"

It might also be worth adding the actual default client's rules to the wiki.


Title: Re: Add option to add a fee to dead transactions
Post by: gentakin on June 26, 2011, 11:49:05 AM
"9. Reject if any other tx in the pool uses the same transaction output as one used by this tx, and has a greater or equal transaction fee"

+1. The "low/no fee" transaction should of course be dropped by miners when a high fee transactions comes in.

I think this is the most elegant solution to the fee problem.


Title: Re: Add option to add a fee to dead transactions
Post by: Pieter Wuille on June 26, 2011, 12:38:25 PM
"9. Reject if any other tx in the pool uses the same transaction output as one used by this tx, and has a greater or equal transaction fee"

+1. The "low/no fee" transaction should of course be dropped by miners when a high fee transactions comes in.

I think this is the most elegant solution to the fee problem.

In the general case, it is impossible to use the same outputs, as you may need more/different inputs to pay the fee, resulting in a different change, implying different outputs. It is not possible to detect which outputs are change, so you can't in general detect whether one transaction would be allowed to replace another one.


Title: Re: Add option to add a fee to dead transactions
Post by: TierNolan on June 26, 2011, 12:56:59 PM
In the general case, it is impossible to use the same outputs, as you may need more/different inputs to pay the fee, resulting in a different change, implying different outputs. It is not possible to detect which outputs are change, so you can't in general detect whether one transaction would be allowed to replace another one.

You could just decrease the output payment in order to increase the fee.

As long as a transaction hasn't been incorporated into the main chain, it is perfectly acceptable to reverse it.


Title: Re: Add option to add a fee to dead transactions
Post by: gentakin on June 26, 2011, 01:02:55 PM
In the general case, it is impossible to use the same outputs, as you may need more/different inputs to pay the fee, resulting in a different change, implying different outputs. It is not possible to detect which outputs are change, so you can't in general detect whether one transaction would be allowed to replace another one.

Yes, it's not always possible to use the same outputs as input for the new (overriding) transaction, because adding fees sometimes implies using a different input.

So the question actually is about which transaction wins when two transactions share at least one output from a previous transaction as their input. One of the transaction needs to be dropped as invalid.
AFAIK, 0.3.23 will drop the incoming transcation and keep the old one.

There are more ways to deal with an "output/input" conflict, and some (or all) of them might be entirely stupid. To be honest, I can't really tell the consequences..

* Drop the old transaction, use the new incoming transaction as "valid". This probably allows me to increase fees, decrease fees, and even to "cancel" the transaction (by replacing the output of the transaction to go to my own wallet). Of course, if a miner refuses to drop the old transaction, it might still be included in a block and become valid, even though the user canceled/changed his transaction.
* Drop the transaction with lower fees (or lower fee/kb). This allows to increase fees and might solve the "coins in limbo with 0 confirmations" problem.

The client needs some logic to actually allow changing a transaction by the user. An intentional conflict needs to be created that makes sure the old transaction is considered "invalid" by most miners. (Since we can't control what logic miners use to deem transactions invalid, this can never be 100%..)


Title: Re: Add option to add a fee to dead transactions
Post by: kjj on June 26, 2011, 08:04:39 PM
The only way to cancel an unconfimed transaction is to make it invalid and the only way to make a valid transaction invalid is to spend one of its inputs and get this confirmed in a block before the unwanted transaction makes its way into a block.

The problem I see here is that as long as the unwanted transaction is already hanging around in the miners' memory pools the new transaction would not even be allowed to enter their memory pools.

Should be trivial to recognize and allow a transaction that differs only in the fee (and thus, the change).

It might also be a good idea to alter the spend behavior to include a little change in the input even when not strictly necessary, so that the resend attempt doesn't need to add a new input.


Title: Re: Add option to add a fee to dead transactions
Post by: TiagoTiago on June 26, 2011, 08:06:26 PM
I think it would be better if multiple variations of the transaction requests were allowed to coexist in the network, leaving the pruning to be done only when creating blocks (so for example if a philanthropic mining operation sees two variations they can choose to process the one with the cheaper transaction fee); but of course, any copies laying around after one has been included in the blockchain would be terminated on sight.


Title: Re: Add option to add a fee to dead transactions
Post by: FreeMoney on June 26, 2011, 08:14:12 PM
What's interesting to me is that a receiver can add a fee by making a tx that is dependent on the unconfirmed one that has a fee. Any miner who wants it has to include the dependent tx.


Title: Re: Add option to add a fee to dead transactions
Post by: TierNolan on June 26, 2011, 08:31:29 PM
What's interesting to me is that a receiver can add a fee by making a tx that is dependent on the unconfirmed one that has a fee. Any miner who wants it has to include the dependent tx.

That's true.  However, there is still the problem that non-fee paying transactions aren't forwarded.  It is easier to check for collisions than to scan all transactions in memory and try to find profitable paths from low fee transactions to the incentive transactions.

It might also be a good idea to alter the spend behavior to include a little change in the input even when not strictly necessary, so that the resend attempt doesn't need to add a new input.

That is only needed if transactions are only cancelled by identical ones.


Title: Re: Add option to add a fee to dead transactions
Post by: Maged on June 26, 2011, 10:23:08 PM
* Drop the old transaction, use the new incoming transaction as "valid". This probably allows me to increase fees, decrease fees, and even to "cancel" the transaction (by replacing the output of the transaction to go to my own wallet). Of course, if a miner refuses to drop the old transaction, it might still be included in a block and become valid, even though the user canceled/changed his transaction.
* Drop the transaction with lower fees (or lower fee/kb). This allows to increase fees and might solve the "coins in limbo with 0 confirmations" problem.

The client needs some logic to actually allow changing a transaction by the user. An intentional conflict needs to be created that makes sure the old transaction is considered "invalid" by most miners. (Since we can't control what logic miners use to deem transactions invalid, this can never be 100%..)
We absolutely cannot allow people to cancel unconfirmed transactions. Thus, one additional check can be made that the outputs are going to the same addresses and are greater than or equal to the original amounts. To prevent compromising which txOut is the change, there can also be additional change addresses added. In fact, this creates a nice side effect: It would then be possible to attach new transactions to existing open transactions. Imagine if the Faucet could rebundle their mega-transaction every time someone requested coins!


Title: Re: Add option to add a fee to dead transactions
Post by: TierNolan on June 26, 2011, 10:36:34 PM
We absolutely cannot allow people to cancel unconfirmed transactions.

If a transaction hasn't been added into the chain, then there isn't any reason why it is more important than any other colliding transaction.

The fee would only be used as a tie breaker if 2 transactions had the same number confirms (normally zero except during a fork).

In fact, allowing higher fee transactions override lower fee transactions increases the chances that everyone sees the same one as high priority.

Quote
Thus, one additional check can be made that the outputs are going to the same addresses and are greater than or equal to the original amounts. To prevent compromising which txOut is the change, there can also be additional change addresses added. In fact, this creates a nice side effect: It would then be possible to attach new transactions to existing open transactions. Imagine if the Faucet could rebundle their mega-transaction every time someone requested coins!

Not sure exactly what you mean here.


Title: Re: Add option to add a fee to dead transactions
Post by: kjj on June 26, 2011, 10:46:36 PM
It might also be a good idea to alter the spend behavior to include a little change in the input even when not strictly necessary, so that the resend attempt doesn't need to add a new input.

That is only needed if transactions are only cancelled by identical ones.

This is just to make it easy for nodes to recognize replacement spends.

Say the first transaction is X+Y in and A+B out, with no fee so that X+Y == A+B.  If it doesn't get picked up quickly enough, the user tried a second one with X+Y in and A+(B-F) out so that F is the fee.

We could also make the rule that the output getting reduced, B, has to have the same address as one of the inputs.  That way we know it is the change getting reduced, and not the spend.  Vendors that don't wait for even a single confirmation could get burned otherwise (but if they don't wait, they are asking to get burned anyway).

I'm not sure if this is a good road to go down or not, but if we do decide that a double spend is acceptable in this circumstance, at least it is recognizable, and somewhat safe.


Title: Re: Add option to add a fee to dead transactions
Post by: Maged on June 26, 2011, 10:53:44 PM
We absolutely cannot allow people to cancel unconfirmed transactions.

If a transaction hasn't been added into the chain, then there isn't any reason why it is more important than any other colliding transaction.
Oh, I know. A Finney Attack would still allow double-spending, but the idea is to not make double-spending as simple as clicking "cancel".


Title: Re: Add option to add a fee to dead transactions
Post by: TierNolan on June 26, 2011, 11:04:43 PM
Say the first transaction is X+Y in and A+B out, with no fee so that X+Y == A+B.  If it doesn't get picked up quickly enough, the user tried a second one with X+Y in and A+(B-F) out so that F is the fee.

It would make it easier to require an absolute match, since transaction data management is potentially a big overhead for miners.

Quote
We could also make the rule that the output getting reduced, B, has to have the same address as one of the inputs.  

Currently, the default client doesn't send anything back to the original address.  What was the reason for that?  Was it to create a psuedo mix-net?

Quote
I'm not sure if this is a good road to go down or not, but if we do decide that a double spend is acceptable in this circumstance, at least it is recognizable, and somewhat safe.

I think the idea that it must go back to the same address as the input would fix the issue anyway.  Most of the mixing could be preserved if only a small portion of the money was placed in the "reserve".

However, the hardening of bitcoins is incorporation into the main chain.  Some miners will accept transactions being sent directly to them.


Title: Re: Add option to add a fee to dead transactions
Post by: kjj on June 26, 2011, 11:14:14 PM
Say the first transaction is X+Y in and A+B out, with no fee so that X+Y == A+B.  If it doesn't get picked up quickly enough, the user tried a second one with X+Y in and A+(B-F) out so that F is the fee.

It would make it easier to require an absolute match, since transaction data management is potentially a big overhead for miners.

Quote
We could also make the rule that the output getting reduced, B, has to have the same address as one of the inputs.  

Currently, the default client doesn't send anything back to the original address.  What was the reason for that?  Was it to create a psuedo mix-net?

Quote
I'm not sure if this is a good road to go down or not, but if we do decide that a double spend is acceptable in this circumstance, at least it is recognizable, and somewhat safe.

I think the idea that it must go back to the same address as the input would fix the issue anyway.  Most of the mixing could be preserved if only a small portion of the money was placed in the "reserve".

However, the hardening of bitcoins is incorporation into the main chain.  Some miners will accept transactions being sent directly to them.

Ahh.  The reason is probably because reusing keys is a bad idea, in general.  Many cryptosystems have are weak when an attacker is able to collect many cyphertexts all using the same key.  I don't know how much of a problem it is when using ECDSA, but all the textbooks since WWII have said not to do it unnecessarily.

The client could reduce this problem by keeping track of the addresses getting change, and chaining them into the following spend on the changeless side.

In this X+Y==A+(B-F) example, B is the change.  The next transaction the client does would be B+Z==C+D, or C+(D-F).  Then the next would be D+W, and so on.


Title: Re: Add option to add a fee to dead transactions
Post by: TierNolan on June 26, 2011, 11:43:46 PM
Ahh.  The reason is probably because reusing keys is a bad idea, in general.  Many cryptosystems have are weak when an attacker is able to collect many cyphertexts all using the same key.  I don't know how much of a problem it is when using ECDSA, but all the textbooks since WWII have said not to do it unnecessarily.

Good to know :). 

Quote
The client could reduce this problem by keeping track of the addresses getting change, and chaining them into the following spend on the changeless side.

In this X+Y==A+(B-F) example, B is the change.  The next transaction the client does would be B+Z==C+D, or C+(D-F).  Then the next would be D+W, and so on.

Ideally, there would be some kind of incentive to reduce the total number of active transactions.  The recommended fee could be lower for transactions that combined lots of low value coins into high value coins.

Minimum fees based on the size of the transaction already mean that tiny coins have less value than larger coins.


Title: Re: Add option to add a fee to dead transactions
Post by: kjj on June 27, 2011, 12:17:49 AM
Ideally, there would be some kind of incentive to reduce the total number of active transactions.  The recommended fee could be lower for transactions that combined lots of low value coins into high value coins.

Minimum fees based on the size of the transaction already mean that tiny coins have less value than larger coins.

Right now the default client enforces some limits on transaction fees, mostly to stop transaction spam.  Over time, those will go away, and each network node and miner will be able to set their own rules for accepting and relaying transactions, which means that there will be a market of sorts for fees.

But, when there are a lot of transactions to process, the miners will evaluate them in the terms that make sense to them.  That mostly means that they will want a high ratio of fee to work.  Each input to a transaction involves a certain amount of work, so there is no possible way to provide incentives to miners for consolidation transactions.

So, while consolidation may be useful to other parts of the network, it will not be prioritized by the group capable of making it actually happen.


Title: Re: Add option to add a fee to dead transactions
Post by: Insti on June 27, 2011, 06:20:24 AM
Why isn't a dependent transaction which pays a fee all that is needed? This requires no dropping of existing transactions.

1. A+B -> C + D
2. D+E -> F + (G - Fee)

In this case the Fee pays for both transaction 1 and transaction 2.
The only problem seems to be if you make a transaction with zero change and then want to add an additional fee. But that is up to the choice of the sender anyway.


Title: Re: Add option to add a fee to dead transactions
Post by: TierNolan on June 27, 2011, 09:04:54 AM
Why isn't a dependent transaction which pays a fee all that is needed? This requires no dropping of existing transactions.

1. A+B -> C + D
2. D+E -> F + (G - Fee)

In this case the Fee pays for both transaction 1 and transaction 2.

A zero fee transaction won't propagate and neither will one that collides with another transaction.  If you have a rule that higher transaction fee transactions that have all the same inputs and outputs replace the older transaction, then it doesn't matter if the node has seen the earlier transaction or not, it will propagate.


Title: Re: Add option to add a fee to dead transactions
Post by: TierNolan on June 27, 2011, 09:47:18 PM
Ahh.  The reason is probably because reusing keys is a bad idea, in general.  Many cryptosystems have are weak when an attacker is able to collect many cyphertexts all using the same key.  I don't know how much of a problem it is when using ECDSA, but all the textbooks since WWII have said not to do it unnecessarily.

Good to know :). 


Does this mean that you shouldn't ask people to send money to an address once you have sent money from the address?