Bitcoin Forum
November 04, 2024, 12:27:34 AM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 4 5 6 7 »  All
  Print  
Author Topic: Reminder: zero-conf is not safe; $1000USD reward posted for replace-by-fee patch  (Read 18299 times)
Peter Todd (OP)
Legendary
*
expert
Offline Offline

Activity: 1120
Merit: 1160


View Profile
April 18, 2013, 11:46:01 AM
Last edit: August 31, 2013, 07:33:18 AM by retep
Merited by ABCbits (2)
 #1

EDIT: As it turns out replace-by-fee will eventually allow for fairly safe zero-confirmation transactions, ironic really: https://bitcointalk.org/index.php?topic=251233.msg2669189#msg2669189


Someone by the name of John Dillon (john.dillon892@googlemail.com) emailed the bitcoin-development email list earlier this morning offering a $500USD reward to anyone who implements a transaction replacement-by-fee patch. That's an idea I posted on the email list two days ago:

Quote
In any case, the more pressing issue re: replacement is changing fees attached to transactions after they have been broadcast. Lots of users are getting their transactions stuck with few options to fix them.

The more I think about the issue the more I think we should nip this zero-conf madness in the bud: change the relay rules so that transactions are replaced based on fees regardless of how that changes transaction outputs. Of course, this does make double-spending an unconfirmed transaction trivial. On the other hand, this makes changing fees after the fact trivial, and it lets us implement a limited 'undo' button for when people screw up. It also allows for many of the applications transaction replacement was meant for in the first place anyway, and all the applications where it's actually secure.

We keep saying over and over again to stop accepting zero-conf transactions, but people do it anyway because it seems secure. It's a very dangerous situation because the security of zero-conf transactions can change overnight simply by some fraction of the hashing power implementing that exact change.

Some thought is required as to exactly what "replace by fees" looks like, economically optimal is a bit complex due to it's dependency on overall mempool backlog, but a rough first version should be easy to hammer out.
-Re: [Bitcoin-development] [bitcoin] Enable tx replacement on testnet. (#2516)

Like it or not, zero-conf is dangerous when you don't trust the other party. I wrote the above replace-by-fee idea because I really think we run a risk if we lull people into complacency. The blockchain and the proof-of-work system is how Bitcoin comes to a consensus about which transactions are or are not valid; trusting anything else is dangerous.

When you accept a zero-conf transaction the method of determining consensus basically comes down to hoping that all miners implement the default "no-replacement" rules, rules that can fail due to a bunch of other reasons like propagation failures. Mining pools these days are run by individuals as a (serious) hobby, and are usually hosted on insecure VPS services. The security of zero-conf transactions can change overnight by one of those pools getting hacked, or anyone with hashing power deciding to change the relay policy they use; about 10% of all blocks have unknown origins.

Trying to bolt on a second consensus mechanism, like nodes rejecting blocks if there are transactions in them that they haven't seen before, or conflict with existing transactions, is dangerous. That second consensus mechanism becomes a way to attack Bitcoin, and it can be as simple as just broadcasting different transactions to different miners so they don't know what transaction was first.

Full disclosure: I'm considering writing that patch and collecting that $1000 reward myself.

EDIT: reward has increased

Remember remember the 5th of November
Legendary
*
Offline Offline

Activity: 1862
Merit: 1011

Reverse engineer from time to time


View Profile
April 18, 2013, 11:50:53 AM
 #2

In a nutshell, what does this mean?

BTC:1AiCRMxgf1ptVQwx6hDuKMu4f7F27QmJC2
TierNolan
Legendary
*
Offline Offline

Activity: 1232
Merit: 1104


View Profile
April 18, 2013, 12:14:51 PM
 #3

In a nutshell, what does this mean?

If I tell my client to pay you, you will receive notification of that transaction from the network.

This means that my transaction has flooded the network, and more importantly, flooded to all miners.

If I then send a transaction that spends the input back to myself, the network won't relay it and miners will ignore it.

So, if you receive the transaction from lots of neighbors, you can be reasonably sure I broadcast it to the network.

The proposal allows replacement of transactions.  If I resend the transaction, but with a higher fee, it is replaced with the new transaction.  This doesn't allow me to reverse the transaction though.

In addition, the proposal, lets me change the outputs, as long as I up the fee.

This means I can send a tx to you with 0.01 fee and then revers with with a 0.02 fee.

All this requires that it hasn't been incorporated in a block.  Once it is in a block, then it is locked down (unless that block is orphaned).

Full replacement allows people to update a transaction if it is not being included in the chain.  It would also act to discourage people accepting transactions, until at least one block confirmation happens, since until it is confirmed, it is easy to reverse.

1LxbG5cKXzTwZg9mjL3gaRE835uNQEteWF
Remember remember the 5th of November
Legendary
*
Offline Offline

Activity: 1862
Merit: 1011

Reverse engineer from time to time


View Profile
April 18, 2013, 12:18:58 PM
 #4

Makes sense. Any other drawbacks?

BTC:1AiCRMxgf1ptVQwx6hDuKMu4f7F27QmJC2
🏰 TradeFortress 🏰
Bitcoin Veteran
VIP
Legendary
*
Offline Offline

Activity: 1316
Merit: 1043

👻


View Profile
April 18, 2013, 12:21:31 PM
 #5

Isn't this already done?

Quote
change the relay rules so that transactions are replaced based on fees regardless of how that changes transaction outputs.

John Dillon seems to want to make it so that clients won't check outputs are the same before replacing the transaction.
Zeilap
Full Member
***
Offline Offline

Activity: 154
Merit: 100


View Profile
April 18, 2013, 12:47:03 PM
 #6

Isn't this already done?

Quote
change the relay rules so that transactions are replaced based on fees regardless of how that changes transaction outputs.

John Dillon seems to want to make it so that clients won't check outputs are the same before replacing the transaction.

With the same inputs, the total value of the transaction is the same. Changing the fees means that the total value must be redistributed in a different way to before. Thus some outputs must get less if the fees are to get more.
I'm assuming he doesn't mean to allow adding or removing outputs.
Peter Todd (OP)
Legendary
*
expert
Offline Offline

Activity: 1120
Merit: 1160


View Profile
April 18, 2013, 01:00:32 PM
 #7

Isn't this already done?

Quote
change the relay rules so that transactions are replaced based on fees regardless of how that changes transaction outputs.

John Dillon seems to want to make it so that clients won't check outputs are the same before replacing the transaction.

TX replacement of any form is currently disabled.

I previously proposed a safe version that would only add additional inputs and outputs to a transaction, never changing the existing output set, and only if no transactions existed that spent any output of the transaction being replaced. Since no outputs can be replaced, you can't double-spend effectively.

The problem is, since no outputs can be replaced, if you need to change the fee again the transaction gets bigger each time. (making it public knowledge which existing output is change would break privacy) You also can't implement an 'undo' function so users can fix accidental mistakes like cutting and pasting the wrong address.

My second pure replace-by-fee proposal, which I believe is what John Dillon is willing to fund, would replace any unconfirmed transaction with another one if doing so would gain the miner higher overall fees, regardless of the circumstances. A miner following those rules acts in a perfectly economically rational way, at least in the short term. It's been argued that miners have an incentive to not mine double-spends, but I'm unconvinced; each individual miner has nothing to lose by mining a double-spend, and an immediate gain from the fee they collect. It's vastly weaker than the 'suicide-pact' rational miners have to follow the Bitcoin rules, where any deviation means every other node will reject your blocks. On the other hand, the block reward is so high right now miners have little incentive to do anything but use the reference client as-is.

I also wrote on the email list how with 1MB blocks it's pretty safe to assume that broadcasting a transaction means all miners have a copy of it within a few seconds. On the other hand, if we raise the blocksize that assumption isn't going to be true anymore - transaction load will be high enough that nodes have to drop transactions some of the time, which means not all miners will have a copy of every transaction broadcast. Thus it becomes much easier to broadcast a second copy later, double-spending the first.

meowmeowbrowncow
Sr. Member
****
Offline Offline

Activity: 322
Merit: 250



View Profile
April 18, 2013, 01:05:03 PM
 #8

...

Full disclosure: I'm considering writing that patch and collecting that $500 reward myself.



Do It™

"Bitcoin has been an amazing ride, but the most fascinating part to me is the seemingly universal tendency of libertarians to immediately become authoritarians the very moment they are given any measure of power to silence the dissent of others."  - The Bible
amincd
Hero Member
*****
Offline Offline

Activity: 772
Merit: 501


View Profile
April 18, 2013, 03:01:10 PM
 #9

Quote from: retep
Like it or not, zero-conf is dangerous when you don't trust the other party. I wrote the above replace-by-fee idea because I really think we run a risk if we lull people into complacency.

So you want to make it easier to pull off a double spend on a zero-confirmation transaction?

Being able to rely on the zero-confirmation transaction "no-replacement" rule is extremely important for the usability of bitcoin. I think it would take a big hit if it were eliminated.

It would basically mean that unless you can rely on the spender being honest, you cannot accept a zero-conf transaction from them. Right now, all you need to be able to trust is that most miners are honest and are using the standard bitcoin client software with the no-replacement rule. It's been working so far, so I don't see the point of changing it.

What's the worst that can happen if the rule is not revoked? Someone pulls off a double spend? OK, then people will stop relying on zero-conf txs. The price of not doing any thing is possibly a few double spend attacks, probably with a combined value of less than the $500 John Dillon is offering as a bounty. The price of doing something is that people are guaranteed to not be able to use zero-conf txs any more, which would remove millions, if not hundreds of millions of dollars worth of value from bitcoin.

There could also be possible solutions to making zero-conf txs safer from double spends, and by eliminating them altogether, we'll never be able to try them and find out. This proposal is completely unnecessary.

Quote
and it lets us implement a limited 'undo' button for when people screw up.

If you want an undo button, add a feature in the client where after a user presses 'send', a timer starts, that after 10 minutes, fires off the transaction, and an 'undo' button on that timer, that if clicked before the timer reaches zero, cancels the countdown. No need to make double-spending zero-conf transactions trivial.
TierNolan
Legendary
*
Offline Offline

Activity: 1232
Merit: 1104


View Profile
April 18, 2013, 03:11:50 PM
 #10

So you want to make it easier to pull off a double spend on a zero-confirmation transaction?

What about a time delay for fee by replace.  Transactions would only be replaced by a fee if at least 2 blocks and 20 minutes have passed since the original transaction was received.

That would require a queue though, nodes would delay updated transactions until at least the time has passed.

1LxbG5cKXzTwZg9mjL3gaRE835uNQEteWF
jl2012
Legendary
*
Offline Offline

Activity: 1792
Merit: 1111


View Profile
April 18, 2013, 03:19:40 PM
 #11

Quote from: retep
Like it or not, zero-conf is dangerous when you don't trust the other party. I wrote the above replace-by-fee idea because I really think we run a risk if we lull people into complacency.

So you want to make it easier to pull off a double spend on a zero-confirmation transaction?

Being able to rely on the zero-confirmation transaction "no-replacement" rule is extremely important for the usability of bitcoin. I think it would take a big hit if it were eliminated.

It would basically mean that unless you can rely on the spender being honest, you cannot accept a zero-conf transaction from them. Right now, all you need to be able to trust is that most miners are honest and are using the standard bitcoin client software with the no-replacement rule. It's been working so far, so I don't see the point of changing it.

What's the worst that can happen if the rule is not revoked? Someone pulls off a double spend? OK, then people will stop relying on zero-conf txs. The price of not doing any thing is possibly a few double spend attacks, probably with a combined value of less than the $500 John Dillon is offering as a bounty. The price of doing something is that people are guaranteed to not be able to use zero-conf txs any more, which would remove millions, if not hundreds of millions of dollars worth of value from bitcoin.

There could also be possible solutions to making zero-conf txs safer from double spends, and by eliminating them altogether, we'll never be able to try them and find out. This proposal is completely unnecessary.

Quote
and it lets us implement a limited 'undo' button for when people screw up.

If you want an undo button, add a feature in the client where after a user presses 'send', a timer starts, that after 10 minutes, fires off the transaction, and an 'undo' button on that timer, that if clicked before the timer reaches zero, cancels the countdown. No need to make double-spending zero-conf transactions trivial.

A timer is useless because a new block could be found in the next second, or next hour. Anyway, an undo button would be useful.

And this will also solve the SD problem.

Donation address: 374iXxS4BuqFHsEwwxUuH3nvJ69Y7Hqur3 (Bitcoin ONLY)
LRDGENPLYrcTRssGoZrsCT1hngaH3BVkM4 (LTC)
PGP: D3CC 1772 8600 5BB8 FF67 3294 C524 2A1A B393 6517
amincd
Hero Member
*****
Offline Offline

Activity: 772
Merit: 501


View Profile
April 18, 2013, 03:22:32 PM
 #12

What about a time delay for fee by replace.  Transactions would only be replaced by a fee if at least 2 blocks and 20 minutes have passed since the original transaction was received.

I don't see why replacing transactions with output changes is needed.

Quote from: jl2012
A timer is useless because a new block could be found in the next second, or next hour.

When the timer hits zero, the transaction would be created, so it wouldn't matter when the next block was found.

Quote
And this will also solve the SD problem.

What SD problem? More transactions is good for bitcoin..
Maged
Legendary
*
Offline Offline

Activity: 1204
Merit: 1015


View Profile
April 18, 2013, 10:08:07 PM
 #13

Sorry, but I've got to stop this right here before this idea gets out of hand.

On the other hand, this makes changing fees after the fact trivial, and it lets us implement a limited 'undo' button for when people screw up.
You also can't implement an 'undo' function so users can fix accidental mistakes like cutting and pasting the wrong address.
Have you even thought through the implications of this? An "undo" button would train the users into thinking that:
1) Bitcoin transactions can be reversed for a few minutes after they send the transaction.
2) The reversal is guaranteed.

#1 isn't true at all, since any manner of variables can come into play that could ultimately make the undo button useless almost immediately. How would you explain to people that the undo button might work for anywhere between seconds and hours?
As for #2, if a merchant is making a great deal of profit off the transaction, they could secretly pay certain miners to choose the original transaction over the undo transaction.

It also allows for many of the applications transaction replacement was meant for in the first place anyway, and all the applications where it's actually secure.
Tell me, what applications would this allow that a more locked-down transaction replacement system doesn't?

We keep saying over and over again to stop accepting zero-conf transactions, but people do it anyway because it seems secure. It's a very dangerous situation because the security of zero-conf transactions can change overnight simply by some fraction of the hashing power implementing that exact change.
That's because it is somewhat secure! Consider the case where zero-confirmation transactions take the place of existing credit card transactions. A determined attacker can reverse 100% of their credit card transactions, but they would only be able to reverse a small percentage of their zero-confirmation transactions. This would allow casual attackers to reverse most of their zero-confirmation transactions, whereas they can't do that with credit cards as a casual attacker.

Like it or not, zero-conf is dangerous when you don't trust the other party. I wrote the above replace-by-fee idea because I really think we run a risk if we lull people into complacency.
We have the Bitcoin Foundation, don't we? One of their goals should be educating businesses about the responsible handling of zero-conf transactions.

The blockchain and the proof-of-work system is how Bitcoin comes to a consensus about which transactions are or are not valid; trusting anything else is dangerous.

When you accept a zero-conf transaction the method of determining consensus basically comes down to hoping that all miners implement the default "no-replacement" rules, rules that can fail due to a bunch of other reasons like propagation failures. Mining pools these days are run by individuals as a (serious) hobby, and are usually hosted on insecure VPS services. The security of zero-conf transactions can change overnight by one of those pools getting hacked, or anyone with hashing power deciding to change the relay policy they use; about 10% of all blocks have unknown origins.
Yes, but you're confusing absolute security with "good-enough" security. Hence why people still accept credit cards.

Trying to bolt on a second consensus mechanism, like nodes rejecting blocks if there are transactions in them that they haven't seen before, or conflict with existing transactions, is dangerous. That second consensus mechanism becomes a way to attack Bitcoin, and it can be as simple as just broadcasting different transactions to different miners so they don't know what transaction was first.
I agree, but they may be other options that we haven't considered.

The problem is, since no outputs can be replaced, if you need to change the fee again the transaction gets bigger each time. (making it public knowledge which existing output is change would break privacy)
As you eluded to, they can increase the fee by adding a fee to a dependent transaction. As far a breaking privacy, here are a few ideas of preventing that. First, you have to consider the future where merchants would be the one adding the fee, as Mike Hearn has often suggested would happen. In that case, who added the fee? The user, or the merchant? If that's not good enough for you, we can add a third output that is fairly small and use that to add dependent fees.


It's been argued that miners have an incentive to not mine double-spends, but I'm unconvinced; each individual miner has nothing to lose by mining a double-spend, and an immediate gain from the fee they collect.
Not at all, and you have the invention of ASICs to thank for that. Mining now requires a large up-front investment that would be completely useless if Bitcoin were to collapse, unlike when we were in the age of GPU mining. Miners have an interest in having Bitcoin be used in as many use-cases as possible.

I also wrote on the email list how with 1MB blocks it's pretty safe to assume that broadcasting a transaction means all miners have a copy of it within a few seconds. On the other hand, if we raise the blocksize that assumption isn't going to be true anymore - transaction load will be high enough that nodes have to drop transactions some of the time, which means not all miners will have a copy of every transaction broadcast. Thus it becomes much easier to broadcast a second copy later, double-spending the first.
Again, thanks to ASICs, mining is a serious operation. Miners will hold onto as many transactions as possible, and they will use enterprise-grade equipment to do so.

grue
Legendary
*
Offline Offline

Activity: 2058
Merit: 1446



View Profile
April 18, 2013, 10:21:26 PM
 #14

Quote
And this will also solve the SD problem.

What SD problem? More transactions is good for bitcoin..
no it isn't. the last thing we need is SD accounting for 50% of the transactions and slowing down the confirmation time of legitimate transactions.

It is pitch black. You are likely to be eaten by a grue.

Adblock for annoying signature ads | Enhanced Merit UI
justusranvier
Legendary
*
Offline Offline

Activity: 1400
Merit: 1013



View Profile
April 18, 2013, 11:29:01 PM
 #15

When you accept a zero-conf transaction the method of determining consensus basically comes down to hoping that all miners implement the default "no-replacement" rules, rules that can fail due to a bunch of other reasons like propagation failures. Mining pools these days are run by individuals as a (serious) hobby, and are usually hosted on insecure VPS services. The security of zero-conf transactions can change overnight by one of those pools getting hacked, or anyone with hashing power deciding to change the relay policy they use; about 10% of all blocks have unknown origins.
Your description of the problem contains within it the solution you are ignoring or not seeing.

Because the mining pools are run by individuals, and because merchants have an economic incentive to avoid double spending attacks, and because the individuals operating mining pools like money, there is a way for the merchants and the pool operators to work out an arrangement that is beneficial to all parties.

It's you've spent too much time playing The Sims and forget that both merchants and pool operators are sentient, intelligent beings instead of automatons.

If the risks of zero conf double spends are worth expending resources to reduce or eliminate then the merchants will find a way to get it done. There exists nothing that would make a solution impossible, so it will be implemented when it makes sense economically.
kwukduck
Legendary
*
Offline Offline

Activity: 1937
Merit: 1001


View Profile
April 19, 2013, 03:25:58 AM
 #16

I don't think allowing to change the original output is a good option (adding new ones is fine), other than that i think it's a good proposal.

14b8PdeWLqK3yi3PrNHMmCvSmvDEKEBh3E
jdillon
Member
**
Offline Offline

Activity: 70
Merit: 18


View Profile
April 19, 2013, 04:28:46 AM
 #17

retep: To clarify the $500USD reward is meant to be for a proof-of-concept implementation. To collect it you do not need to implement unit-tests or recursive fee computation. You also do not need to make the undo RPC command do anything more than broadcast a replacement with a single output going to yourself. It is OK if the wallet code doesn't handle the undo nicely. I will consider offering further rewards if the initial one works out.

maged: I'm not offering this reward because I think an undo button is important. That feature is just an interesting side effect and yes it's one that users will likely misunderstand. The problem is people like you justus and Mike Hearn will be more than happy to screw up Bitcoin in a desperate attempt to stop double spends when it becomes a big issue. You all have this vision of mining pools signing each others blocks, making commitments to only mine chains with certain transactions and other centralized crap. If achieving consensus in a distributed fashion was that easy Bitcoin wouldn't need a proof-of-work system.

By breaking zero-conf security now there won't be pressure to implement all that crap. The most badly affected will be Satoshidice and they should not be using the blockchain the way they do.

retep and others have pointed out a few times that ASICs are actually more cost effective for small scale mining than large. You are also naive for thinking that some up-front investment will somehow make people act altruistically.

justusranvier: Zero-conf double spends will be fixed but not by screwing up Bitcoin's decentralization.
im3w1l
Sr. Member
****
Offline Offline

Activity: 280
Merit: 250


View Profile
April 19, 2013, 04:30:51 AM
 #18

-1 on this idea.

Also bribing miners to replace a TX is a horrible precedent.
etotheipi
Legendary
*
expert
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
April 19, 2013, 04:57:41 AM
 #19

Like jdillon, I believe that in the long term, many miners will allow paid replacements of transactions and zero-conf transactions will become as useless as what we're afraid of.  You can talk about ethics, and what's in the "best interest of miners", but that is just wishful thinking that in a completely-decentralized system everyone will have the same ethics and motives.  I'd rather just see it happen and let the ecosystem adjust to the loss of remaining zero-conf security/sanity, instead of naively hope that everyone will follow the same guidelines that are not bound to follow.  Especially when there is economic incentive to breaking these guidelines.  Not all miners are dependent on the security of zero-conf transactions.  Many of them will just do what's best for their bottom line.

I've seen the phrase "allow" when referring to miners replacing zero-conf transactions.  Above, im3w1l mentioned "setting a precedent".  This is meaningless, because no one has control over all the miners, and they don't need to seek anyone's permission to do something that is entirely within the rules of the system.  The best we can do is "recommend" guidelines by making it part of the default client, but that's it.  It's part of the blessing&curse of being decentralized.  Sure, a lot of miners won't do it.  But some will, and you only need any to do it, in order for it to dramatically degrade this system.

Therefore, we are adapting ourselves (and letting others adapt) to a false reality by designing systems with an assumption that there is some security in zero-conf transactions.  I'd much rather just write it off completely, and let businesses and users adapt to the idea that zero-conf transactions are basically useless for exchanges between untrusted parties.  Forget it.  If you don't trust the person, don't mess with zero-confirmation transactions.  Period.



Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
im3w1l
Sr. Member
****
Offline Offline

Activity: 280
Merit: 250


View Profile
April 19, 2013, 05:04:43 AM
 #20

If thinking like this starts creeping in than we are on a slippery slope. Why not reverse a 1-Conf transaction, if the pay is good? I think we should try to nip it in the bud. Encourage good behavior by orphaning transaction reversal blocks.
Pages: [1] 2 3 4 5 6 7 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!