Reminder: zero-conf is not safe; $1000USD reward posted for replace-by-fee patch

[Peter Todd]

EDIT: As it turns out replace-by-fee will eventually allow for fairly safe zero-confirmation transactions, ironic really: https://bitcointalk.org/index.php?topic=251233.msg2669189#msg2669189


Someone by the name of John Dillon (john.dillon892@googlemail.com) emailed the bitcoin-development email list earlier this morning offering a $500USD reward to anyone who implements a transaction replacement-by-fee patch. That's an idea I posted on the email list two days ago:

In any case, the more pressing issue re: replacement is changing fees attached to transactions after they have been broadcast. Lots of users are getting their transactions stuck with few options to fix them.

The more I think about the issue the more I think we should nip this zero-conf madness in the bud: change the relay rules so that transactions are replaced based on fees regardless of how that changes transaction outputs. Of course, this does make double-spending an unconfirmed transaction trivial. On the other hand, this makes changing fees after the fact trivial, and it lets us implement a limited 'undo' button for when people screw up. It also allows for many of the applications transaction replacement was meant for in the first place anyway, and all the applications where it's actually secure.

We keep saying over and over again to stop accepting zero-conf transactions, but people do it anyway because it seems secure. It's a very dangerous situation because the security of zero-conf transactions can change overnight simply by some fraction of the hashing power implementing that exact change.

Some thought is required as to exactly what "replace by fees" looks like, economically optimal is a bit complex due to it's dependency on overall mempool backlog, but a rough first version should be easy to hammer out.

-Re: [Bitcoin-development] [bitcoin] Enable tx replacement on testnet. (#2516)

Like it or not, zero-conf is dangerous when you don't trust the other party. I wrote the above replace-by-fee idea because I really think we run a risk if we lull people into complacency. The blockchain and the proof-of-work system is how Bitcoin comes to a consensus about which transactions are or are not valid; trusting anything else is dangerous.

When you accept a zero-conf transaction the method of determining consensus basically comes down to hoping that all miners implement the default "no-replacement" rules, rules that can fail due to a bunch of other reasons like propagation failures. Mining pools these days are run by individuals as a (serious) hobby, and are usually hosted on insecure VPS services. The security of zero-conf transactions can change overnight by one of those pools getting hacked, or anyone with hashing power deciding to change the relay policy they use; about 10% of all blocks have unknown origins.

Trying to bolt on a second consensus mechanism, like nodes rejecting blocks if there are transactions in them that they haven't seen before, or conflict with existing transactions, is dangerous. That second consensus mechanism becomes a way to attack Bitcoin, and it can be as simple as just broadcasting different transactions to different miners so they don't know what transaction was first.

Full disclosure: I'm considering writing that patch and collecting that $1000 reward myself.

EDIT: reward has increased

[1714316630]

1714316630

[1714316630]

1714316630

[1714316630]

1714316630

[Remember remember the 5th of November]

In a nutshell, what does this mean?

[TierNolan]

In a nutshell, what does this mean?

If I tell my client to pay you, you will receive notification of that transaction from the network.

This means that my transaction has flooded the network, and more importantly, flooded to all miners.

If I then send a transaction that spends the input back to myself, the network won't relay it and miners will ignore it.

So, if you receive the transaction from lots of neighbors, you can be reasonably sure I broadcast it to the network.

The proposal allows replacement of transactions.  If I resend the transaction, but with a higher fee, it is replaced with the new transaction.  This doesn't allow me to reverse the transaction though.

In addition, the proposal, lets me change the outputs, as long as I up the fee.

This means I can send a tx to you with 0.01 fee and then revers with with a 0.02 fee.

All this requires that it hasn't been incorporated in a block.  Once it is in a block, then it is locked down (unless that block is orphaned).

Full replacement allows people to update a transaction if it is not being included in the chain.  It would also act to discourage people accepting transactions, until at least one block confirmation happens, since until it is confirmed, it is easy to reverse.

[Remember remember the 5th of November]

Makes sense. Any other drawbacks?

[🏰 TradeFortress 🏰]

Isn't this already done?

change the relay rules so that transactions are replaced based on fees regardless of how that changes transaction outputs.

John Dillon seems to want to make it so that clients won't check outputs are the same before replacing the transaction.

[Zeilap]

Isn't this already done?

change the relay rules so that transactions are replaced based on fees regardless of how that changes transaction outputs.

John Dillon seems to want to make it so that clients won't check outputs are the same before replacing the transaction.

With the same inputs, the total value of the transaction is the same. Changing the fees means that the total value must be redistributed in a different way to before. Thus some outputs must get less if the fees are to get more.
I'm assuming he doesn't mean to allow adding or removing outputs.

[Peter Todd]

Isn't this already done?

change the relay rules so that transactions are replaced based on fees regardless of how that changes transaction outputs.

John Dillon seems to want to make it so that clients won't check outputs are the same before replacing the transaction.

TX replacement of any form is currently disabled.

I previously proposed a safe version that would only add additional inputs and outputs to a transaction, never changing the existing output set, and only if no transactions existed that spent any output of the transaction being replaced. Since no outputs can be replaced, you can't double-spend effectively.

The problem is, since no outputs can be replaced, if you need to change the fee again the transaction gets bigger each time. (making it public knowledge which existing output is change would break privacy) You also can't implement an 'undo' function so users can fix accidental mistakes like cutting and pasting the wrong address.

My second pure replace-by-fee proposal, which I believe is what John Dillon is willing to fund, would replace any unconfirmed transaction with another one if doing so would gain the miner higher overall fees, regardless of the circumstances. A miner following those rules acts in a perfectly economically rational way, at least in the short term. It's been argued that miners have an incentive to not mine double-spends, but I'm unconvinced; each individual miner has nothing to lose by mining a double-spend, and an immediate gain from the fee they collect. It's vastly weaker than the 'suicide-pact' rational miners have to follow the Bitcoin rules, where any deviation means every other node will reject your blocks. On the other hand, the block reward is so high right now miners have little incentive to do anything but use the reference client as-is.

I also wrote on the email list how with 1MB blocks it's pretty safe to assume that broadcasting a transaction means all miners have a copy of it within a few seconds. On the other hand, if we raise the blocksize that assumption isn't going to be true anymore - transaction load will be high enough that nodes have to drop transactions some of the time, which means not all miners will have a copy of every transaction broadcast. Thus it becomes much easier to broadcast a second copy later, double-spending the first.

[meowmeowbrowncow]

...

Full disclosure: I'm considering writing that patch and collecting that $500 reward myself.

Do It™

[amincd]

Like it or not, zero-conf is dangerous when you don't trust the other party. I wrote the above replace-by-fee idea because I really think we run a risk if we lull people into complacency.

So you want to make it easier to pull off a double spend on a zero-confirmation transaction?

Being able to rely on the zero-confirmation transaction "no-replacement" rule is extremely important for the usability of bitcoin. I think it would take a big hit if it were eliminated.

It would basically mean that unless you can rely on the spender being honest, you cannot accept a zero-conf transaction from them. Right now, all you need to be able to trust is that most miners are honest and are using the standard bitcoin client software with the no-replacement rule. It's been working so far, so I don't see the point of changing it.

What's the worst that can happen if the rule is not revoked? Someone pulls off a double spend? OK, then people will stop relying on zero-conf txs. The price of not doing any thing is possibly a few double spend attacks, probably with a combined value of less than the $500 John Dillon is offering as a bounty. The price of doing something is that people are guaranteed to not be able to use zero-conf txs any more, which would remove millions, if not hundreds of millions of dollars worth of value from bitcoin.

There could also be possible solutions to making zero-conf txs safer from double spends, and by eliminating them altogether, we'll never be able to try them and find out. This proposal is completely unnecessary.

and it lets us implement a limited 'undo' button for when people screw up.

If you want an undo button, add a feature in the client where after a user presses 'send', a timer starts, that after 10 minutes, fires off the transaction, and an 'undo' button on that timer, that if clicked before the timer reaches zero, cancels the countdown. No need to make double-spending zero-conf transactions trivial.

[TierNolan]

So you want to make it easier to pull off a double spend on a zero-confirmation transaction?

What about a time delay for fee by replace.  Transactions would only be replaced by a fee if at least 2 blocks and 20 minutes have passed since the original transaction was received.

That would require a queue though, nodes would delay updated transactions until at least the time has passed.

[jl2012]

Like it or not, zero-conf is dangerous when you don't trust the other party. I wrote the above replace-by-fee idea because I really think we run a risk if we lull people into complacency.

So you want to make it easier to pull off a double spend on a zero-confirmation transaction?

Being able to rely on the zero-confirmation transaction "no-replacement" rule is extremely important for the usability of bitcoin. I think it would take a big hit if it were eliminated.

It would basically mean that unless you can rely on the spender being honest, you cannot accept a zero-conf transaction from them. Right now, all you need to be able to trust is that most miners are honest and are using the standard bitcoin client software with the no-replacement rule. It's been working so far, so I don't see the point of changing it.

What's the worst that can happen if the rule is not revoked? Someone pulls off a double spend? OK, then people will stop relying on zero-conf txs. The price of not doing any thing is possibly a few double spend attacks, probably with a combined value of less than the $500 John Dillon is offering as a bounty. The price of doing something is that people are guaranteed to not be able to use zero-conf txs any more, which would remove millions, if not hundreds of millions of dollars worth of value from bitcoin.

There could also be possible solutions to making zero-conf txs safer from double spends, and by eliminating them altogether, we'll never be able to try them and find out. This proposal is completely unnecessary.

and it lets us implement a limited 'undo' button for when people screw up.

If you want an undo button, add a feature in the client where after a user presses 'send', a timer starts, that after 10 minutes, fires off the transaction, and an 'undo' button on that timer, that if clicked before the timer reaches zero, cancels the countdown. No need to make double-spending zero-conf transactions trivial.

A timer is useless because a new block could be found in the next second, or next hour. Anyway, an undo button would be useful.

And this will also solve the SD problem.

[amincd]

What about a time delay for fee by replace.  Transactions would only be replaced by a fee if at least 2 blocks and 20 minutes have passed since the original transaction was received.

I don't see why replacing transactions with output changes is needed.

A timer is useless because a new block could be found in the next second, or next hour.

When the timer hits zero, the transaction would be created, so it wouldn't matter when the next block was found.

And this will also solve the SD problem.

What SD problem? More transactions is good for bitcoin..

[Maged]

Sorry, but I've got to stop this right here before this idea gets out of hand.

On the other hand, this makes changing fees after the fact trivial, and it lets us implement a limited 'undo' button for when people screw up.

You also can't implement an 'undo' function so users can fix accidental mistakes like cutting and pasting the wrong address.

Have you even thought through the implications of this? An "undo" button would train the users into thinking that:
1) Bitcoin transactions can be reversed for a few minutes after they send the transaction.
2) The reversal is guaranteed.

#1 isn't true at all, since any manner of variables can come into play that could ultimately make the undo button useless almost immediately. How would you explain to people that the undo button might work for anywhere between seconds and hours?
As for #2, if a merchant is making a great deal of profit off the transaction, they could secretly pay certain miners to choose the original transaction over the undo transaction.

It also allows for many of the applications transaction replacement was meant for in the first place anyway, and all the applications where it's actually secure.

Tell me, what applications would this allow that a more locked-down transaction replacement system doesn't?

We keep saying over and over again to stop accepting zero-conf transactions, but people do it anyway because it seems secure. It's a very dangerous situation because the security of zero-conf transactions can change overnight simply by some fraction of the hashing power implementing that exact change.

That's because it is somewhat secure! Consider the case where zero-confirmation transactions take the place of existing credit card transactions. A determined attacker can reverse 100% of their credit card transactions, but they would only be able to reverse a small percentage of their zero-confirmation transactions. This would allow casual attackers to reverse most of their zero-confirmation transactions, whereas they can't do that with credit cards as a casual attacker.

Like it or not, zero-conf is dangerous when you don't trust the other party. I wrote the above replace-by-fee idea because I really think we run a risk if we lull people into complacency.

We have the Bitcoin Foundation, don't we? One of their goals should be educating businesses about the responsible handling of zero-conf transactions.

The blockchain and the proof-of-work system is how Bitcoin comes to a consensus about which transactions are or are not valid; trusting anything else is dangerous.

When you accept a zero-conf transaction the method of determining consensus basically comes down to hoping that all miners implement the default "no-replacement" rules, rules that can fail due to a bunch of other reasons like propagation failures. Mining pools these days are run by individuals as a (serious) hobby, and are usually hosted on insecure VPS services. The security of zero-conf transactions can change overnight by one of those pools getting hacked, or anyone with hashing power deciding to change the relay policy they use; about 10% of all blocks have unknown origins.

Yes, but you're confusing absolute security with "good-enough" security. Hence why people still accept credit cards.

Trying to bolt on a second consensus mechanism, like nodes rejecting blocks if there are transactions in them that they haven't seen before, or conflict with existing transactions, is dangerous. That second consensus mechanism becomes a way to attack Bitcoin, and it can be as simple as just broadcasting different transactions to different miners so they don't know what transaction was first.

I agree, but they may be other options that we haven't considered.

The problem is, since no outputs can be replaced, if you need to change the fee again the transaction gets bigger each time. (making it public knowledge which existing output is change would break privacy)

As you eluded to, they can increase the fee by adding a fee to a dependent transaction. As far a breaking privacy, here are a few ideas of preventing that. First, you have to consider the future where merchants would be the one adding the fee, as Mike Hearn has often suggested would happen. In that case, who added the fee? The user, or the merchant? If that's not good enough for you, we can add a third output that is fairly small and use that to add dependent fees.

It's been argued that miners have an incentive to not mine double-spends, but I'm unconvinced; each individual miner has nothing to lose by mining a double-spend, and an immediate gain from the fee they collect.

Not at all, and you have the invention of ASICs to thank for that. Mining now requires a large up-front investment that would be completely useless if Bitcoin were to collapse, unlike when we were in the age of GPU mining. Miners have an interest in having Bitcoin be used in as many use-cases as possible.

I also wrote on the email list how with 1MB blocks it's pretty safe to assume that broadcasting a transaction means all miners have a copy of it within a few seconds. On the other hand, if we raise the blocksize that assumption isn't going to be true anymore - transaction load will be high enough that nodes have to drop transactions some of the time, which means not all miners will have a copy of every transaction broadcast. Thus it becomes much easier to broadcast a second copy later, double-spending the first.

Again, thanks to ASICs, mining is a serious operation. Miners will hold onto as many transactions as possible, and they will use enterprise-grade equipment to do so.

[grue]

And this will also solve the SD problem.

What SD problem? More transactions is good for bitcoin..

no it isn't. the last thing we need is SD accounting for 50% of the transactions and slowing down the confirmation time of legitimate transactions.

[justusranvier]

When you accept a zero-conf transaction the method of determining consensus basically comes down to hoping that all miners implement the default "no-replacement" rules, rules that can fail due to a bunch of other reasons like propagation failures. Mining pools these days are run by individuals as a (serious) hobby, and are usually hosted on insecure VPS services. The security of zero-conf transactions can change overnight by one of those pools getting hacked, or anyone with hashing power deciding to change the relay policy they use; about 10% of all blocks have unknown origins.

Your description of the problem contains within it the solution you are ignoring or not seeing.

Because the mining pools are run by individuals, and because merchants have an economic incentive to avoid double spending attacks, and because the individuals operating mining pools like money, there is a way for the merchants and the pool operators to work out an arrangement that is beneficial to all parties.

It's you've spent too much time playing The Sims and forget that both merchants and pool operators are sentient, intelligent beings instead of automatons.

If the risks of zero conf double spends are worth expending resources to reduce or eliminate then the merchants will find a way to get it done. There exists nothing that would make a solution impossible, so it will be implemented when it makes sense economically.

[kwukduck]

I don't think allowing to change the original output is a good option (adding new ones is fine), other than that i think it's a good proposal.

[jdillon]

retep: To clarify the $500USD reward is meant to be for a proof-of-concept implementation. To collect it you do not need to implement unit-tests or recursive fee computation. You also do not need to make the undo RPC command do anything more than broadcast a replacement with a single output going to yourself. It is OK if the wallet code doesn't handle the undo nicely. I will consider offering further rewards if the initial one works out.

maged: I'm not offering this reward because I think an undo button is important. That feature is just an interesting side effect and yes it's one that users will likely misunderstand. The problem is people like you justus and Mike Hearn will be more than happy to screw up Bitcoin in a desperate attempt to stop double spends when it becomes a big issue. You all have this vision of mining pools signing each others blocks, making commitments to only mine chains with certain transactions and other centralized crap. If achieving consensus in a distributed fashion was that easy Bitcoin wouldn't need a proof-of-work system.

By breaking zero-conf security now there won't be pressure to implement all that crap. The most badly affected will be Satoshidice and they should not be using the blockchain the way they do.

retep and others have pointed out a few times that ASICs are actually more cost effective for small scale mining than large. You are also naive for thinking that some up-front investment will somehow make people act altruistically.

justusranvier: Zero-conf double spends will be fixed but not by screwing up Bitcoin's decentralization.

[im3w1l]

-1 on this idea.

Also bribing miners to replace a TX is a horrible precedent.

[etotheipi]

Like jdillon, I believe that in the long term, many miners will allow paid replacements of transactions and zero-conf transactions will become as useless as what we're afraid of.  You can talk about ethics, and what's in the "best interest of miners", but that is just wishful thinking that in a completely-decentralized system everyone will have the same ethics and motives.  I'd rather just see it happen and let the ecosystem adjust to the loss of remaining zero-conf security/sanity, instead of naively hope that everyone will follow the same guidelines that are not bound to follow.  Especially when there is economic incentive to breaking these guidelines.  Not all miners are dependent on the security of zero-conf transactions.  Many of them will just do what's best for their bottom line.

I've seen the phrase "allow" when referring to miners replacing zero-conf transactions.  Above, im3w1l mentioned "setting a precedent".  This is meaningless, because no one has control over all the miners, and they don't need to seek anyone's permission to do something that is entirely within the rules of the system.  The best we can do is "recommend" guidelines by making it part of the default client, but that's it.  It's part of the blessing&curse of being decentralized.  Sure, a lot of miners won't do it.  But some will, and you only need any to do it, in order for it to dramatically degrade this system.

Therefore, we are adapting ourselves (and letting others adapt) to a false reality by designing systems with an assumption that there is some security in zero-conf transactions.  I'd much rather just write it off completely, and let businesses and users adapt to the idea that zero-conf transactions are basically useless for exchanges between untrusted parties.  Forget it.  If you don't trust the person, don't mess with zero-confirmation transactions.  Period.

[im3w1l]

If thinking like this starts creeping in than we are on a slippery slope. Why not reverse a 1-Conf transaction, if the pay is good? I think we should try to nip it in the bud. Encourage good behavior by orphaning transaction reversal blocks.

[etotheipi]

If thinking like this starts creeping in than we are on a slippery slope. Why not reverse a 1-Conf transaction, if the pay is good? I think we should try to nip it in the bud. Encourage good behavior by orphaning transaction reversal blocks.

Reversing 1-confirmation transaction is almost always economically unfavorable.  You don't need to discourage that, because miners are bleeding money for every second they aren't mining off the top block.  

And as I said ... you can encourage, wish, complain, etc, all you want, but if it goes against their bottom line, it's not going to make a bit of difference if they're acting within the prescribed rules of the system (which is that there is no economic incentive not to do this)  So it will be done.

On the other hand, if we implement something that makes it economically infeasible, then that's a different story.  But you can't regulate this problem away.  You have to adjust the rules of the system and let it reach equilibrium, which hopefully doesn't include that behavior.  But I'm not sure if this is something we can achieve.

EDIT: about your "orphaning transasction reversal blocks":  there's no way to do that with zero-confirmation transactions.  For 1-conf, it would be possible, and if you hit a critical mass of miners willing to reduce their effective hash rate, they might be willing to do it.  But again, all miners have the incentive to mine off the top block.  if they are not mining the top block, they are losing money.  (enter caveats about extreme circumstances like someone putting a 200 BTC fee on a tx to try to out-spend that economic motive).

[AlphaWolf]

If you accept zero-conf transactions, you're accepting the risk of being screwed.  Zero-conf is the equivalent of "The check is in the mail!".    It's worthless.  You accept the risk, you accept the consequences.

[Nagato]

Like jdillon, I believe that in the long term, many miners will allow paid replacements of transactions and zero-conf transactions will become as useless as what we're afraid of.  You can talk about ethics, and what's in the "best interest of miners", but that is just wishful thinking that in a completely-decentralized system everyone will have the same ethics and motives.  I'd rather just see it happen and let the ecosystem adjust to the loss of remaining zero-conf security/sanity, instead of naively hope that everyone will follow the same guidelines that are not bound to follow.  Especially when there is economic incentive to breaking these guidelines.  Not all miners are dependent on the security of zero-conf transactions.  Many of them will just do what's best for their bottom line.

I've seen the phrase "allow" when referring to miners replacing zero-conf transactions.  Above, im3w1l mentioned "setting a precedent".  This is meaningless, because no one has control over all the miners, and they don't need to seek anyone's permission to do something that is entirely within the rules of the system.  The best we can do is "recommend" guidelines by making it part of the default client, but that's it.  It's part of the blessing&curse of being decentralized.  Sure, a lot of miners won't do it.  But some will, and you only need any to do it, in order for it to dramatically degrade this system.

Therefore, we are adapting ourselves (and letting others adapt) to a false reality by designing systems with an assumption that there is some security in zero-conf transactions.  I'd much rather just write it off completely, and let businesses and users adapt to the idea that zero-conf transactions are basically useless for exchanges between untrusted parties.  Forget it.  If you don't trust the person, don't mess with zero-confirmation transactions.  Period.

This.

Ive always felt that some people on this forum live in a fantasy world and assume ethics and charity can make the world go round. Like Mike Hearn's belief that NACs can fund the security of a infinite-sized blockchain.

[jdillon]

Ive always felt that some people on this forum live in a fantasy world and assume ethics and charity can make the world go round. Like Mike Hearn's belief that NACs can fund the security of a infinite-sized blockchain.

What is so brilliant about Bitcoin is how when you use it you put the absolute minimum of trust in others. You validate everything on the network and the one thing you let others decide is the order of transactions, and that decision is done with a democratic vote. (an odd type of vote similar to the two-party system in dynamic behavior but it is a vote in essence) Having looked into this nSquence transaction replacement stuff I am not so sure Satoshi really understood what he created but never the less Bitcoin is what it is.

I and some partners have known about Bitcoin for some time and we have owned coins as a long term investment for almost as long. (for what it's worth: 2409f355c8910721fbbb5c54a01b8f9c692cfb292c3b4f7baf5b8151e44fef21 gmaxwell would understand) I have not participated much due other commitments. I also haven't been a programmer for quite awhile. But recently retep's posts about decentralization off-chain transactions and the blocksize have I guess woken me up. Replace-by-fee is just a small thing, but I see it as an important step to getting people to understand how Bitcoin really works. As etotheipi says it is the blessing and the curse of decentralization, but we can adapt and gain the benefits of true decentralization.

Having said that I have some advice for you Peter Todd: Write some code for once or people will never take you seriously. You appear to have an English degree rather than a Computer Science degree.

[TierNolan]

Not all miners are dependent on the security of zero-conf transactions.  Many of them will just do what's best for their bottom line.

The incentives might be fixable by a rule change.  For example, if the rule was to not build on a block that has a double spend for 30 seconds, unless the old transaction is at least 24 hours old, then miners who broadcast those blocks are hurt.  The incentives for a miner is to always include the transactions that they see first, since those are likely to to be one that the other miners saw first.  If anything it would create an incentive not to include either of them.

It also creates an incentive to distribute info about double spends between miners.

[amincd]

Ive always felt that some people on this forum live in a fantasy world and assume ethics and charity can make the world go round.

Why assume they won't? Let's assume nothing. So far, 4 years in, zero-conf transactions have been working. They might start to fail one day, they might not. Why not let the market decide whether to accept zero-conf transactions, rather than going out of our way to make zero-conf transactions unviable?

[Nagato]

Ive always felt that some people on this forum live in a fantasy world and assume ethics and charity can make the world go round.

Why assume they won't? Let's assume nothing. So far, 4 years in, zero-conf transactions have been working. They might start to fail one day, they might not. Why not let the market decide whether to accept zero-conf transactions, rather than going out of our way to make zero-conf transactions unviable?

I actually support the replace-by-fee feature as a usability enhancement(you sent to the wrong address accidentally and you can still attempt to undo that before it's inclusion in any block) instead of trying to change people's behaviour. I agree with you, the market will force people to adapt when people start double spending txns with or without this change.

[🏰 TradeFortress 🏰]

And this will also solve the SD problem.

What SD problem? More transactions is good for bitcoin..

no it isn't. the last thing we need is SD accounting for 50% of the transactions and slowing down the confirmation time of legitimate transactions.

Are you going to say the same thing when Western Union accepts bitcoin or what and they account for 50% of the transactions? Are they legitimate but SatoshiDice is?

[thanke]

Isn't replace-by-fee incompatible with miners grouping transactions and evaluating the group-fee (groups of transactions depending on each other)? Because the owner of any output of the transaction can easily and arbitrarily increase the group-fee associated to any of the previous versions of the transaction. Since group-fee evaluation is inevitable, I don't see how replace-by-fee can work.

[caveden]

It's a very dangerous situation because the security of zero-conf transactions can change overnight simply by some fraction of the hashing power implementing that exact change.

Therefore, we are adapting ourselves (and letting others adapt) to a false reality by designing systems with an assumption that there is some security in zero-conf transactions.  I'd much rather just write it off completely, and let businesses and users adapt to the idea that zero-conf transactions are basically useless for exchanges between untrusted parties.  Forget it.  If you don't trust the person, don't mess with zero-confirmation transactions.  Period.

Those are very good points.

Full disclosure: I'm considering writing that patch and collecting that $500 reward myself.

Such patch would not be that useful if it's not used by most relays and at least a few generators. But it's a start anyway.

Have you even thought through the implications of this? An "undo" button would train the users into thinking that:
1) Bitcoin transactions can be reversed for a few minutes after they send the transaction.
2) The reversal is guaranteed.

#1 isn't true at all, since any manner of variables can come into play that could ultimately make the undo button useless almost immediately. How would you explain to people that the undo button might work for anywhere between seconds and hours?

It's sort of like GMail undo when you click Send. You have a few moments to change your mind, but once the mail is gone, you can't bring it back.
An eventual undo button should be disabled as soon as a confirmation is seen.

As for #2, if a merchant is making a great deal of profit off the transaction, they could secretly pay certain miners to choose the original transaction over the undo transaction.

Good point. Warnings would be welcome.

Not at all, and you have the invention of ASICs to thank for that. Mining now requires a large up-front investment that would be completely useless if Bitcoin were to collapse

Come on, you must admit that some double-spent of 0-conf transactions would never make Bitcoin collapse, that's an exaggeration. Particularly if people understand that a 0-conf tx can be easily undone.

[xanatos]

What happens if a block becomes orphan? Its transactions are readded to the transaction pool, so they could be changed by the sender... So you would only need to wait for a split in the network to double spend your money?

[TierNolan]

Another compromise rule would be that double spending would result in both transaction being removed from the memory pool.  The one with the higher fee would be placed in a 1 hour delay queue before being included and the lower fee one would be forgotten after 1 hour (or maybe 6 - 10 blocks).

Both would still be propagated though, with the second one received being flagged as a double spend.  Therefore all nodes on the network would have both transactions removed from the main memory pool and placed in the pending/to be discarded memory pool/queue.

The disadvantage is that the 2nd transaction is propagated.  However, the merchant would have a chance to see the double spend notification.

[caveden]

What happens if a block becomes orphan? Its transactions are readded to the transaction pool, so they could be changed by the sender... So you would only need to wait for a split in the network to double spend your money?

I've never analysed the data myself, but I'd guess that honest splits tend to carry almost (if not exactly) the same transactions on each side of the split.

[dserrano5]

Another compromise rule would be that double spending would result in both transaction being removed from the memory pool.

1) Go to the counter
2) Get a Whopper®
3) Pay with bitcoin
4) Go out
5) Attempt a double-spend, now both txs are removed from the pool
6) Enjoy your meal

[kjj]

Not all miners are dependent on the security of zero-conf transactions.  Many of them will just do what's best for their bottom line.

The incentives might be fixable by a rule change.  For example, if the rule was to not build on a block that has a double spend for 30 seconds, unless the old transaction is at least 24 hours old, then miners who broadcast those blocks are hurt.  The incentives for a miner is to always include the transactions that they see first, since those are likely to to be one that the other miners saw first.  If anything it would create an incentive not to include either of them.

It also creates an incentive to distribute info about double spends between miners.

Better rule changes have been proposed for better reasons, all rejected.  Chain validation is very nearly stateless for a reason.

[TierNolan]

1) Go to the counter
2) Get a Whopper®
3) Pay with bitcoin
4) Go out
5) Attempt a double-spend, now both txs are removed from the pool
6) Enjoy your meal

It seemed like a good idea, at the time .

[jgarzik]

Not at all, and you have the invention of ASICs to thank for that. Mining now requires a large up-front investment that would be completely useless if Bitcoin were to collapse

Come on, you must admit that some double-spent of 0-conf transactions would never make Bitcoin collapse, that's an exaggeration. Particularly if people understand that a 0-conf tx can be easily undone.

More to the point, zero-conf transactions have been double-spent already.  It is proven they are not safe today, ignoring any proposed changes.

[justusranvier]

It is proven they are not safe today, ignoring any proposed changes.

Not safe compared to what?

Most merchants out in the real world already accept payment methods that can be trivially reversed and manage to make it work.

[kjj]

It is proven they are not safe today, ignoring any proposed changes.

Not safe compared to what?

Most merchants out in the real world already accept payment methods that can be trivially reversed and manage to make it work.

Not safe compared to how safe people think they are, of course.

[Mike Hearn]

I am not aware of any merchant that has ever been double spent with 0-conf transactions except the OKPAY example during the chain split. Which was almost certainly caused by mining nodes being restarted and not syncing their mempools - quite easy to fix.

[meowmeowbrowncow]

Not at all, and you have the invention of ASICs to thank for that. Mining now requires a large up-front investment that would be completely useless if Bitcoin were to collapse

Come on, you must admit that some double-spent of 0-conf transactions would never make Bitcoin collapse, that's an exaggeration. Particularly if people understand that a 0-conf tx can be easily undone.

More to the point, zero-conf transactions have been double-spent already.  It is proven they are not safe today, ignoring any proposed changes.

Exactly.  Why are we arguing about 0 conf tx?  They are unsafe to begin with.


wtf? 

[warpio]

Question: Would just 1 confirmation be safe if you were to confirm it yourself on your own local copy of the blockchain? Would you not need to wait for 6 confirmations in that case?

I don't see any reason why it wouldn't be safe, or is there something I'm missing?

[caveden]

I am not aware of any merchant that has ever been double spent with 0-conf transactions except the OKPAY example during the chain split.

OKPay was not victim of a 0-conf, but a much more serious situation (the huge reorg caused by the 0.7 bug).
I think at least Satoshi Dice has already been the victim of a 0-conf double-spend.
But yeah, they are rare to the point of being negligible, if you compare them with CC fraud.

That doesn't change OP's point though: it's only safe because most miners are behaving as per the default bitcoind implementation. There's no strong guarantee that will remain being the case for long. (even then 0-conf would still remain relatively safe for many use cases, for example when you know your customer or when you can suspend your service easily)

[Mike Hearn]

There's no guarantee, but Satoshi's paper addresses the dynamics of this - rational miners shouldn't want to undermine the validity of their own wealth. Doing things that significantly reduce the utility of the system is self-defeating even over the medium term because it'd lead people to just give up on the system in disgust and sell their coins, driving down the price. I think it's fair to say that being unable to buy basic things like food or drinks in person would reduce the utility of Bitcoin for a lot of people.

[xanatos]

What happens if a block becomes orphan? Its transactions are readded to the transaction pool, so they could be changed by the sender... So you would only need to wait for a split in the network to double spend your money?

I've never analysed the data myself, but I'd guess that honest splits tend to carry almost (if not exactly) the same transactions on each side of the split.

They probably collect the transactions in a "best effort" way... But if 2 blocks are orphaned, perhaps you can't put all their transactions in a new block.

[jgarzik]

I am not aware of any merchant that has ever been double spent with 0-conf transactions except the OKPAY example during the chain split. Which was almost certainly caused by mining nodes being restarted and not syncing their mempools - quite easy to fix.

SatoshiDICE has been double-spent.  There are other incidents as well.

[justusranvier]

SatoshiDICE has been double-spent.  There are other incidents as well.

Does the rate of incidents as a fraction of total transactions compare favorably or unfavorably to the rate of reversed transaction typical to credit and debit cards?

[Peter Todd]

There's no guarantee, but Satoshi's paper addresses the dynamics of this - rational miners shouldn't want to undermine the validity of their own wealth.

Similar to how rational coal power plant operators and chemical plant owners wouldn't want to undermine the cleanliness of the air they breath and water they drink.

[Peter Todd]

SatoshiDICE has been double-spent.  There are other incidents as well.

piuk has said that the blockchain.info send-shared mixer has been double-spent a few times.

[d'aniel]

There's no guarantee, but Satoshi's paper addresses the dynamics of this - rational miners shouldn't want to undermine the validity of their own wealth.

Similar to how rational coal power plant operators and chemical plant owners wouldn't want to undermine the cleanliness of the air they breath and water they drink.

Eh?  The cleanliness of the air they breathe and water they drink doesn't affect their bottom line.

[Mike Hearn]

Yeah, the blockchain thing was long chains of transactions that wouldn't confirm for ages and then relying on mempool churn to replace one of them along the way.

I don't see any need to change Bitcoin here. If you don't think 0-conf txns are reliable, OK, wait for a block. Or tell people who will listen to wait for a block. Those who want to see how reliable they can be made using technical fixes like mempool sync, doublespend alerts, risk analysis and so on can then go ahead and do so. The market will end up deciding who is right. If merchants keep getting double spent they'll go to waiting for a block. If they don't, then we all win with a more useful currency.

[amincd]

There's no guarantee, but Satoshi's paper addresses the dynamics of this - rational miners shouldn't want to undermine the validity of their own wealth.

Similar to how rational coal power plant operators and chemical plant owners wouldn't want to undermine the cleanliness of the air they breath and water they drink.

If a coal power plant operator controlled 20% of the coal plants in the world, they might be concerned, as there would be a measurable effect on their health from them choosing a more polluting way to burn coal. This is comparable to the impact that the decisions of mining pool operators have.

Now regardless of the wisdom of trusting 0-conf transactions, the fact remains that no one is forced to accept them, and removing the option by a deliberate change in standard client rules is leaving people who would otherwise choose to risk a double spend on a 0-conf transaction without the option to do so. How can that be rationalized? "We know what's best for merchants" isn't a compelling argument.

By all means, warn merchants that 0-conf transactions can be double spent, and explain the methods an attacker could use, but still let them choose what level of risk they want to bear.

I don't see any need to change Bitcoin here. If you don't think 0-conf txns are reliable, OK, wait for a block. Or tell people who will listen to wait for a block. Those who want to see how reliable they can be made using technical fixes like mempool sync, doublespend alerts, risk analysis and so on can then go ahead and do so. The market will end up deciding who is right. If merchants keep getting double spent they'll go to waiting for a block. If they don't, then we all win with a more useful currency.

+1

[caveden]

There's no guarantee, but Satoshi's paper addresses the dynamics of this - rational miners shouldn't want to undermine the validity of their own wealth.

But they wouldn't be undermining their own wealth, I'd say the contrary actually.

Doing things that significantly reduce the utility of the system is self-defeating even over the medium term because it'd lead people to just give up on the system in disgust and sell their coins, driving down the price.

You're being overly dramatic here, admit it. Being able to replace 0-conf tx would not be such a bad thing.
I'd say that "selling" a false impression of security could actually do more damage. And the fact is that 0-conf are, by definition, replaceable.

I think it's fair to say that being unable to buy basic things like food or drinks in person would reduce the utility of Bitcoin for a lot of people.

But they'd still be able to do it.
First, if the merchant knows his/her customer, no major problem in accepting 0-conf.
If he doesn't, still, he can rely on insurance contracts that will on their turn have miners committing to mine a particular transaction instead of any replacement.
Besides that, merchants could also collectively - and voluntarily - try to blacklist double-spenders.
Sums all that, and you'll have very little fraud, if any.

Actually, why am I telling all these things to you? All these ideas are yours after all... hum... Friday evening, you're already drinking or something?

[Mike Hearn]

0-conf transactions aren't replaceable by definition. The definition is the source code and it drops double spends against the memory pool. That's the whole point - peter wants to change the definition of what a zero-conf transaction means.

All the things I suggested for ways to improve the security of unconfirmed transactions are still valid, but they obviously assume the system isn't trying to actively undermine you.

Like I said, what's the downside to being laissez-faire about this? Live and let live. The market will sort it out.

[caveden]

What you're calling "definition" is a set of "good practices" implemented in bitcoind and followed by most (all?) miners. But the protocol itself allows for 0-conf replacements. That's what I meant by "definition".

[im3w1l]

Retail will want transactions satisfying:

Time to verify: < 10 seconds
Risk of charge-back/double spend: < 1%

If we change 0-conf double spends from possible to trivial, it will no longer be possible to satisfy those constraints with bitcoin.

[kjj]

Retail will want transactions satisfying:

Time to verify: < 10 seconds
Risk of charge-back/double spend: < 1%

If we change 0-conf double spends from possible to trivial, it will no longer be possible to satisfy those constraints with bitcoin.

The problem is that it isn't up to us.  There is no "we".  No one on the entire planet has the ability to prevent miners from picking transactions according to their own criteria.  If anyone is making assumptions about transactions that are not backed by actual, verifiable work, they are doing it wrong, and it is only through good fortune that they haven't been burned yet, assuming that they haven't been burned already.

[im3w1l]

The problem is that it isn't up to us.  There is no "we".

There is a "we". We who want BTC to be usable as a you know, currency.

No one on the entire planet has the ability to prevent miners from picking transactions according to their own criteria.

We can change their incentives. Bubbleboy posted a very nice proposal. See: https://bitcointalk.org/index.php?topic=180640.0

[unk]

There's no guarantee, but Satoshi's paper addresses the dynamics of this - rational miners shouldn't want to undermine the validity of their own wealth. Doing things that significantly reduce the utility of the system is self-defeating even over the medium term because it'd lead people to just give up on the system in disgust and sell their coins, driving down the price.

i believe this is an analytical error that i've tried to correct many times before. for one thing, it ignores dynamic market effects (for example, someone who profits from a put option, a short sale, or even a regular sale). it is usually a mistake to predict too confidently what 'rational' parties will do based on an incomplete understanding of their behaviour.

[Maged]

Retail will want transactions satisfying:

Time to verify: < 10 seconds
Risk of charge-back/double spend: < 1%

If we change 0-conf double spends from possible to trivial, it will no longer be possible to satisfy those constraints with bitcoin.

The problem is that it isn't up to us.  There is no "we".  No one on the entire planet has the ability to prevent miners from picking transactions according to their own criteria.  If anyone is making assumptions about transactions that are not backed by actual, verifiable work, they are doing it wrong, and it is only through good fortune that they haven't been burned yet, assuming that they haven't been burned already.

There is a "we": society. No one on the entire planet has the ability to prevent people from walking around naked. And yet, people don't walk around naked in most public locations. Sure, there are pockets of areas where this happens, but if you were randomly teleported to a populated area, you can reasonably calculate the risk of seeing naked people. Of course, you'll say that this only happens because people are being threatened with force by a government. However, governments are merely a social construct, so that argument doesn't really apply. Where it does apply, though, is when you bring up the argument that governments make it harder bring forth social change. Which brings up my next point: because of the power Bitcoin-Qt currently has, adding this change by default is very likely to be successful if you can convince the core developers. 0-conf transactions would become impossible to use in any even slightly untrusted scenario, and miners go forward with the understanding that it is socially acceptable to put personal profits first (in bitcoin, not necessarily fiat at that point). Because miners have been conditioned to only think of themselves, and society agrees, it might become socially acceptable to centralize mining. After all, that is the most profitable (again, it bitcoin) future for miners. Once you control over 50% of the mining power, you get to say who is allowed to produce blocks and who isn't, thus preventing competition from undercutting them by allowing lower transaction fees and making more money individually in the short-term. What behavior we make as default in the client now is very likely to persist for quite some time because of this effect.

Therefore, let me clarify my earlier statements: it's fine (and actually great!) that you make this patch and allow people to use it. The market should be allowed to make that decision. But, we should not force it by making it the default in the client, especially because it would be hard to go back to the way things are, much like how you can't stop centralized mining once it is in the majority.

[phelix]

As a miner I would certainly not mine at a pool that implements this practice.

It would be nice to have a neighboorhood pool watch for pools that (probably) replace transactions because of higher fees.

I hope something like this will gain ground: https://bitcointalk.org/index.php?topic=163751 Mining Codex

[killerstorm]

0-conf transactions aren't replaceable by definition. The definition is the source code and it drops double spends against the memory pool. That's the whole point - peter wants to change the definition of what a zero-conf transaction means.

It is snake oil: these rules provide no protection whatsoever, but users now erroneously believe that probability of double-spend attack is very low.

Obviously, user who wishes to double-spend can communicate directly to miners who offer this kind of service.

For example, I can implement Bitcoin Wallet Double-Spending Edition(tm) which will automatically create a double-spend for each transaction (if user ticks a checkbox) and sends a double-spending tx to a feed. Miners who wish to participate in this program will fetch transactions from this feed to get higher fees.

So "definition in the source code" is absolutely irrelevant.

Like I said, what's the downside to being laissez-faire about this? Live and let live. The market will sort it out.

Miners aren't using replace-by-fee only because they do not care much about tx fees yet. Let's see how it will work in 4 years...

By the way, I don't buy an argument that miners will care about keeping zero-conf payments somewhat secure. If zero-conf payments are accepted by merchants, then users do not care much when their tx will get into a block, so they will pay a tiny fee.

So it is in the best interest of miners to show that accepting payments with no confirmations is insecure. Then more people will try to get their transaction into block as soon as possible, paying a competitive fee.

I think it's fair to say that being unable to buy basic things like food or drinks in person would reduce the utility of Bitcoin for a lot of people.

It is crucial to buy food and drinks with plain Bitcoin transactions?

There is a plenty of options: shared wallets, green addresses, multi-signature transactions...

It is definitely possible to make payments ACTUALLY secure, it just requires a bit more effort...

[Mike Hearn]

Reality check - when I look around at the complaints people have about Bitcoin, merchants constantly being double spent doesn't come up. It isn't "snake oil" to point this out and you should give the people actually using Bitcoin more credit - they know how to avoid losses.

Your double-spending wallet edition is just theoretical - you have to find miners that will take your double spends, and then you have to find users who are willing to accept that most often their attempts to double spend will fail because the bad miners won't find the next block. And then people who use it will discover that if they aren't perfectly anonymous then they will get taken to court for wire fraud, and most likely they will lose.

So there are lots of reasons why people might not do this. And that may explain why this topic is so old and stale. It was being brought up back in 2010 and here we are, years later, people still arguing about this topic and yet there are many more thousands of merchants still accepting these transactions and not losing money.

I remember another topic that used to create endless raging arguments, the 10 minute block interval. Eventually that was solved by Charlie creating Litecoin and now people who hate the 10 minute wait can just go use that instead of creating endless forum threads. I think it's a good way to resolve such disputes - go set up an alt coin that works the way you think it should and then let the market figure it out. Or maybe the Litecoin guys would be willing to incorporate such a change.

[jdillon]

So there are lots of reasons why people might not do this. And that may explain why this topic is so old and stale. It was being brought up back in 2010 and here we are, years later, people still arguing about this topic and yet there are many more thousands of merchants still accepting these transactions and not losing money.

Double-spending is not yet much of an issue because very few merchants are vulnerable to it. You have the BlockChain.info mixer, and SatoshiDice and its clones. The latter easily respond by spending double-spend attempts 100% to fees, and proving their honesty after the fact by keeping records of the double-spend.

On the other hand, the biggest use of Bitcoin for commercial transactions is buying drugs. (I exclude BitPay's Avalon orders because they are related to the system itself) The Silk Road requires six confirmations for a deposit, and implements an off-chain transaction system for privacy and convenience.

Code:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Peter pointed out that a DoS-resistant replace-by-fee implementation requires
the implementation of either recursive fee evaluation, or strictly limiting
unconfirmed depth.

He has told me he has taken the former approach, and is done most of a
recursive fee evaluation implementation with reasonable O() scaling. Along
those lines I've increased the reward to $1000USD, again with the advice that
the reward is for a proof of concept, and rigorous engineering is not required.
4-8 days of work should be your target effort to keep hourly reasonable to the
level of a professional early in their career.

Yes Peter, you still are in competition with anyone else taking on the
challenge. I stand by my comment about what you need to do to be taken
seriously. Good luck. :)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBAgAGBQJRdn4OAAoJEEWCsU4mNhiPI3gIAKETBQlXqi20vQ81yKT83aDM
VMGuFFSs/PApy5B+24N3+UBlLql2rGpOJlQYCYHpSdTDcIwFtnYkAGzWL2VkF7RL
Pc6xk+TUEpWiPhITvxXp2e7Mi4zX2I0GVABSC9QjhgB5257pb1ufHcTYX2oTw0EA
XQUdz8wNw1VeyZkEg5bniveIRMZ/fOP3Fb2Xqlm/BxOOw7vNWi7UwmPmUAl/leGQ
P/o+qtYCkhjILlj4x2ORa29aiIEgGvrTlqGwmibNsbjovaA4s/47kY2/CGTaRpsR
/7nRqIzuWYq+/URa1b7VKfdUp/jRGW9QsDxux0L7fIhLt6a7eEghrjZEDDoeqkE=
=DX3c
-----END PGP SIGNATURE-----

[killerstorm]

Double-spending is not yet much of an issue because very few merchants are vulnerable to it. You have the BlockChain.info mixer, and SatoshiDice and its clones. The latter easily respond by spending double-spend attempts 100% to fees, and proving their honesty after the fact by keeping records of the double-spend.

From what I've heard a lot of merchants who have physical contact with customer (e.g. a restaurant) accept payments with zero confirmations. Each time this is mentioned on reddit, somebody says "don't worry, you just need to wait a couple of seconds". Yeah, right...

Is it still possible to make transaction which is unlikely to be included into a block? Perhaps just low-priority one (freshly sent coins) and no fee.

This will likely give you at least a few hours of opportunity to pull off a double-spend. So you can start once you're already far away from this merchant physically.

Also, I doubt that merchant will notice that money disappeared and associate it with you.


Once this patch is ready I'll try to help with the front end. I'm currently working on web wallet, so double-spending might become shockingly easy =)

[etotheipi]

I've spent a lot of time thinking about this, discussing it here, and on IRC with smart people.  I have some extra clarification and context to this discussion that is worth mentioning.  Apparently, my strong argument in favor of this is equally applicable to a feature of the network I had not realized would also be broken.  Not that it changes my argument, but it does expand the breadth of consequences of accepting that this will happen (which I still strong believe is true).

We need to separate the two concepts and make sure we're talking about the same thing:

(1):  You have final transaction that is sitting in miners' memory pools, waiting to go into a block.  Now a second final transaction comes along spending one of the same inputs, but with a higher fee.   What does the miner do?  Well, the current default behavior is to drop it and mine the first transaction they see.  This is the behavior of the majority of the network right now, but there is nothing stopping individual miners/pools from modifying their source code to do the "unethical" behavior of replacing the original with the higher-fee transaction.  This is what I originally thought this thread was about:  a call for a patch to make this "worst case" equal to the "best case" to prevent the system from adapting to a false reality.

But there's another situation we would all like to depend on (and perhaps, have assumed will be usable in the future),  but which is equally subject to the above argument:

(2):  You have a non-final, replaceable transaction that is sitting in miners' memory pools, waiting for the locktime to expire so they they can mine it into a block.  Now a second transaction comes along that meets all the requirements of transaction-replacement (increased sequence numbers, etc).   The intended behavior is that the miner will drop that the original transaction and replace it with the new one.  There doesn't even have to be an increased fee, especially because it's essentially zero-cost for the miner to update their memory pool with the new tx.  However, for similar reasons above... the miner doesn't have to replace the transaction, and if there was economic incentive to mine the old one (perhaps a second transaction with a huge fee that spends an older version of the replaced tx), then there's nothing stopping them from ignoring the replacements.  The original (to-be-replaced) transactions are completely valid to mine after the locktime, leading to a standard race between good and bad miners.  This allows for one party in a HFT (rapidly-adjusted micropayment) contract to have some probability of screwing over the other party.

This is troubling, because there's a lot of cool things that become possible with transaction replacement, but nodes are not obligated to replace transactions, they only have to allow it if they want.   Fortunately, these two situations are not exactly identical.  One thing that might save #2 is if you can change the locktime on the replacement to a couple blocks sooner, to almost guarantee it can be mined by honest parties, before the older one can mined by dishonest parties.  But I don't know if you can change the locktime on a replaced transaction, and it would severely limit how many replacements could be made.  But still better than nothing...


One last thing to consider is that, in #1 the exchange is usually happening between parties that have never met and have zero-trust (or at least one direction has zero trust).  Merchant has zero trust of this random customer that just walked into the store.   But #2 has the pretense that the parties already have some association with each other, and thus >zero trust, or else they wouldn't be setting up this replaceable/contract with each other.  It doesn't stop it from happening, but it does imply that one party may have recourse if the other one screws him over

[killerstorm]

(2):  You have a non-final, replaceable transaction that is sitting in miners' memory pools,

Currently non-final transactions are treated as non-standard, so they are not added to memory pools and are not propagated.

And that's good... We are no longer restricted by transactions replacement rules. Which are no more than inconvenience: they provide no guarantees, but make it hard to use a contract which require a different kind of replacement.

[etotheipi]

(2):  You have a non-final, replaceable transaction that is sitting in miners' memory pools,

Currently non-final transactions are treated as non-standard, so they are not added to memory pools and are not propagated.

And that's good... We are no longer restricted by transactions replacement rules. Which are no more than inconvenience: they provide no guarantees, but make it hard to use a contract which require a different kind of replacement.

My post was not about how it is at this moment, but under the assumption that tx-replacement become standard at some point.  It would become standard under the assumption that it would be useful for a variety of things discussed in the community over the past couple years.  I was just pointing out that it's not so clear-cut if you assume that miners will not follow "ethical" replacement rules.

[killerstorm]

My post was not about how it is at this moment, but under the assumption that tx-replacement become standard at some point.  It would become standard under the assumption that it would be useful for a variety of things discussed in the community over the past couple years.  I was just pointing out that it's not so clear-cut if you assume that miners will not follow "ethical" replacement rules.

If there are miners who do not follow "ethical" rules than the whole concept of replacement rules is completely useless as these rules cannot be relied on.

So we have two scenarios:

1. all miners are ethical: thus you can completely
2. some miners are not ethical: you cannot rely on replacement rules

Are you talking about third case where miners can violate some rules, but always honor other? Half-ethical miners?

[etotheipi]

My post was not about how it is at this moment, but under the assumption that tx-replacement become standard at some point.  It would become standard under the assumption that it would be useful for a variety of things discussed in the community over the past couple years.  I was just pointing out that it's not so clear-cut if you assume that miners will not follow "ethical" replacement rules.

If there are miners who do not follow "ethical" rules than the whole concept of replacement rules is completely useless as these rules cannot be relied on.

So we have two scenarios:

1. all miners are ethical: thus you can completely
2. some miners are not ethical: you cannot rely on replacement rules

Are you talking about third case where miners can violate some rules, but always honor other? Half-ethical miners?

#2 was jdillon's original point, and the one I was supporting.  But not in the context of non-final transactions -- I thought the post was about replacing already-final transactions, and I was pointing out there was no way to avoid some miners doing it, and thus undermining the usefulness of them.

But five posts back, I was pointing out my own revelation that any such arguments as made in #1 (already final tx), are necessarily applicable to #2 as well (replaceable tx).  And pointing out to readers that there are two distinct contexts here, that have been kind of jumbled together in this thread.   This revelation bothers me a bit, because the fact that already-final zero-conf tx are useless was not news to me.  But the fact that tx replacement may also be useless for the same reasons is unfortunate.  I hadn't considered that the same arguments applied.  

But I don't think it's a lost cause.  It's feasible that such transactions/contracts are useful enough because they aren't typically made between zero-trust parties... usually there is some degree of association between them, and the quantities of money at stake doesn't have to be very high.  And perhaps, if you only plan a couple dozen replacements, it still works as long as you reduce the locktime each time.  

It was really just food for thought.

[killerstorm]

This revelation bothers me a bit, because the fact that already-final zero-conf tx are useless was not news to me.  But the fact that tx replacement may also be useless for the same reasons is unfortunate.  I hadn't considered that the same arguments applied.  

1. Non-final transactions are still useful, you just shouldn't assume that you get automagic protection against double-spends.
2. You can protect against double-spends from your counter party using multi-signature scripts.

Basically, contracts which assume that double-spending is not possible are sloppily designed. Rewrite them in such a way that multi-signature script will be used, and they will become safe.

Moreover, they will become safe and implementable RIGHT NOW, not in a hypothetical future with ethical miners.

Any contract which relies on transaction replacement rules can be made secure with help of a third party.

So, basically, you do not need to rely on all miners being honest. You just need to rely on a third party which can be chosen by both parties of contract.

As a bonus, fidelity bond can be used to make sure that third party has incentive to be honest. So you can locally emulate features of proof-of-stake system.

[killerstorm]

Here's the theory, as I see it:

To make sure that "transaction replacement rules" actually work, you need to discourage miners from breaking them. For example, punishing them in some way.

 * Proof-of-work system must follow one main rule: longest valid chain wins. If you add other rules it becomes less stable. So it is hard, or even impossible, to add punishment as a rule, and you cannot punish miners in any other way because they are anonymous.
 * In proof-of-stake system punishment is very straightforward: if there is evidence that miner have broken the rule, his stake is just destroyed.

Thus proof-of-stake system is more flexible: it allows you to create pretty much arbitrarily complex rules as long as they are verifiable using cryptography.

Multi-signature scripts and fidelity bonds allow us to create an emulation of proof-of-stake system within proof-of-work system.

So I think it would be great if people abandon fantasies about "transaction replacement rules" and will instead listen retep: his ideas are much more sound and implementable.

[TierNolan]

* Proof-of-work system must follow one main rule: longest valid chain wins. If you add other rules it becomes less stable. So it is hard, or even impossible, to add punishment as a rule, and you cannot punish miners in any other way because they are anonymous.

You should mine on the chain that most likely has most of the miners on it.

Apparently, if you paint the horns of a prey animal, it greatly increases the chances of them being killed by predators.  The reason is that it helps coordination between the predators.  They don't have to communicate to pick a target, which can be helpful for stealth.

The same thing works with blocks.  The highest POW block is "painted" by custom, and so all try to build on it.

However, miners might be willing to mine on older blocks if the tx fees are high enough.

For example, imagine there was a block with a 500BTC tx fee (maybe a donation to miners or someone messed up).

You could build on that block and get nothing, or build on the previous block and possibly get the the fee.  If you win, maybe you pay out some of the tx fee to "true".  Paying to true is the same as paying to fee, since the next miner gets to direct it to their own address.

Maybe a miner might take the viewpoint that they should scan back along the blocks.

The latest block is block X and the previous blocks paid the following fees.

X-4) paid 27 in fees
X-3) paid 26 in fees
X-2) paid 1000 in fees
X-1) paid 29 in fees
X) paid 26 in fees

The "selfish" POW could be calculated as (block fees - (average fee)*depth).  That gives the X-2 block the highest POW.

If most miners use the selfish POW, then it fulfils itself.  Most will build on X-2, so you would be a fool to build on X.

What you want is a rule that is a Nash equilibrium.  It is in the best interests of a miner to stick to their current strategy as long as nobody else changes theirs.

Breaking ties against blocks that violate the customs adds an incentive to not break them.

A feature of the selfish POW is that it allows donors to dump fees into one block and forces miners to share them with later blocks.  If you take more than the average, then miners won't build on your block.

[Maged]

One last thing to consider is that, in #1 the exchange is usually happening between parties that have never met and have zero-trust (or at least one direction has zero trust).  Merchant has zero trust of this random customer that just walked into the store.

Why is this assumed to be true? Credit cards have proven that you can calculate a non-zero amount of trust for a large group of people. In fact, "pay-what-you-want" schemes, where each customer is trusted completely have also shown to be somewhat successful. Not everyone needs a system that requires zero-trust, so why should they be forced into one? Because of things like successful "pay-what-you-want" schemes working well-enough, I suspect that even if this patch was included in every Bitcoin client, some merchants would still depend on zero-conf transactions because they trust that their users will usually not reverse their transactions. A music store, for instance, might do that. All this would do is weaken the system.

[etotheipi]

One last thing to consider is that, in #1 the exchange is usually happening between parties that have never met and have zero-trust (or at least one direction has zero trust).  Merchant has zero trust of this random customer that just walked into the store.

Why is this assumed to be true? Credit cards have proven that you can calculate a non-zero amount of trust for a large group of people. In fact, "pay-what-you-want" schemes, where each customer is trusted completely have also shown to be somewhat successful. Not everyone needs a system that requires zero-trust, so why should they be forced into one? Because of things like successful "pay-what-you-want" schemes working well-enough, I suspect that even if this patch was included in every Bitcoin client, some merchants would still depend on zero-conf transactions because they trust that their users will usually not reverse their transactions. A music store, for instance, might do that. All this would do is weaken the system.

Because that was the goal of Bitcoin, to remove the requirement for pre-existing trust between transacting parties without a third-party.  That's how it's advertised.  I have no dispute with your statement that a lot of transactions happen right now that don't actually require zero-trust.  But if the system fails to meet that criteria in some contexts (like expecting your transaction to be replaced when it actually may not), users should be uniquely aware that it's not a good option for zero-trust situations.

Trust can go to zero as confirmations go to infinity.  But before inifinite confirmations, you have to have a trade-off between that security and the convenience/functionality.  The point of my post was to state the revelation that rapidly-adjusted micropayments are not trustless the way it was originally suggested.  It's critical to know that the next time I transact with someone in Nigeria, I do not use that technique.  

Your point is that it's not useless.  I agree -- I don't think it's useless.  I just think it's worth mentioning that it shouldn't be used in zero-trust situations.  And luckily, most people who would be using this, already have some degree of trust.  So it's not so bad, just use it carefully.

[killerstorm]

What you want is a rule that is a Nash equilibrium.  It is in the best interests of a miner to stick to their current strategy as long as nobody else changes theirs.

Well... What if there is some sort of collusion among majority of miners (i.e. they have more than 50% of hash power combined): they will agree to decide block to mine on top of using some set of rules and some sort of a practical Byzantine fault tolerant algorithm for synchronization?

They can simply drop blocks of miners which do not agree to participate in this PBFT-synced collusion, and as far as I can tell it is stable as long as colluding miners have a majority. (I'm not quite sure about that, but I guess it will be 'stable enough' for practical purposes.)

Within this collusion pretty much arbitrary rules can be enforced.

[TierNolan]

What you want is a rule that is a Nash equilibrium.  It is in the best interests of a miner to stick to their current strategy as long as nobody else changes theirs.

Well... What if there is some sort of collusion among majority of miners (i.e. they have more than 50% of hash power combined): they will agree to decide block to mine on top of using some set of rules and some sort of a practical Byzantine fault tolerant algorithm for synchronization?

A Nash equilibrium doesn't require any collusion at all.  It is something that just happens.

For example, lets say you play a game.  No communication is allowed between players and their moves are secret.

Each player picks a number from 1 to 10.  The results are announced and the players that picked the most popular number (say 7) are given $5.

The game is repeated, what number do you think will win?

Next, the referee says "For the next round, I strongly recommend 4.  You can still vote for any of the 10, but really, 4 is a great number".  Which one do you think would win?  7 might still win, but 4 has a pretty good chance.

They can simply drop blocks of miners which do not agree to participate in this PBFT-synced collusion, and as far as I can tell it is stable as long as colluding miners have a majority. (I'm not quite sure about that, but I guess it will be 'stable enough' for practical purposes.)

You don't need a majority.  Say there is one pool which has 10% of the power and they say that they will queue all blocks that have the 2nd (or later) of a double spent transaction in them for 5 minuted before building on them.

Other miners now know that if they add those txs into their blocks, only 90% of the hashing power of the network will build on them for the first 5 minutes.

Within this collusion pretty much arbitrary rules can be enforced.

There are limits before the Nash equilibrium breaks down. 

Simple Nash equilibrium are the best.  That is why build on the highest POW chain is so strong.

However, as I show in my previous post, with large fees, that Nash equilibrium can break down.

[jdillon]

#2 was my jdillon's original point, and the one I was supporting.  But not in the context of non-final transactions -- I thought the post was about replacing already-final transactions, and I was pointing out there was no way to avoid some miners doing it, and thus undermining the usefulness of them.

You mention that some miners may not honor replacement undermining the usefulness.

But remember they may not honor replacement not because of any decision by them but because of limitations of technology. Old software is obvious. Slow network connections is another.

Part of what worries me about zero-conf is that if people rely on it rules may be put into place that punish miners who  fail to honor replacements or detect double-spends because they happen to be behind bandwidth constrained network connections and simply did not get the information that other, larger, more centralized miners did.

It is ironic that unavoidable technical limitations means that with unlimited blocksizes zero-confirmation transactions definitely will not be safe requiring all the decentralized trust required by the off-chain transactions that make large blocks unrequired.

[jdillon]

An initial implementation is available for testing.

See new thread here: https://bitcointalk.org/index.php?topic=199947.0

[jonytk]

It should never allow you to reemplace a transaction, or to change outputs,
it should allow you to give more priority to a transaction that it's stuck on the limbo of 0 fee/smaller fee transactions.

Only replace the transaction if it's the same transaction with bigger fee. to the same outputs.

Any other thing is not bitcoin, btc is not reversible.
or you just killed satoshi dice.

But talking about improving you could also change the 10 min confirmation to 2 and slash the rewards 5 times.
then they will change to a 1or 2 confirmations .

[etotheipi]

It should never allow you to reemplace a transaction, or to change outputs,
it should allow you to give more priority to a transaction that it's stuck on the limbo of 0 fee/smaller fee transactions.

Only replace the transaction if it's the same transaction with bigger fee. to the same outputs.

Any other thing is not bitcoin, btc is not reversible.
or you just killed satoshi dice.

But talking about improving you could also change the 10 min confirmation to 2 and slash the rewards 5 times.
then they will change to a 1or 2 confirmations .

You just completely ignored the entire discussion in this thread.  You don't get to decide what transactions get to go in a block.  I don't, no one else does, either.  Only the miners get to decide for themselves what transactions go in the block.  And if it involves more money, they will do it eventually (though they generally are not doing it, yet).  Because they can do so while operating completely within the rules of the system.  You are welcome to add your own mining power to the system and refuse to replace transactions with your share of power.  But you don't get to tell others what they get to do with their share.

As the number of confirmations goes to infinity, transactions are irreversible.  But zero-confirmation transactions are, by definition, not confirmed, and thus there is no such thing as "reversing them."  They were never confirmed to begin with, and should never have been trusted at all until they had confirmations.   (Note:  this applies only to trustless transactions:  zero-conf can still be useful if there is a persistent relationship between the two parties, but Bitcoin was never designed to do instantaneous transactions with zero-trust)

[jgarzik]

It should never allow you to reemplace a transaction, or to change outputs,
it should allow you to give more priority to a transaction that it's stuck on the limbo of 0 fee/smaller fee transactions.

Only replace the transaction if it's the same transaction with bigger fee. to the same outputs.

Any other thing is not bitcoin, btc is not reversible.
or you just killed satoshi dice.

Zero-confirmation transactions were never safe.  Note that Satoshi DICE apparently waits for confirmations, on higher value bets -- an admission that SD themselves know zero-conf are not safe.

However, with regards to transaction replacement, it should be noted that it introduces race conditions that increase non-determinism.

The first step towards improved determinism is, instead, making transactions expire after a certain amount of time in memory pools, without being mined.

[kjj]

It should never allow you to reemplace a transaction, or to change outputs,
it should allow you to give more priority to a transaction that it's stuck on the limbo of 0 fee/smaller fee transactions.

Only replace the transaction if it's the same transaction with bigger fee. to the same outputs.

Any other thing is not bitcoin, btc is not reversible.
or you just killed satoshi dice.

Bitcoin is a system for deciding which of two or more transactions is the "right" one.  We use a global network and consume quadrillions of hashes every few minutes not for our own amusement, but because the problem really is that hard.

A transaction is exactly as secure as the quantity of work it would take to reverse it.  Zero work?  Zero security.  STOP ACCEPTING ZERO WORK TRANSACTIONS.  You've spent too long in dreamland, assuming that the intention to hash in the future was as good as hashes already done in the past.  They aren't, they never were, and now everyone will be forced to accept reality as it is.

But talking about improving you could also change the 10 min confirmation to 2 and slash the rewards 5 times.
then they will change to a 1or 2 confirmations .

Confirmations don't secure transactions.  Hashes do.  Making the blocks more frequent would mean that they have correspondingly fewer hashes of security.  Changing the lump size that hashes come in won't help anyone.

[Peter Todd]

It should never allow you to reemplace a transaction, or to change outputs,
it should allow you to give more priority to a transaction that it's stuck on the limbo of 0 fee/smaller fee transactions.

Only replace the transaction if it's the same transaction with bigger fee. to the same outputs.

Any other thing is not bitcoin, btc is not reversible.
or you just killed satoshi dice.

Zero-confirmation transactions were never safe.  Note that Satoshi DICE apparently waits for confirmations, on higher value bets -- an admission that SD themselves know zero-conf are not safe.

Yup. I've spoken with evoorhees about this issue and while I don't know what their plans are exactly they had no objections. In fact, out of the roughly half dozen people running services I have either contacted, or who have contacted myself or John Dillon, nobody has actually asked us not to implement replace-by-fee. I don't want to pretend I'm speaking for them, but I suspect the few merchants that actually need zero-conf security would like to see a genuinely secure solutions be implemented like trusted computing, fidelity bonds w/ double-spend fraud proofs and off-chain mechanisms rather than the current half-measure of hoping everyone follows a de-facto standard.

However, with regards to transaction replacement, it should be noted that it introduces race conditions that increase non-determinism.

The first step towards improved determinism is, instead, making transactions expire after a certain amount of time in memory pools, without being mined.

Mind expanding a bit on what you mean by non-determinism?

[jonytk]

okay so let me understand, this problem already exist and can be triggered by bad miners,

but don't you think you are overreaching, you were ask to do a way to take transactions out of limbo.
I know very well that because i have one of those chain games, and i time i configured it under .0001 btc and
the transactions that didn't include fee never arrived, yet still pending.

Don't be naive, the coding you are doing is exactly what the bad miner needs,
if i give you 1000$ will you code me an alt-coin and an exchange for me?
Because i'm in...

A few days ago i saw a thread "how to make a genesisblock" by some kid from 4chan trying to do a noob-coin...
if you see a new alt-coin every 4 hours now you know why...

PD: i'm not questioning retep nor do i have shares on SD. Just want this discussed further before he releases any of the code.

[etotheipi]

okay so let me understand, this problem already exist and can be triggered by bad miners,

but don't you think you are overreaching, you were ask to do a way to take transactions out of limbo.
I know very well that because i have one of those chain games, and i time i configured it under .0001 btc and
the transactions that didn't include fee never arrived, yet still pending.

Don't be naive, the coding you are doing is exactly what the bad miner needs,
if i give you 1000$ will you code me an alt-coin and an exchange for me?
Because i'm in...

A few days ago i saw a thread "how to make a genesisblock" by some kid from 4chan trying to do a noob-coin...
if you see a new alt-coin every 4 hours now you know why...

PD: i'm not questioning retep nor do i have shares on SD. Just want this discussed further before he releases any of the code.

This patch makes the best case the same as the worst case: zero-confirmation transactions can be "overridden" if another transaction with a higher fee is broadcast before it makes it into a block.  We are arguing that this unquestionably how miners will operate in the future, anyway, so we change the default now to prevent any further adaptation to a  false sense of security/reality.  i.e. users currently experience transactions not being replaced so readily, so they build around an assumption that zero-conf are okay for zero-trust situations.  They're not. 

I agree this is not inline with the original intention of the software, but I don't think there's a way around it.  Miners will do this eventually, even if most of them aren't, yet.  Let's not pretend they won't.

Plus, this patch comes with the nice benefit that "stuck" tx can be easily fixed.  Though, I do agree that a non-deterministic "undo" button is a bad idea.  It would be unreliable in many ways, and would also give a false sense of reality to users.  Instead, I'd like to see a button/dialog that says "This transaction has been waiting more than one hour without being accepted.  Would you like to increase the fee to try to speed up its acceptance?" 

[Peter Todd]

Plus, this patch comes with the nice benefit that "stuck" tx can be easily fixed.  Though, I do agree that a non-deterministic "undo" button is a bad idea.  It would be unreliable in many ways, and would also give a false sense of reality to users.  Instead, I'd like to see a button/dialog that says "This transaction has been waiting more than one hour without being accepted.  Would you like to increase the fee to try to speed up its acceptance?" 

Yeah, the user-interface aspects will be tricky to communicate. For instance if an undo button is implemented, it probably should actually be called something like "Attempt to cancel", and at least initially be hidden off in the RPC interface anyway.

Increasing the fee isn't such a big problem, although a dialog box that points out you have the option after some reasonable delay is a good idea. It should probably be called "Offer a higher fee" so users aren't as surprised if the lower fee version goes through anyway.

[colindean]

What is the effect of this change on retail sales? Won't this make it possible to effectively re-route a transaction after an exchange of physical goods has already taken place?

Or is there an assumption or recommendation that Bitcoin not be used for transactions that are immediate, such as point-of-sale? I know it's always been recommended to wait for at least one confirmation, but up until now, there's reasonable security in accepting unconfirmed as long as you've got a full blockchain by which to compare the transaction's inputs and outputs.

[thezerg]

We are arguing that this unquestionably how miners will operate in the future, anyway, so we change the default now to prevent any further adaptation to a  false sense of security/reality.

I doubt it.  If there is ever a significant zero-conf market (especially a brick & mortar), they can pay mining pools to NOT allow replacement for TXNs originating from them.

This would reduce the likelihood of a double-spend dramatically.

As the credit-card business aptly shows, problems not theoretically solvable are often 99.93% (7 cents out of 100 bucks) solvable in practice http://en.wikipedia.org/wiki/Credit_card_fraud

[kjj]

up until now, there's reasonable security in accepting unconfirmed as long as you've got a full blockchain by which to compare the transaction's inputs and outputs.

Except that there really hasn't been any security in that, just hope.  People have been living on the hope that those transactions would be confirmed some block in the future.  Most of them got away with it.  But they never had any security, not really.

[Maged]

Or is there an assumption or recommendation that Bitcoin not be used for transactions that are immediate, such as point-of-sale?

No, this is an assertion that bitcoins not be used for transactions that are immediate, even if a business is already capable of handling the levels of fraud the current system would entail.

In fact, out of the roughly half dozen people running services I have either contacted, or who have contacted myself or John Dillon, nobody has actually asked us not to implement replace-by-fee.

Of course not. Until bitcoins become commonplace, the most common user of zero-confirmation transactions - brick and mortar businesses - won't really exist. If this change is inevitable like you guys claim, why not wait for it to happen naturally?

[etotheipi]

Or is there an assumption or recommendation that Bitcoin not be used for transactions that are immediate, such as point-of-sale?

No, this is an assertion that bitcoins not be used for transactions that are immediate, even if a business is already capable of handling the levels of fraud the current system would entail.

In fact, out of the roughly half dozen people running services I have either contacted, or who have contacted myself or John Dillon, nobody has actually asked us not to implement replace-by-fee.

Of course not. Until bitcoins become commonplace, the most common user of zero-confirmation transactions - brick and mortar businesses - won't really exist. If this change is inevitable like you guys claim, why not wait for it to happen naturally?

False sense of security.

The point of all this is that zero-conf tx should not be used for zero-trust situations.  But merchants will, because it "seems to work" right now.  What we want is merchants to recognize this lack of security and decrease required trust, say, by having the buyer show ID before they walk out with the merchandise.  Just like they do for cashing a check (which carries much of the same risks).  As long as there is a way to use the legal system as backup, then they have appropriately compensated for the lack of security behind the zero-conf tx.  

Merchants could learn the hard way, and then they'll stop trusting zero-conf tx.  But if they do that, there's no reason any more to artificially maintain this illusion that they are somehow secure.  Might as well just write it off now and let everyone start adapting now. 

[edmundedgar]

I agree this is not inline with the original intention of the software, but I don't think there's a way around it.  Miners will do this eventually, even if most of them aren't, yet.  Let's not pretend they won't.

I'm not intending this as a point for or against this patch, but I reading this set me wondering how many of the other assumptions we'd normally make about Bitcoin would still hold given this (fairly reasonable-sounding) idea that a majority of miners will be acting purely out of economic self-interest, regardless of damage they might do to the Bitcoin ecosystem.

One assumption that immediately falls is the idea that Bitcoins are censorship-resistant. If miners are all acting in their own rational self-interest, I can pay miners to blacklist transactions coming from your address. If you tried to spend, I'd tip any miner who creates a block that doesn't include your transactions at a higher rate than their fee. If they do manage to get into a block, I can (for a greater cost) also tip people for ignoring that block and building on another one.

If I'm trying to do this on a large scale - say I'm the DEA trying to interfere with the flow of money to drug smugglers - I can keep that taint going on all the way down the chain through future spends, so that once you take money from a drug smuggler, that money will be forever less valuable than other money. If you don't want to receive dud money that's hard to spend, you're going to have to check for the taint as well. I can run a convenient web service so that you can check for black-lists, and also white-lists of people who have confirmed their identity with me so you can be sure I won't bribe people to taint their coins. Hey presto, everybody is cooperating with me to do AML checks...

This wouldn't fly now because miners are
a) Decent people, not purely rational economic actors.
b) Fairly [shock horror] centralized, which makes them resistant to a Tragedy of the Commons. BTC Guild and Slush won't cooperate with my evil scheme for fear of damaging the future of Bitcoin, which costs them more in the long run.

[etotheipi]

I'm not intending this as a point for or against this patch, but I reading this set me wondering how many of the other assumptions we'd normally make about Bitcoin would still hold given this (fairly reasonable-sounding) idea that a majority of miners will be acting purely out of economic self-interest, regardless of damage they might do to the Bitcoin ecosystem.

One assumption that immediately falls is the idea that Bitcoins are censorship-resistant. If miners are all acting in their own rational self-interest, I can pay miners to blacklist transactions coming from your address. If you tried to spend, I'd tip any miner who creates a block that doesn't include them at a higher rate than their fee. If they do manage to get into a block, I can (for a greater cost) also tip people for ignoring that block and building on another one.

If I'm trying to do this on a large scale - say I'm the DEA trying to interfere with the flow of money to drug smugglers - I can keep that taint going on all the way down the chain through future spends, so that once you take money from a drug smuggler, that money will be forever less valuable than other money. If you don't want to receive dud money that's hard to spend, you're going to have to check for the taint as well. I can run a convenient web service so that you can check for black-lists, and also white-lists of people who have confirmed their identity with me so you can be sure I won't bribe people to taint their coins. Hey presto, everybody is cooperating with me to do AML checks...

This wouldn't fly now because miners are
a) Decent people, not purely rational economic actors.
b) Fairly [shock horror] centralized, which makes them resistant to a Tragedy of the Commons. BTC Guild and Slush won't cooperate with my evil scheme for fear of damaging the future of Bitcoin, which costs them more in the long run.

You do bring up some interesting context.  And I will spend some time thinking about it.  But I wholly dispute this statement:

..regardless of damage they might do to the Bitcoin ecosystem

Replacing unconfirmed transactions doesn't do harm to the Bitcoin ecosystem.  It's how the system operates.  We're not "removing" security, it was never there to begin with.  The success of Bitcoin never depended on it, in any way.  We're just guaranteeing that no one is ever misled about that aspect of the system.

Also, your comment about blacklisting is really not the same at all (nor feasible).  Zero-conf replacement requires only a few miners to participate for it to make zero-conf transactions pretty much useless in zero-trust transactions.  That's not the same as blacklisting, which needs 100% miner participation to work.  Or rather, I only need a few miners to agree to mine my transaction for it to be eventually accepted.  And convincing miners to not mine the top block is going to cost you a $#!+load of money...every 10 minutes...forever.

Your point is not lost on me, I just didn't like your specific examples

[kjj]

I'm not intending this as a point for or against this patch, but I reading this set me wondering how many of the other assumptions we'd normally make about Bitcoin would still hold given this (fairly reasonable-sounding) idea that a majority of miners will be acting purely out of economic self-interest, regardless of damage they might do to the Bitcoin ecosystem.

I reject your premise.  The health of the bitcoin ecosystem is hopelessly intertwined with the self-interests of miners.  This relationship goes in both directions.  Bitcoin cannot survive contrary to the self-interest of miners, and miners cannot survive without the common good of bitcoin.

If it turns out that the bitcoin system does not ultimately align these two forces, it does not deserve to be continued

The good news is that bitcoin does not appear to contain this flaw.  The people that are damaging the ecosystem are those that are advocating and defending practices that are unsafe and cannot be made safe, such as accepting transactions secured by hope rather than work.

[edmundedgar]

And convincing miners to not mine the top block is going to cost you a $#!+load of money...every 10 minutes...forever.

I shouldn't need to do this forever, because, once I set up a reasonable assumption that I can keep this thing going (remember I'm the DEA, I just seized a shedload of bitcoins in a drug bust) it's in everyone's interests not to mine that block. They want to mine on top of the longest chain because everyone else wants to mine on top of the longest chain, but given purely rational economic actors who don't care about Bitcoin, that rule, like everything else, is up for sale.

Hell, I may not even need to spend any money - I just need to convince all these rational economic actors that I could and would spend that money if necessary, then it'll be in the interests of each individual miner to start following my rules, not Satoshi's.

Edit: "in the network's interests" -> "in the interests of each individual miner" to distinguish from the interests of miners in general, which are different here because it's a classic Tragedy of the Commons situation.

[edmundedgar]

Zero-conf replacement requires only a few miners to participate for it to make zero-conf transactions pretty much useless in zero-trust transactions.

I don't think that's right - people accepting zero-confirmation transactions are already playing the odds. 1% of mining power taking the later arrival with the higher transaction fee still leaves you getting the payment 99% of the time, so in a lot of cases it would still be worth it for the extra sales.

Double-spend attackers welcome, buy 99 pizzas and get one free...

[Maged]

Of course not. Until bitcoins become commonplace, the most common user of zero-confirmation transactions - brick and mortar businesses - won't really exist. If this change is inevitable like you guys claim, why not wait for it to happen naturally?

False sense of security.

The point of all this is that zero-conf tx should not be used for zero-trust situations.

Why are you on the internet right now? After all, it is impossible* to get a virus if you don't have an internet connection.

*Yeah, there are other ways, but ignore that for now.

What about your money? I suspect that you have it all in gold (because USD can't be trusted) stored in a vault that you personally designed (because someone else may have put a backdoor in their design) stored under your personal supervision. After all, every action you ever take must require zero-trust, right?

Please tell me that you get where I'm going with this. This is a problem that should be solved through general education, not reducing effective security. Also, stores don't even need to ask for ID - they just need to have a camera, which is something that they should have to prevent general shoplifting anyway.

Zero-conf replacement requires only a few miners to participate for it to make zero-conf transactions pretty much useless in zero-trust transactions.

I don't think that's right - people accepting zero-confirmation transactions are already playing the odds. 1% of mining power taking the later arrival with the higher transaction fee still leaves you getting the payment 99% of the time, so in a lot of cases it would still be worth it for the extra sales.

Double-spend attackers welcome, buy 99 pizzas and get one free...

And that's worst case, when every order is made by an attacker.

[justusranvier]

Security in this context is being inappropriately treated like a binary concept.

There's an entire consumer economy out there based around charge cards which, in bitcoin terms, take 90 days to confirm transactions. Trillions of dollars are being transacted out in the real world via payment methods that are no less insecure than zero-confirmation Bitcoin transactions.

Accepting zero-conf transactions is an issue of risk management and business planning, not a case of "secure" vs "insecure".

[Gavin Andresen]

Security in this context is being inappropriately treated like a binary concept.

+1

[TierNolan]

Also, your comment about blacklisting is really not the same at all (nor feasible).  Zero-conf replacement requires only a few miners to participate for it to make zero-conf transactions pretty much useless in zero-trust transactions.  That's not the same as blacklisting, which needs 100% miner participation to work. 

No, it doesn't even require > 50%.

For example, assume 75% of the miners are profit seeking and 10% of the miners said that they won't build on blocks with any of <list> of transactions for at least 2 blocks.

If you add a block containing one of those transactions, then there is a 10% chance that the alternative miners will get the next block.  Rational miners would then switch to that block, since if it wins, it get 100% of the hashing.  If the other block wins, it still only gets 90% of the hashing, since the "bad" transaction has to be buried two deep.

The effect is that 85% of the miners end up supporting the blacklisting.

Mine a "clean" block
- you get 100% of miners to build on yours

Miner a "tainted" block
- you get at most 90% of the miners to build on yours

If the 10% find another block
- there are 2 equal POW blocks
-- miners following the rules stay on the first block (15%)
-- taint enforcers stay on alt branch (10%)
-- rational miners switch to alt branch (75%)

If the 90% find the next block
-- all switch to that chain

So, if you mine a "bad" block, you have a 10% chance of it being matched by the taint enforcers and then an 85% of it then be superseded.  This gives an 8.5% chance of you losing your tx and minting rewards. 

As long as only a small number of transactions are tainted, it isn't worth including them.

The defense would be to have lots of p2p mixing operations.  If most coins are tainted, then none are.

[Stephen Gornick]

Is it conceivable that a judge could conclude that because the miner was running this replace-by-fee code and received compensation for doing this that the miner could liable for losses by the merchant? [Edit: or charged as an accessory to the crime of fraud even?]

[kjj]

Is it conceivable that a judge could conclude that because the miner was running this replace-by-fee code and received compensation for doing this that the miner could liable for losses by the merchant? [Edit: or charged as an accessory to the crime of fraud even?]

Sure.  But it would be silly to do so.

The blockchain is what defines the order that transactions happen.  If a miner can be liable for replacing a transaction, why not for failing to replace it?  The network makes no promises to deliver transactions to nodes in some magically special order, except when that delivery is as a block.  What if the miner gets the replacement first?  How do you prove otherwise?

Judges, in general, aren't stupid, and this is simple stuff, easily explained.

[Peter Todd]

Is it conceivable that a judge could conclude that because the miner was running this replace-by-fee code and received compensation for doing this that the miner could liable for losses by the merchant? [Edit: or charged as an accessory to the crime of fraud even?]

Sure.  But it would be silly to do so.

The blockchain is what defines the order that transactions happen.  If a miner can be liable for replacing a transaction, why not for failing to replace it?  The network makes no promises to deliver transactions to nodes in some magically special order, except when that delivery is as a block.  What if the miner gets the replacement first?  How do you prove otherwise?

+1

[bytemaster]

So here is a business model for an unethical company:

1) Invest in a large amount of mining hardware and create a p2pool mining pool.
2) Create a bitcoin client with a checkbox for 'attempt double spend'
3) Anytime the user checks the 'attempt double spend', one trx is sent to the general network, and
    another is sent to the mining pool where the fee is 50% of the 'double spend'.
4) Anytime a double spend is 'successful' (this pool finds a block) it has a high-payout due to extra fees.

Miners who join this 'unethical' company will see more $$ than those who mine under existing pools.  Scammers of the world would have an 'easy-to-use' way to 'double-spend' some fraction of the time. 

The only way to shutdown this 'unethical' pool / client would be for everyone to wait one confirmation *or* to know your customer.

Considering such an attack could be setup and launched today, we are really operating under a false sense of security to accept 0 confirmation transactions from anonymous sources.

Given that this is possible today, everyone should *assume* it is going on.  Turning an attack into a 'feature' could have added benefits.  For example:

Suppose a thief stole your bitcoins, you could have an alarm triggered the moment you saw a transaction on the network.  You could then quickly issue a 'lock-down' transaction that would send the coins to a 'dispute mediation' address and include a higher fee.   Honest miners would then see the 'conflict' and prefer the transaction to the dispute mediator over the earlier transaction for 2 reasons:  1) reputation and 2) financial incentive. 

The reality is that miners could already offer this 'service', but if it was 'built-in' then everyone would be able to take advantage of 'trx replacement' as a SOLUTION to fraud/theft instead of just a way to commit fraud/theft.

[caveden]

The only way to shutdown this 'unethical' pool / client would be for everyone to wait one confirmation *or* to know your customer.

You could also try to blacklist fraudsters' address with this technique.
Additionally, merchants willing to accept 0-conf transactions from unknown customers could also subscribe to insurance contracts that would reimburse the merchant in case of loss. To decrease the chances of fraud, these insurers could directly finance miners (send a part of their premium to "honest" miners which don't accept double-spends, at least not for insured transactions). The insurers could also make assurance contracts, but I'm not sure if you can select a minimal transaction-set you want to assure, would need to reread about it.

Considering such an attack could be setup and launched today, we are really operating under a false sense of security to accept 0 confirmation transactions from anonymous sources.

Given that this is possible today, everyone should *assume* it is going on. 

Perhaps. On the other hand... it's not really happening. There are other things at play. For significant amounts, people don't accept 0-conf from unknown customers. And for cheap transactions.. is it really worth trying to defraud the merchant? Also... most people are honest and wouldn't steal.
So, maybe it's just too early to start worrying about this.

Suppose a thief stole your bitcoins, you could have an alarm triggered the moment you saw a transaction on the network.

Yes, clients should sound alarms if they detect a double-spend attempt. Also, if eventually this "malicious pool" is created, merchants' clients should even try to subscribe to it in order to known if they are under attack. The only problem is that the pool could easily disconnect those that don't submit a minimum amount of shares.

You could then quickly issue a 'lock-down' transaction that would send the coins to a 'dispute mediation' address and include a higher fee.   Honest miners would then see the 'conflict' and prefer the transaction to the dispute mediator over the earlier transaction for 2 reasons:  1) reputation and 2) financial incentive. 

I wouldn't call that "dispute mediation" but yeah, that's a good idea. Insurers could also attempt that. (you should not expect merchants themselves to take all these measures... merchants don't want to worry with these technicalities... they have their own business to focus on)

[Maged]

The reality is that miners could already offer this 'service', but if it was 'built-in' then everyone would be able to take advantage of 'trx replacement' as a SOLUTION to fraud/theft instead of just a way to commit fraud/theft.

No. Anybody spending money that is willing to defraud the receiver of the money would be willing to spend up to all but one satoshi in fees to commit a double-spend, unless they are an uneconomic attacker. Since we're assuming the worst in this thread, we must assume this to always be the case. Therefore, the only defense to this attack is for the victim of the double-spend to re-spend the entirety of the transaction to fees, preventing the attacker from winning most of the time. Therefore, the best SOLUTION to fraud/theft is the current one, because honest nodes would ignore BOTH transactions with larger fees (the re-spend to fees on the merchant side could be a replacement of a send-to-self transaction broadcast when the initial transaction was received), meaning that attackers would almost never win (they could still win thanks to propagation delay) and merchants would not lose in proportion to how many honest nodes there are.

[Ashaman]

-1
IMHO, this breaks far more than it fixes.

[iCEBREAKER]

Reality check - when I look around at the complaints people have about Bitcoin, merchants constantly being double spent doesn't come up. It isn't "snake oil" to point this out and you should give the people actually using Bitcoin more credit - they know how to avoid losses.

Your double-spending wallet edition is just theoretical - you have to find miners that will take your double spends, and then you have to find users who are willing to accept that most often their attempts to double spend will fail because the bad miners won't find the next block. And then people who use it will discover that if they aren't perfectly anonymous then they will get taken to court for wire fraud, and most likely they will lose.

So there are lots of reasons why people might not do this. And that may explain why this topic is so old and stale. It was being brought up back in 2010 and here we are, years later, people still arguing about this topic and yet there are many more thousands of merchants still accepting these transactions and not losing money.

I remember another topic that used to create endless raging arguments, the 10 minute block interval. Eventually that was solved by Charlie creating Litecoin and now people who hate the 10 minute wait can just go use that instead of creating endless forum threads. I think it's a good way to resolve such disputes - go set up an alt coin that works the way you think it should and then let the market figure it out. Or maybe the Litecoin guys would be willing to incorporate such a change.

Update: Hearn@google.mil's FUD and bitchcraft has failed; RBF isn't old and stale any more.

Peter Todd Fri, 19 Jun 2015

Yesterday F2Pool, currently the largest pool with 21% of the hashing
power, enabled full replace-by-fee (RBF) support after discussions with
me. This means that transactions that F2Pool has will be replaced if a
conflicting transaction pays a higher fee. There are no requirements for
the replacement transaction to pay addresses that were paid by the
previous transaction.

Instead of creating endless forum threads about 1MB blocks, why don't you go set up an alt coin that works the way you think it should and then let the market figure it out?

[phelix]

Peter Todd Fri, 19 Jun 2015

Yesterday F2Pool, currently the largest pool with 21% of the hashing
power, enabled full replace-by-fee (RBF) support after discussions with
me. This means that transactions that F2Pool has will be replaced if a
conflicting transaction pays a higher fee. There are no requirements for
the replacement transaction to pay addresses that were paid by the
previous transaction.

Haha, that went well.


It's an illusion to believe Bitcoin is a completely trust less system. We need a majority of miners to act responsibly. This has shown in the 2013 database fork and recently with the lazy block validating.

There is no problem trusting the miners - only with pools controlling too much mining power.

IMHO something like a mining codex will be established sooner or later.

[JorgeStolfi]

It's an illusion to believe Bitcoin is a completely trust less system. We need a majority of miners to act responsibly. This has shown in the 2013 database fork and recently with the lazy block validating.

There is no problem trusting the miners - only with pools controlling too much mining power.

Last time I checked, the top 3 pools and the top 4 Chinese pools had ~55% of the total hashpower.  The top 5 pools (the Chinese one plus the Ukranian BitFury) had over 70%.

Objectively, bitcoin failed.  Users and holders must trust that those top 5 miners will not do anything nasty.  If they conspired to put all other miners out of business, they could do it.  If they agreed to block an account forever, they could do it.  If they agreed to make some change to the protocol, they could easily force everybody do accept it.  There would be much cursing ang grawing of teeth, but the only choices for all users and holders would be to either accept the change, of lose their coins.  Since the whole point of bitcoin was to eliminate the need to trust an intermediary, the system has become pointless.

It is often stated that miners don't have that power, and it is the "economic majority" that matters; or that they will never want to do it because "it would destroy the value of their investment".  That is bullshit, it is the fairy tale that has been spun to hide a problem that has no solution.  Indeed, if the miners do force a change in the protocol, those same people will spin the tale that it was no big deal, that the change was in fact approved by the majority, that it is in fact "good for bitcoin" etc. -- because, like now, the value of their investment will depend on preserving the public image of security and stability.

By the way, we know that the Chinese miners can get together and agree to a common policy, with a document signed and stamped with red stars.

This is not the way that bitcoin was supposed to be.  Implicit in the original design was the assumption that, while full relaying nodes might end up restricted to businesses that could afford to run a server 24/7,  mining would be done by ordinary clients, as an alternative to buying coins.  As long as that was the case, having the integrity of the system depend on a mining majority would actually ensure the independence and security of the system.

Satoshi was quite upset when someone published the first GPU mining program, because such greedy miners would invalidate mining by ordinary clients.  That problem unfortunately got worse and worse, ending with the current situation -- where the network is just one small step away from being run and controlled by a closed consortium of private miners.

The immediate cause of the problem was the disparity between the fixed block reward (currently 25 BTC/bk) and the market price of the coin (currently 270 USD/BTC).  That combination made mining into a business with an extremely large revenue stream, now still ~1 million USD/day  By comparison, BitPay, the largest bitcoin payment processor in 2014, processed less than 0.5 million USD/day during that year -- including payments related to mining (i.e., internal to the bitcoin system as a whole) and purchase of precious metals (which was basically investors switching from bitcoin to another investment asset).  Even assuming that BitPay processed only a fraction of all the e-payments using bitcoin, the cost of maintaining the network (including or excluding the miner's profit) is totally out of proportion to the actual use of bitcoin as a currency -- the purpose for which it was designed and implemented.

The high price of bitcoin, in turn, resulted from speculative investment, fueled by expectations of extremely high value in some vague future, based in turn on its alleged scarcity, "deflationary" character, and dreams of it capturing a significant slice of the credit card market.  While the future may still bring surprises, those claims have been largely debunked since the crash of the 2013 bubble.

Another consequence of this overvaluation of the coin is that the huge cost of mining is not borne by the users of the coin, or by its current investors, but by the new investors who are buying the coin today.  Theirs is the only money flowing into the bitcoin system: that money flows out of the system as miners' revenue (1 million USD/day), the payoff of the early investors who are reducing their holdings (an unknown amount) and part of the fees of bitcoin exchanges, payment processors, and other bitcoin services (also an unknown amount, but probably much less than 1 million USD/day).

Because of this unhealthy economic structure, the bitcoin system is creating an increasing mountain of "moral debt": the money that people have invested in bitcoins, and expect to get back with at least some profit.  Of course, there is no entity guaranteeing to pay this debt.  It is not possible to compute this "Bitcoin National Debt" (BND), because there is no way to know the price that each bitcoin holder paid for his coins; but we can tell that it is somewhere between 400 million and 17 billion USD.  The only way the current holders can recover their investment is by selling their coins to new investors; but that will only increase the BND...  Obviously this snowball rolling cannot go on forever, and it can only end in tears.

[iCEBREAKER]

Last time I checked, the top 3 pools and the top 4 Chinese pools had ~55% of the total hashpower.  The top 5 pools (the Chinese one plus the Ukranian BitFury) had over 70%.

Objectively, bitcoin failed.  Users and holders must trust that those top 5 miners will not do anything nasty.  If they conspired to put all other miners out of business, they could do it.  If they agreed to block an account forever, they could do it.  If they agreed to make some change to the protocol, they could easily force everybody do accept it.

You keep using the word "could" as if it implies 'likely' or 'easily and without consequences.'

The sun "could" suddenly/irrationally/maliciously explode and destroy all copies of the blockchain: ZOMG BITCOIN HAS OBJECTIVELY FAILED!!1!

Bitcoin, unlike Petrobras or UA or the NY/Shenzhen/Greek stock exchanges, is working perfectly.  I can easily and securely and almost instantly transfer value equivalent to millions of dollars to anyone anywhere, all for about the price of a 1st class postage stamp.

Sure, I don't have metaphysical certainty that nothing "could" possibly go wrong, but then again I'm not an obsessed try-hard buttcoiner either.   

[JorgeStolfi]

You keep using the word "could" as if it implies 'likely' or 'easily and without consequences.'

I think that it is very likely that, sooner or later, the miners will use their power to force changes to the protocol. 

It will not be totally trivial, but it will be much easier and quicker than the BIP66 or BIP100/BIP101 changes.  There would be no lengthy discussion on forums, blockchain voting, etc.  They cartel will just announce their decision to change the protocol with a couple of months in advance, with a suitable "for the good of bitcoin" spin.  They will put up the modified programs for download, that trigger a hard fork at a specified block number.  They will warn everybody that those who fail upgrade before the deadline will be unable to use their coins until they do.

Knowing that the cartel means what it says, most clients, nodes, services, and non-cartel miners will upgrade before the fork, and will sail smoothly through it.   The cartel will use their hashpower to kill the old branch, if some recalcitrant miners will insist on mining it; then any clients still running the old version will be unable to move their coins.  Those laggards will either upgrade (and find their coins still where they left them) or lose their coins.  And that will be it.

bitcoin is working perfectly.  I can easily and securely and almost instantly transfer value equivalent to millions of dollars to anyone anywhere, all for about the price of a 1st class postage stamp.

Indeed, I am convinced that most bitcoiners will not care about a protocol change imposed by a mining cartel, if it does not affect them directly.  In fact they will support the cartel and pretend that they approve the change, to preserve the value of their coin.  I am pretty certain that you will be among them.  If you don't have a problem with 5 pools having 70% of the power, or with the 5$ cost of your transaction being subsidized by investors, or with the growing pyramid of "debt" -- then you surely will not have a problem with a mere postponement of the next halving, or with the extinction of independent relay nodes, for example.

[JorgeStolfi]

I think that it is very likely that, sooner or later, the miners will use their power to force changes to the protocol. 

Since we are predicting the future, I'm going to run with this...

...And some users won't agree with those changes. And those user will want to make transactions on the original Bitcoin block chain. And they will pay handsomely to do so. And some miners will want to earn those transaction fees so they will mine on the "old" chain. And we will have a fork. And we will have two functioning yet distinct block chains. And everyone who controlled private keys before the fork will have coins on both chains. And some exchanges will see an opportunity to gather fees on the exchange of fork coins and original bitcoins. And ultimately, the chains will survive depending on whether or not people value the properties of the protocols which produce those chains.

Personally, I can't want to see it all unfold, and I really, really want to trade some fork coins for original bitcoins.

Well, forking bitcoin would be a simple and natural solution to the scaling problem. 

However, the changes that the cartel may want to make will be such that they increase their revenue, or bring some other benefit.  In that case, the compensation that the orhodox users would be willing to pay may not be enough for the cartel to tolerate the old branch.

For example, suppose that the cartel wants to postpone the next halving for 2 years.  They could let the old chain prosper, and put some of their miners to work on it.  However, in my math, that would give them tens of millions of dollars less revenue that if they kill the old chain immediately at the fork, and put all miners to work on the new one.

[phelix]

Last time I checked, the top 3 pools and the top 4 Chinese pools had ~55% of the total hashpower.  The top 5 pools (the Chinese one plus the Ukranian BitFury) had over 70%.

Objectively, bitcoin failed.  Users and holders must trust that those top 5 miners will not do anything nasty.  If they conspired to put all other miners out of business, they could do it.  If they agreed to block an account forever, they could do it.

It seems you are mixing miners and pools? Pools have only short term power because miners will leave fishy pools. RBF in F2Pool was a good example for that.

Satoshi was quite upset when someone published the first GPU mining program

Do you have a link? I'm honestly interested.

Obviously this snowball rolling cannot go on forever, and it can only end in tears.

As it did with gold?

[iCEBREAKER]

Personally, I can't want to see it all unfold, and I really, really want to trade some fork coins for original bitcoins.

Indeed, the only known possible counter to MPEX's Gavincoin Short tactic is MAD.  (Implying the only way to win is not to play.)

Well, forking bitcoin would be a simple and natural solution to the scaling problem. 

"Would be?"

There are hundreds of forks which already provide parallel capacity to augment the Main Chain, LTC being the most significant.

bitcoin is working perfectly.  I can easily and securely and almost instantly transfer value equivalent to millions of dollars to anyone anywhere, all for about the price of a 1st class postage stamp.

Indeed, I am convinced that most bitcoiners will not care about a protocol change imposed by a mining cartel, if it does not affect them directly.  In fact they will support the cartel and pretend that they approve the change, to preserve the value of their coin.  I am pretty certain that you will be among them.  If you don't have a problem with 5 pools having 70% of the power, or with the 5$ cost of your transaction being subsidized by investors, or with the growing pyramid of "debt" -- then you surely will not have a problem with a mere postponement of the next halving, or with the extinction of independent relay nodes, for example.

I do have a problem with "the 5$ cost of your transaction being subsidized by investors."

You must have missed my participation in the furious blocksize debate raging in cypherdoc's thread.

TL;DR:  My position is the network should be weaned off block subsidies ASAP, by allowing the fee market to mature (IE ossify) at 1MB by charging the highest price the market will bear, rather than continue (via larger blocks) the outrageous undercharging practice whereby users pay a nickle to use 5$ or more with of electricity/infrastructure/C&M/etc.

Sorry, it was my mistake to emphasize the value-transaction aspect of Bitcoin to the exclusion of its primary value-storage function.

Allow me to add:

bitcoin is working perfectly.  I can easily and securely and almost instantly store value equivalent to millions of dollars and access it anywhere anytime, all for about the price of a 1st class postage stamp.

The store of value function is why you are wrong about "most bitcoiners" not caring about "a protocol change imposed by a mining cartel."

Perhaps on a one-man-one-vote basis you are right, but the economic power majority of BTC (the whales) are most certainly directly affected by a 70%/5 pool coup attempt.

They simply won't allow it, and will prevent it via the mechanisms graciously explained by Holliday.

For someone who has studied BTC for so long and in such depth, your objections and concerns sound very much like the knee-jerk reactions of someone (very intelligent of course) hearing for the first time about Satoshi's whitepaper.

The system's design accounts for the scenario you believe "could" happen, and its incentive structure was calibrated to keep that theoretical possibility in the realm of imagination.

The 'ZOMGWTF if the top [n] pools collude - BTC is doomed!!1!' FUD has for years been rehashed and debunked.

And yet, there you are, proclaiming the Death Of Bitcoin^TM for what, the 51st time?   

[TransaDox]

Objectively, bitcoin failed.  Users and holders must trust that those top 5 miners will not do anything nasty.  If they conspired to put all other miners out of business, they could do it.  If they agreed to block an account forever, they could do it.  If they agreed to make some change to the protocol, they could easily force everybody do accept it.  There would be much cursing ang grawing of teeth, but the only choices for all users and holders would be to either accept the change, of lose their coins.  Since the whole point of bitcoin was to eliminate the need to trust an intermediary, the system has become pointless.

You know what? I think they should - before some government does when we can't do anything about it. At least for a 48hr period to demonstrate the consequences and it is a real threat that needs to be addressed. Lock the main devs bitcoin addresses for a couple of days to focus their attention; first on the testnet then on the real net. Put out a public warning a week before and goad the devs to stop them if they can. Treat it as a white-hat DOS test with full disclosure.

[JorgeStolfi]

Satoshi was quite upset when someone published the first GPU mining program

Do you have a link? I'm honestly interested.

Sorry, my mind exaggerated the memory it seems.  I cannot find his reaction to the first actual GPU miner (if there was one), but there is a well known plea of his, from Dec/2009, for a moratorium on such miners:

https://bitcointalk.org/index.php?topic=12.msg54#msg54

The average total coins generated across the network per day stays the same.  Faster machines just get a larger share than slower machines.  If everyone bought faster machines, they wouldn't get more coins than before.

We should have a gentleman's agreement to postpone the GPU arms race as long as we can for the good of the network.  It's much easer to get new users up to speed if they don't have to worry about GPU drivers and compatibility.  It's nice how anyone with just a CPU can compete fairly equally right now.

[JorgeStolfi]

It seems you are mixing miners and pools? Pools have only short term power because miners will leave fishy pools. RBF in F2Pool was a good example for that.

F2Pool is still the second-largest pool, after AntPool; in spite of RBF and their role in the "Fork of July" incident (and of them saying that, sorry, but they will not change their methods).

Define "fishy".  Most likely, a miner's cartel will use its power to increase their total revenue over some future time span.  That means more money for their members.  If the pool managers conclude that their plan is worth the trouble and risk, why would the pool members dislike it?

There are hundreds of forks which already provide parallel capacity to augment the Main Chain, LTC being the most significant.

Those are "deep forks" that split before their genesis blocks.  

I meant a hard fork with network partition and coin split, like the Beast of Apocalypse of the Anagavinists.  The fork creates two independent bitcoins ("series A" and "series B"), each with its own set of miners and nodes, and with its own blockchain.  Each blockchain is invalid by the other coin's rule, but the chains share a common prefix that is valid for both.  So the history of both coins is the same up to the fork, and just after that each person owns the same amount of coins in the same addresses, on both chains.  Each person can move either coin independently by using the proper version of software and wallet.  

Each coin will have its exchanges and market price. The price (and therefore market cap) of the original bitcoin will probably be split among the two clones.   Then, those who don't like one of the coins can sell all their holdings of the same and buy more of the other coin.  One nice effect is that the nertwork capacity of each coin will be the same as that of the original bitcoin.  So the two coins together can support twice as many users; however, to send payments across the "ocean" between them, people will have to use an exchange or a broker.

I do have a problem with "the 5$ cost of your transaction being subsidized by investors."

You must have missed my participation in the furious blocksize debate raging in cypherdoc's thread.

TL;DR:  My position is the network should be weaned off block subsidies ASAP, by allowing the fee market to mature (IE ossify) at 1MB by charging the highest price the market will bear, rather than continue (via larger blocks) the outrageous undercharging practice whereby users pay a nickle to use 5$ or more with of electricity/infrastructure/C&M/etc.

Good that you agree about the block reward problem.  But you must have missed my preaching on /r/bitcoin about the illusion of the "fee market".

The "new devs" are impatient for the network to saturate so that the "fee market" will arise, and they have invented several spiffy gadgets to handle it that they are dying to use.  No matter how people explain, they don't want to understand that

* even 80% of saturation would render the use of bitcoin a nightmare, driving users away from bitcoin and stopping its growth well before full saturation.

* when the traffic will stop growing, say at ~80% of capacity in daily average, the "traffic jams" will arise erratically at peak hours and at random variations  of demand and supply;

* the queue lengths will vary pretty fast during a jam,  and will be emptied in a few hours;

* the fees will be useless outside of the jams;

* the right fees will be impossible to predict during a jam, because each jam will be different, the queue status will vary too fast;

* the right fee that you should use to be among the first 2500 entries of the queue that will get into the next block depends on the 3000 transactions that will be issued in the next 10 minutes, by 3000 clients who are trying to choose a fee that will put their transactions ahead of yours;

* twidding the fees will not reduce the average delay by one second, only make everybody spend more money and make the individual delays more unpredictable;

* many clients will end up paying much more than the standard fee, only to wait more than if the transactions were processed first-come, first served;

* most clients -- espcially those who issue important transactions -- have better things to do with their time than sit two hours watching the queues and playing the silly "fee hopscotch" game.

It is useless to point out to them that no business in the world, from lemonade stands to airline manufacturers, uses (or could use) anything remotely like this bizarre pricing mechanism.  It is useless to explain to them that the "fee market" will not be a "fRee market", because they have this silly libertarian definition of "free market" as "a market without government regulations".

Unfortunately, they are so detached from the real world, so fired up with the "fee market" fantasy, and so anxious to see their toys at work -- that they cannot even understand how that "fee market" will transform the act of paying something with bitcoin from a mildly complicated routine into an absolutely stupid, lengthy, incomprehensible and frustrating game.

They are like kids who set fire to the house because they want to play firemen and put the filre out with their water guns.

The system's design accounts for the scenario you believe "could" happen, and its incentive structure was calibrated to keep that theoretical possibility in the realm of imagination.

No it doesn't.  From the beginning, Satoshi himself was aware, and said so clearly, that the system would work only if a majority of the mining power acted according to their selfish immediate interest of maximinzing their revenue.  If a mining majority decided to do something else, all bets were off.  He was counting on mining remaining decentralized, so that such cartels could not form.

And yet, there you are, proclaiming the Death Of Bitcoin^TM for what, the 51st time?  

Bitcoin has been dead for more than a year, and was very sick before that.  What is still twitching there is a bizarre and terribly expensive payment system that resembles bitcoin, and works like it for some puroses; but with a crucial difference -- it depends on 5 people not having bad ideas.

There is still some hope though: if the price crashes to below 1$, all the industrail miners will go bankrupt, and bitcoin may then return to how it was in 2009, with mining well distributed through the remaining clients.

That, if the "new devs" can be stopped and the fire can be put out before the house burns to the ground.

[iCEBREAKER]

"Fork of July"

Beast of Apocalypse of the Anagavinists

Heh, very droll.  Tres bot mots!   

Yes, I do understand the categorical but academic difference between a chain fork and a codebase fork.  I was intentionally conflating them to illustrate their functional equivalency insofar as adding to BTC's Mainchain additional parallel capacity.

The "new devs" are impatient for the network to saturate so that the "fee market" will arise, and they have invented several spiffy gadgets to handle it that they are dying to use.  No matter how people explain, they don't want to understand that

[generic r/buttcoin litany of Reasons^TM why Bitcoin is dead (but inexplicably hasn't stopped moving yet)]

I'm not sure from where you're getting this "new" adjective.  Adam Back is BTC's grandfather, if not (part of) Satoshi Himself.

Luke-jr as been around since forever.  Ditto Garzik, etc.  AFAIK, gmax is the newest kid on the block(chain).

The ad hom is irrelevant in any case, because the claim you infer from it, that "they have invented several spiffy gadgets to handle it that they are dying to use," is not accurate.  Sidechains and Lightening are spiffy to be sure, but we are seeing them being slowly deployed via the proper (IE whitepaper->peer review->prototype->testnet->listserve feedback->BIP) process, in marked contrast to the Gavinista's insurgent tactics of going straight to the drooling masses and inciting the mob to demand Action Now.

[iCEBREAKER]

they have this silly libertarian definition of "free market" as "a market without government regulations".

Libertarians are not anarchists.  They are in favor of government regulating force/fraud in markets, but not central planning to pick the winners and losers. 

To use a sports analogy, firms are the teams competing to provide the most value to the public, while a proper non-coercive government is the referee who maintains a level playing field.  When the ref gets bribed by one team to make bad calls, the fans in the stands lose.

If you could stop beating your anarchist strawman and pretending it's a victory over libertarians, that would be great.   

[Cryddit]

Politics is unproductive here.

My problem with the current crop of "libertarians" is that they're advocating fundamentally unproductive stuff like destroying public infrastructure.  There's a lot of stuff that's cheaper and more efficient when provided by government, (like roads), and there's a lot of stuff that can't be provided by private firms because other private firms will fuck it up (like clean air and soil) and there's a lot of stuff that supports the economy at large in the long term but does not support any particular private company so private companies won't do it (like universal, USEFUL education). 

My problem with anarchists is that anarchy is merely a transitory state that is a prelude to warlords and petty criminal organizations taking over.  Much larger criminal organizations with subordinate warlords are much more efficient and less dangerous for the population, and eventually start to look like governments that provide actual services - but they take generations to evolve from the first generation of thugs that take over from an anarchy.

[iCEBREAKER]

My problem with the current crop of "libertarians" is that they're advocating fundamentally unproductive stuff like destroying public infrastructure.  There's a lot of stuff that's cheaper and more efficient when provided by government, (like roads), and there's a lot of stuff that can't be provided by private firms because other private firms will fuck it up (like clean air and soil) and there's a lot of stuff that supports the economy at large in the long term but does not support any particular private company so private companies won't do it (like universal, USEFUL education).

I've been involved with small and big L Libertarianism for over 20 years.  Not once have I heard or read anyone propose we all go out and start tearing up the roads, because Death To the Oppressive Orwellian CalTrans or start digging up pipes because Death To the Totalitarian Fascist EBMUD.

Scale matters (the poison is in the dose), and most libertarians are most opposed to Federal provision/regulation of goods/services, but not so much State/local policies (because it's much easier to choose/move between jurisdictions competing for residents than nations).

Your position would be stronger if you at least acknowledged (even implicitly) the smoking crater of ignorance, thuggery, and epic waste our failed public schools have become.

As for "stuff that's cheaper and more efficient when provided by government," the vast majority of people believe that set includes all manner of things which stand to be radically disrupted by Bitcoin, with central banking and its monetary fiat policies being the prime example.

Quick, let's get the government to ban full RBF, because 0-conf tx are a Public Good and double-spending "is a form of payment fraud, exactly like check kiting!"  

[JorgeStolfi]

"Fork of July"
Beast of Apocalypse of the Anagavinists

Heh, very droll.  Tres bot mots!  

The first one is not mine; I got it from /r/bitcoin.  Thanks for the other...  

I'm not sure from where you're getting this "new" [ core dev ] adjective.  

"New" in the sense that they took control of the "reference" implementation after Gavin stepped aside.

The ad hom is irrelevant in any case, because the claim you infer from it, that "they have invented several spiffy gadgets to handle it that they are dying to use," is not accurate.  Sidechains and Lightening are spiffy to be sure

I was referring to replace-by-fee, child-pays-for-parent, and other tools that they have developed for the "fee market".

I understand that BIP66 fixes a malleability bug and a security risk, but I got the impression that it got implemented now because it was necessary for those tools to work.

but we are seeing them being slowly deployed via the proper (IE whitepaper->peer review->prototype->testnet->listserve feedback->BIP) process, in marked contrast to the Gavinista's insurgent tactics of going straight to the drooling masses and inciting the mob to demand Action Now.

AFAIK, Gavin has been discussing the block increase with the other devs for a long time.  He implemented a 20 MB prototype, tested it on the test net with full load, and wrote many blog posts about it.  He sought the main affected parties -- major miners, exchanges, and payment processors, etc. -- and got their support (something that the "new devs" apparently are not willing to do).  But since the "new devs" were bent on the "let it clog" plan, he brought the issue to the general bitcoin public (something that the "new devs" did not do) and went on to support the BitcoinXT code fork.  Which is what any dev is supposed to do when he is unhappy with the evolution of an open source project.

I am not a fan of Gavin; I cannot forget that he supported the Bitcoin Foundation well after it started to smell bad, and apparently still likes the title of "Chief Scientist of TBF" even though he is not being paid by them.  But I cannot have any respect for the "new devs" for their use of ad-hominem and vague technical FUD to prevent a block increase; and for their stated plans to push all person-to-person traffic off the bitcoin network and appropriate the experiment for their company's use.  I cannot understand how the bitcoiners could be happy with that plan...

The Lightning Network is is still in the paper napkin stage, and "Sidechain" is just another word for "something".  But what is certain is that the "overlay network" will have a few large "hubs", and  individual users would have to open accounts with those hubs and lock all their circulating bitcoins in those accounts.  I have a hunch about the names of the first two hubs: one will start with "C" and end with "e", while the other will end with "e" and start with "C"...

[Cryddit]

Oh, yes, I will be the first to acknowledge that USA's current educational system is an utter piece of crap. 

It seems to be the first thing gutted when money is needed elsewhere, the thing most inconsistently supported, and the thing subject to absolutely the worst planning and analysis in the US government (except for perhaps the SEC).  Most damning of all, it is terribly ineffective.  US graduates are generally lower in quality of learning than people who attended school in the third world, and completely inadequate for the personnel needs of our own high-tech businesses. 

We can examine school systems from all over the world of that actually work. We should use that information to fix our own.  But we don't.  Education in the US has been the victim of many different idiots with "solutions" that are simple, easy to talk about, and wrong.

[iCEBREAKER]

Oh, yes, I will be the first to acknowledge that USA's current educational system is an utter piece of crap. 

It seems to be the first thing gutted when money is needed elsewhere, the thing most inconsistently supported, and the thing subject to absolutely the worst planning and analysis in the US government (except for perhaps the SEC).  Most damning of all, it is terribly ineffective.  US graduates are generally lower in quality of learning than people who attended school in the third world, and completely inadequate for the personnel needs of our own high-tech businesses. 

We can examine school systems from all over the world of that actually work. We should use that information to fix our own.  But we don't.  Education in the US has been the victim of many different idiots with "solutions" that are simple, easy to talk about, and wrong.

The USA has many incomparably excellent schools, most of which are in the private sector.  Excellent schools in the public sector owe their excellence to the involvement of private parties (PTA moms, alumni associations) and excel despite their handicap.

The 'poor underfunded schools' canard is bogus.  We lavish money on failing "Taj Mahal" schools (and especially on their corrupt top heavy admin layers).

Anyway, I was hoping you would respond less to the OT digression about schools and more to my ultimate point:

As for "stuff that's cheaper and more efficient when provided by government," the vast majority of people believe that set includes all manner of things which stand to be radically disrupted by Bitcoin, with central banking and its monetary fiat policies being the prime example.

Quick, let's get the government to ban full RBF, because 0-conf tx are a Public Good and double-spending "is a form of payment fraud, exactly like check kiting!"   

[Cryddit]

Why the hell do you keep talking about ducks???

[iCEBREAKER]

The Lightning Network is is still in the paper napkin stage, and "Sidechain" is just another word for "something".  

Why not do a cursory google search before making trivially disprovable assertions?

https://github.com/ElementsProject/lightning

https://github.com/ElementsProject/elements

RBF was just a paper napkin when the OP was posted.  Now it's real.

Why the hell do you keep talking about ducks???

Sorry, I'm terrible at puzzles and riddles.   

[JorgeStolfi]

The Lightning Network is is still in the paper napkin stage, and "Sidechain" is just another word for "something".  

Why not do a cursory google search before making trivially disprovable assertions?

https://github.com/ElementsProject/lightning

There are a lot of sketches and ideas for its parts, but not a design for the whole that is expected to work.  

I have asked several times to Adam, Luke, adn other "new core devs" to provide a simple but 'complete' example of how a week of "life under LN" would be like for, say 3 consumers, 3 merchants, and one hub.  The discussion invariably ends at that point.  Because, when one tries to think how the pieces would fit and spin together, one sees that they wouldn't.

Not to mention that the LN woudl be the exct opposite of what bitcoin was supposed to be.

https://github.com/ElementsProject/elements

The problem with Sidechains is that the definition is (and must be) so vague that anything can be said to be a sidechain.  I have read the whitepaper carefully, and concluded that the old sofa in my living room is a sidechain.  (In fact, the proof of that is trivial.  The proof that it is a good sidechain requires a bit more argument. I hope to post the proofs eventually.)  

The point is that a sidechain is supposed to be designed, implemented, and managed independently of bitcoin; and therefore cannot be trusted to behave in any specific way, except that it cannot break bitcoin -- which no one is supposed to be able to do anyway.  

And, anyway, the core devs themselves admitted that sidechains will not solve the scalability problem.  

RBF was just a paper napkin when the OP was posted.  Now it's real.

RBF and CPFP are not solutions to scalability, or ingredients of the "overlay network".  They are part of the "fee market" concept that the "new devs" claim will make a block size increase unnecessary, even undesirable.  

Yes, unfortunately there are already implementations of RBF; and Peter Todd convinced F2Pool to go ahead and implement his favorite brand of RBF, that makes it easy to do double-spends of 0-confirmation payments.  (Until others warned F2Pool of the risks, and they pulled back.)  

And Luke has just released his own fork of the core, that implements CPFP and also a relay node filter that discards transactions that he considers "spam" (mostly, SatoshiDice and the like).

[Cryddit]

Why the hell do you keep talking about ducks???

Sorry, I'm terrible at puzzles and riddles.   

Um.  Perhaps we have miscommunicated here.  If you are not talking about ducks, then what the heck do you mean when you use the word 'canard'?

[iCEBREAKER]

Why the hell do you keep talking about ducks???

Sorry, I'm terrible at puzzles and riddles.   

Um.  Perhaps we have miscommunicated here.  If you are not talking about ducks, then what the heck do you mean when you use the word 'canard'?

Same thing as Merriam-Wester: "a false or unfounded report or story; especially :  a fabricated report."

Pardon my French!

[TransaDox]

There is still some hope though: if the price crashes to below 1$, all the industrail miners will go bankrupt, and bitcoin may then return to how it was in 2009, with mining well distributed through the remaining clients.

This is the crux of the issue that I have real concerns about but  I don't think it is as trivial as wait until miners go out of business. That's just a boom/bust cycle waiting to oscillate.

I also think the current lack of real opposition to Bitcoin is down to the old financial institutions seeing that the current trend is pushing Bitcoin into a centralized system. They will just step in at some point and start buying up the infrastructure since whoever controls the miners; controls confirmations and therefore the money flow. How would users fare with 3 of the largest mining pools all owned by Goldman Sachs saying they want to make bitcoin inflationary? How would you stop it?

[justusranvier]

There is still some hope though: if the price crashes to below 1$, all the industrail miners will go bankrupt, and bitcoin may then return to how it was in 2009, with mining well distributed through the remaining clients.

This is the crux of the issue that I have real concerns about but  I don't think it is as trivial as wait until miners go out of business. That's just a boom/bust cycle waiting to oscillate.

I also think the current lack of real opposition to Bitcoin is down to the old financial institutions seeing that the current trend is pushing Bitcoin into a centralized system. They will just step in at some point and start buying up the infrastructure since whoever controls the miners; controls confirmations and therefore the money flow. How would users fare with 3 of the largest mining pools all owned by Goldman Sachs saying they want to make bitcoin inflationary? How would you stop it?

Miners dont really control the  protocol like that.

Broken clients would recognize their fake blocks, but only until we fix the broken clients:

https://gist.github.com/justusranvier/451616fa4697b5f25f60

[JorgeStolfi]

Miners dont really control the  protocol like that.

I have explained a few times, here and on reddit, how a miner or cartel that controls the majority of the hashpower could in fact force a change in the protocol against the will of all other players.  

The procedure that the miners would use to achieve that is not complicated, but understanding it requires a bit of social intelligence: the ability to put oneself in other people's place, and think of how they would actually act in response to one's actions and changed circumstances -- rather than how one would like them to act.

Most bitcoiners just refuse to understand the procedure ("it is impossible to make someone understand something when his revenue depends on him not understanding it").  The few who do understand believe that they can counteract it by the "big red button"defense: let the whole network mine the cartel's version, while the faithful bitcoiners will throw their bitcons away, create a new premined altcoin that can be mined only with CPUS, and declare it to be "the" true bitcoin.

Broken clients would recognize their fake blocks, but only until we fix the broken clients:
https://gist.github.com/justusranvier/451616fa4697b5f25f60

Yes, clients can get and recognize "the" valid blockchain as long as they can contact only one honest node; and the verification can be optimized, as you say, if one of the contacts is honest and is aware of the "false" blockchain(s) that other nodes(s) may be serving to unwary clients.

But that will not prevent the miners from imposing a protocol change, because they will do it in the open and there will never be any question about which is their "reformed" chain and which is the "orthodox" one.  In fact, clients would have to upgrade to the miner-provided software to be able to receive and interact with their branch of the fork at all; and would be forced to do so in order to regain control of their coins.

[TransaDox]

The few who do understand believe that they can counteract it by the "big red button"defense: let the whole network mine the cartel's version, while the faithful bitcoiners will throw their bitcons away, create a new premined altcoin that can be mined only with CPUS, and declare it to be "the" true bitcoin.

That's even worse. A group doesn't even have to takeover the network, only the guardian of the button. The benevolent dictator is not a solution to this.

I think the vehement opposition to admitting the issue is because when the dust settles, the solution will be that there is no safe economic model that allows miners to do anything other than mine for bitcoin. The idea that miners will gradually switch to fees to continue making money to pay for their investment will be the undoing since the alternative is for all the miners to become obsolete once all coins are mined or all clients to be able to collect fees.