A successful DOUBLE SPEND US$10000 against OKPAY this morning.

[macbook-air]

All time UTC+08:00:

08:08 – Well before I knew what later have happened, I deposited $10000-worth Bitcoins to BTC-e over OKPAY's Bitcoin payment, I paid OKPAY address 12z2n8YCJw1BEsJhhQPLCTuLqwH341nKnE 211.9093 BTC and 0.0005 BTC as transaction fee.
09:30 – The transaction was included in version 0.8's fork, block 225446
10:08 – Deposit completed, $9800 credited to my BTC-e account
12:53 – After some study, I recognized, the transaction, though included in version 0.8's fork, was never confirmed by the pre-0.8 fork, so I decided to make two double spend transactions on two of the vins of the OKPAY transaction, and broadcasted them with the raw transaction API, 0.001 BTC transaction fee included in each transaction.
13:01 – The double spend transaction was included in pre-0.8 fork block 225446

You should know what happens next...

I bet merchants would think twice before they decide to accept Bitcoins after the incident.

[1476635909]

1476635909

[1476635909]

1476635909

[1476635909]

1476635909

[Jan]

All time UTC+08:00:

08:08 – Well before I knew what later have happened, I deposited $10000-worth Bitcoins to BTC-e over OKPAY's Bitcoin payment, I paid OKPAY address 12z2n8YCJw1BEsJhhQPLCTuLqwH341nKnE 211.9093 BTC and 0.0005 BTC as transaction fee.
09:30 – The transaction was included in version 0.8's fork, block 225446
10:08 – Deposit completed, $9800 credited to my BTC-e account
12:53 – After some study, I recognized, the transaction, though included in version 0.8's fork, was never confirmed by the pre-0.8 fork, so I decided to make two double spend transactions on two of the vins of the OKPAY transaction, and broadcasted them with the raw transaction API, 0.001 BTC transaction fee included in each transaction.
13:01 – The double spend transaction was included in pre-0.8 fork block 225446

You should know what happens next...

I bet merchants would think twice before they decide to accept Bitcoins after the incident.

Ouch...

[bg002h]

All time UTC+08:00:

08:08 – Well before I knew what later have happened, I deposited $10000-worth Bitcoins to BTC-e over OKPAY's Bitcoin payment, I paid OKPAY address 12z2n8YCJw1BEsJhhQPLCTuLqwH341nKnE 211.9093 BTC and 0.0005 BTC as transaction fee.
09:30 – The transaction was included in version 0.8's fork, block 225446
10:08 – Deposit completed, $9800 credited to my BTC-e account
12:53 – After some study, I recognized, the transaction, though included in version 0.8's fork, was never confirmed by the pre-0.8 fork, so I decided to make two double spend transactions on two of the vins of the OKPAY transaction, and broadcasted them with the raw transaction API, 0.001 BTC transaction fee included in each transaction.
13:01 – The double spend transaction was included in pre-0.8 fork block 225446

You should know what happens next...

I bet merchants would think twice before they decide to accept Bitcoins after the incident.

Good thing someone honest was the cause of the double spend...

[Jan]

Here is the tx: http://blockchain.info/tx/12814b8ad57ce5654ba69eb26a52ddae1bff42093ca20cef3ad96fe7fd85d195

[Elwar]

Ugh...this is worse than we thought.

Is this the first double spend?

[yokosan]

[alir]

Guess they don't owe you anymore.

[Largo]

Thats why merchants should have stopped accepting bitcoin deposits, at least those handing out some other form of cash instantly.

The problem was that 0.8 clients were thinking they are in the right fork (because that was actually the case before miners switched to 0.7) so it didnt stop the RPC interface like the 0.7 clients did.

[Isokivi]

Heres a merchant giving 0-fucks about double spends, go for it. www.btctrinkets.com Every single payment goes trough a payment processor (mtgox) and I verify every single payment by hand.

[etotheipi]

Any guesses as to the reason it didn't show up in the pre-0.8 branch?   I don't see any reason that the original transaction could be systematically rejected by one branch and not the other.  Even transactions that hadn't made it into blocks in the pre-0.7 branch should still be in the nodes' memory pools and double-spends rejected.

gmaxwell pointed out that this could happen if an attacker started by flooding the system with duplicate transactions hoping to create disagreement, but it sounds like macbook-air did not do that.

[casascius]

A serious solution is simply for several major stakeholders to publish signed endorsements and/or kills to blocks. Users can subscribe to them, and if a supermajority agrees to kill a block, the merchant at the very least can be configured to stop and go into safe mode.

I have thought of this long ago, now others might take the idea seriously. It was aggressively rejected in the past under the pretense it was too "centralized".  I believe we need it to raise the bar on the risk of a 51% attack.

[elux]

All time UTC+08:00:

08:08 – Well before I knew what later have happened, I deposited $10000-worth Bitcoins to BTC-e over OKPAY's Bitcoin payment, I paid OKPAY address 12z2n8YCJw1BEsJhhQPLCTuLqwH341nKnE 211.9093 BTC and 0.0005 BTC as transaction fee.
09:30 – The transaction was included in version 0.8's fork, block 225446
10:08 – Deposit completed, $9800 credited to my BTC-e account
12:53 – After some study, I recognized, the transaction, though included in version 0.8's fork, was never confirmed by the pre-0.8 fork, so I decided to make two double spend transactions on two of the vins of the OKPAY transaction, and broadcasted them with the raw transaction API, 0.001 BTC transaction fee included in each transaction.
13:01 – The double spend transaction was included in pre-0.8 fork block 225446

You should know what happens next...

I bet merchants would think twice before they decide to accept Bitcoins after the incident.

Ouch...

Here is the tx: http://blockchain.info/tx/12814b8ad57ce5654ba69eb26a52ddae1bff42093ca20cef3ad96fe7fd85d195

And the other: http://blockchain.info/tx-index/59996016

The "unconfirmed" transaction was included in http://blockchain.info/block-index/357971, later orphaned.

Story seems to check out. How many confirmations did the first transaction get?

[TheButterZone]

move topic> https://bitcointalk.org/index.php?board=85.0 ?

[gmaxwell]

gmaxwell pointed out that this could happen if an attacker started by flooding the system with duplicate transactions hoping to create disagreement, but it sounds like macbook-air did not do that.

One way would be for miners on the other fork to already have long mempool queues, and so the transaction was just waiting to get mined but they had it. The miners were restarted to switch from 0.8 to 0.7 and in doing so that instantly expires their mempools, and nothing will resync them. Once an entry is mempool expired then all it takes it the transaction managing to get relayed to one of them.

[MrCrabs]

How serious is this please. Double spends  I thought were impossible .

[Melbustus]

How serious is this please. Double spends  I thought were impossible .

Was only an issue while there were two chains; ie, for a couple hours last night. The possibility was well-known, which is why it was recommended that merchants, exchanges, etc, stop taking payments/deposits until the issue was resolved, which it now is.

[nevafuse]

Any guesses as to the reason it didn't show up in the pre-0.8 branch?   I don't see any reason that the original transaction could be systematically rejected by one branch and not the other.  Even transactions that hadn't made it into blocks in the pre-0.7 branch should still be in the nodes' memory pools and double-spends rejected.

Is this the only successful double spend from last night?  Maybe the difference will be easier to spot if there are others.  Sounds like there may be other fork issues besides the database problem.

[Spekulatius]

@ OP: Did you repay OK? 

[Ivica]

@ OP: Did you repay OK? 

Nope, until they repay 64.91410001 BTC.

[Nick]

A serious solution is simply for several major stakeholders to publish signed endorsements and/or kills to blocks. Users can subscribe to them, and if a supermajority agrees to kill a block, the merchant at the very least can be configured to stop and go into safe mode.

I have thought of this long ago, now others might take the idea seriously. It was aggressively rejected in the past under the pretense it was too "centralized".  I believe we need it to raise the bar on the risk of a 51% attack.

I have just read this on the reddit thread:

Merchants like this probably need to build something into their systems to automatically go into a safe mode if a 2-3 block fork is detected.

This might be an equally effective but less centralized approach. What do you think?

[blablahblah]

How serious is this please. Double spends  I thought were impossible .

Was only an issue while there were two chains; ie, for a couple hours last night. The possibility was well-known, which is why it was recommended that merchants, exchanges, etc, stop taking payments/deposits until the issue was resolved, which it now is.

Yep, I guess strictly speaking it's "single-spend in 2 currencies". But of course, having multiple chains is ideological anathema, therefore there seems to be no infrastructure to handle it.

[RodeoX]

If I am reading this correctly, it could only happen during this weird period when we had several blockchains floating around. Is that correct?

[Piper67]

If I am reading this correctly, it could only happen during this weird period when we had several blockchains floating around. Is that correct?

Yes, the "Schroedinger Cat period" during which the coins are both alive and dead. As soon as the blockchain became whole again, either one or the other has to materialise.

[Elwar]

This might be an equally effective but less centralized approach. What do you think?

How does one detect a fork? Some guy in his basement can create a fork with his own new code but be useless due to most transactions going with the main fork, but it would still be a fork.

[Spaceman_Spiff]

This might be an equally effective but less centralized approach. What do you think?

How does one detect a fork? Some guy in his basement can create a fork with his own new code but be useless due to most transactions going with the main fork, but it would still be a fork.

That is what waiting for x confirmations is for .  

[Atheros]

This might be an equally effective but less centralized approach. What do you think?

How does one detect a fork? Some guy in his basement can create a fork with his own new code but be useless due to most transactions going with the main fork, but it would still be a fork.

Run several versions of Bitcoin and if they start to mutually deviate over three blocks then that is either a fork or is, a least, sufficient reason for merchants to go into safe mode. Sounds to me like a business opportunity.

Actually, if this is implemented, it would make it much safer for alternative full Bitcoin clients to operate because we could detect the inevitable chain splits quickly and safely.

[somenick]

I've heard he was already catched  

[RodeoX]

If I am reading this correctly, it could only happen during this weird period when we had several blockchains floating around. Is that correct?

Yes, the "Schroedinger Cat period" during which the coins are both alive and dead. As soon as the blockchain became whole again, either one or the other has to materialise.

Haha, I like that analogy.

[mccorvic]

This might be an equally effective but less centralized approach. What do you think?

How does one detect a fork? Some guy in his basement can create a fork with his own new code but be useless due to most transactions going with the main fork, but it would still be a fork.

I have a whole drawer full of forks, but so far all have been ignored

[Micon]

Excuse my non-technical, conceptual-only questions / thoughts / checking if I have this right in my head:


1)  There was no *permanent*  "double spend" right?   i.e. the transaction on the .8 fork would have eventually been orphaned / dumped / never credited to the address specified in the .8 block?

2)  The risk here (and reason why SwC shut off all deposits / withdraws for a few hrs until this was resolved) was that during this very odd period a sophisticated user could use multiple 0-conf services that would all at the same time believe they were getting the same coins, but after the blockchain becomes 1 again only 1 of the transactions would actually be included in the "real" blockchain?

also, if this ever happens again, I think we should use this pic:



and sorry if there has already been a 2 Chainz reference.

[iddo]

A serious solution is simply for several major stakeholders to publish signed endorsements and/or kills to blocks. Users can subscribe to them, and if a supermajority agrees to kill a block, the merchant at the very least can be configured to stop and go into safe mode.

I have thought of this long ago, now others might take the idea seriously. It was aggressively rejected in the past under the pretense it was too "centralized".  I believe we need it to raise the bar on the risk of a 51% attack.

Your centralized protocol with few designated major stakeholders is indeed a terrible idea imho, for the reasons that we discussed in your thread. Fully decentralized proof-of-stake protocol can be sound, and can have several additional features that are very beneficial (in exchange for the additional complexity in the work that the nodes perform), but the security analysis should take into account bribe attacks etc., as discussed in the PoA thread. Peer review for the best protocol that we came up with so far would be very welcome, see here.

[mccorvic]

Excuse my non-technical, conceptual-only questions / thoughts / checking if I have this right in my head:


1)  There was no *permanent*  "double spend" right?   i.e. the transaction on the .8 fork would have eventually been orphaned / dumped / never credited to the address specified in the .8 block?

2)  The risk here (and reason why SwC shut off all deposits / withdraws for a few hrs until this was resolved) was that during this very odd period a sophisticated user could use multiple 0-conf services that would all at the same time believe they were getting the same coins, but after the blockchain becomes 1 again only 1 of the transactions would actually be included in the "real" blockchain?

Your points are my understanding.  The double spend was allowed only because the service provider here wasn't on the ball.

[Gabi]

That is why bitcoin is still beta software

[impulse]

Excuse my non-technical, conceptual-only questions / thoughts / checking if I have this right in my head:


1)  There was no *permanent*  "double spend" right?   i.e. the transaction on the .8 fork would have eventually been orphaned / dumped / never credited to the address specified in the .8 block?

2)  The risk here (and reason why SwC shut off all deposits / withdraws for a few hrs until this was resolved) was that during this very odd period a sophisticated user could use multiple 0-conf services that would all at the same time believe they were getting the same coins, but after the blockchain becomes 1 again only 1 of the transactions would actually be included in the "real" blockchain?

Your points are my understanding.  The double spend was allowed only because the service provider here wasn't on the ball.

Yeah, I think it's important to distinguish here for non-techie people that this is not a double-spend within the merged blockchain, this is a double-spend against OKPAY because of their clearing policy and the unusal circumstances of last night. This does not violate the promise that the blockchain prohibits double spends.

[Gabi]

Forcing the 0.7 clients to upgrade to 0.8 instead of orphaning the chain would have avoided that... but would require everyone in the bitcoin world to upgrade...

[molecular]

This might be an equally effective but less centralized approach. What do you think?

How does one detect a fork? Some guy in his basement can create a fork with his own new code but be useless due to most transactions going with the main fork, but it would still be a fork.

Run several versions of Bitcoin and if they start to mutually deviate over three blocks then that is either a fork or is, a least, sufficient reason for merchants to go into safe mode. Sounds to me like a business opportunity.

Cool idea! And yes, this _is_ a business opportunity of, some size even.

The clients could not only be of different codebase (differnt versions and different clients), but also try to be connected to different "parts" of the network in order to also be able to detect a network split. Now add different configuration options and you have a whole bunch of instances you need to run and maintain.

Hmm, that begs the question: why run the instances yourself in the first place? Why not simply ask a bunch of clients about their opinion about the latest block (height and hash or whatever) via the protocol itself? Would that also be workable or does that approach have a problem?

[molecular]

Any guesses as to the reason it didn't show up in the pre-0.8 branch?   I don't see any reason that the original transaction could be systematically rejected by one branch and not the other.  Even transactions that hadn't made it into blocks in the pre-0.7 branch should still be in the nodes' memory pools and double-spends rejected.

Has this been answered?

[Gabi]

Any guesses as to the reason it didn't show up in the pre-0.8 branch?   I don't see any reason that the original transaction could be systematically rejected by one branch and not the other.  Even transactions that hadn't made it into blocks in the pre-0.7 branch should still be in the nodes' memory pools and double-spends rejected.

Has this been answered?

Indeed, this is a good question. Every client should have refused the second transaction, since it was a double spend. So what happened? A miner somehow received it and confirmed it because of the higher fee?

[pera]

[..]
Hmm, that begs the question: why run the instances yourself in the first place? Why not simply ask a bunch of clients about their opinion about the latest block (height and hash or whatever) via the protocol itself? Would that also be workable or does that approach have a problem?

could be something like this be implemented on the client to measure a potential fork?

[molecular]

Any guesses as to the reason it didn't show up in the pre-0.8 branch?   I don't see any reason that the original transaction could be systematically rejected by one branch and not the other.  Even transactions that hadn't made it into blocks in the pre-0.7 branch should still be in the nodes' memory pools and double-spends rejected.

Has this been answered?

Indeed, this is a good question. Every client should have refused the second transaction, since it was a double spend. So what happened? A miner somehow received it and confirmed it because of the higher fee?

Stupid question: did everyones 0.7-client crash (or restart or clear the mempool) upon failure to commit the big block? That would explain it, no?

If that's the reason, maybe the "mempool" should be a "diskpool".

[Wekkel]

A great opportunity to upgrade the software 

[Gabi]

Clients didn't crash, they just refused the "weird" block. Or did they crash?

[molecular]

[..]
Hmm, that begs the question: why run the instances yourself in the first place? Why not simply ask a bunch of clients about their opinion about the latest block (height and hash or whatever) via the protocol itself? Would that also be workable or does that approach have a problem?

could be something like this be implemented on the client to measure a potential fork?

probably. I don't know enough about the protocol myself to answer this question. But say you have 8 connections and one reports a different hash for the newest block (or a lower chain height)... this wouldn't in itself mean much. You'd have to have access to a larger sample, so maybe it'd make sense to implement this as a service with an api that publishes some kind of "alert level" to be used by merchants to halt accepting bitcoin above a certain treshold.

[etotheipi]

gmaxwell answered it:  a large majority of mining power switched to running 0.7 -- which meant restarting the miners and clearing their memory pools.  With the right dynamics, a rebroadcast would not repropagate because most nodes had not been restarted and would not rebroadcast it.  Thus, those miners that restarted, started mining without it, and accepted the double-spend as a first-spend.

That particular theory, if correct, suggests that it's safer to have saved memory pools between loads, to make the keep/drop decision deterministic.  It would also appease those that want to do far-future locked transactions, whose transactions cannot, by definition, survive a software update cycle.  At least it would give clients/miners a choice about their locked-tx policy, rather than having it decided for them (drop all tx on restart).

[jago25_98]

Out of interest, how far did the spend get before the 0.8 chain went dominant? I take it that it didn't even get to 1 confirmation? How many confirmations was OKPay requiring at the time? 6 confirms at gox annoys me so it's a useful case study.

[dooglus]

That particular theory, if correct, suggests that it's safer to have saved memory pools between loads

Would that have helped in this situation?  The first spend had already been mined on the 0.8 chain, and so wouldn't have been in the mempool any more.  Even if the mempool persisted when the miner switched to 0.7, the first spend wouldn't be known and so the double spend would still be added to the mempool when broadcast.

[Aseras]

Any guesses as to the reason it didn't show up in the pre-0.8 branch?   I don't see any reason that the original transaction could be systematically rejected by one branch and not the other.  Even transactions that hadn't made it into blocks in the pre-0.7 branch should still be in the nodes' memory pools and double-spends rejected.

Has this been answered?

Perfect timing, the second transaction went in because it was mined, and at the same time enough pools reverted while the chain was reloaded that it was 51% and the other block was considered orphaned so it was included.

[molecular]

gmaxwell answered it:  a large majority of mining power switched to running 0.7 -- which meant restarting the miners and clearing their memory pools.  With the right dynamics, a rebroadcast would not repropagate because most nodes had not been restarted and would not rebroadcast it.  Thus, those miners that restarted, started mining without it, and accepted the double-spend as a first-spend.

gmaxwell and gwillen just enlightened me a little further on #bitcoin-dev. The scenario seems likely to me now especially after I got the info that only about 10% of the miners had been on 0.7 before the split. They probably had the initial TX, but just didn't find a block in time.

I also understand why the 0.7 chain was chosen: if the 0.8 chains was chosen, double-spends would even now still be possible against merchants who are still asleep about the issue, which is worse.

A question still: rebroadcast? How does that work? Who would rebroadcasts the tx and triggered by what?

That particular theory, if correct, suggests that it's safer to have saved memory pools between loads, to make the keep/drop decision deterministic.

Saved memory pools, or maybe something in the protocol that enables syncing of the mempool from other peers? Sounds good, but again I don't know enough.

EDIT: fixed 0.7/0.8 mixup

[molecular]

That particular theory, if correct, suggests that it's safer to have saved memory pools between loads

Would that have helped in this situation?  The first spend had already been mined on the 0.8 chain, and so wouldn't have been in the mempool any more.  Even if the mempool persisted when the miner switched to 0.7, the first spend wouldn't be known and so the double spend would still be added to the mempool when broadcast.

Probably true.

<jgarzik> another example of mempool-on-startup being useful to miners: https://bitcointalk.org/index.php?topic=152348.msg1617466#msg1617466
<jgarzik> again, I ponder a -mempool-on-startup command line flag
<jgarzik> default off, but present for miners

[TheSeven]

I also understand why the 0.8 chain was chosen: if the 0.7 chains was chosen, double-spends would even now still be possible against merchants who are still asleep about the issue, which is worse.

That's exactly the wrong way round. The 0.7 chain was chosen, because this would have happened if the 0.8 one would have been chosen.

[molecular]

I also understand why the 0.8 chain was chosen: if the 0.7 chains was chosen, double-spends would even now still be possible against merchants who are still asleep about the issue, which is worse.

That's exactly the wrong way round. The 0.7 chain was chosen, because this would have happened if the 0.8 one would have been chosen.

thanks, fixed in my post

[bitcoinbetas]

If I am reading this correctly, it could only happen during this weird period when we had several blockchains floating around. Is that correct?

Correct. Nothing to see here....

[Largo]

I also understand why the 0.8 chain was chosen: if the 0.7 chains was chosen, double-spends would even now still be possible against merchants who are still asleep about the issue, which is worse.

That's exactly the wrong way round. The 0.7 chain was chosen, because this would have happened if the 0.8 one would have been chosen.

If the 0.8 chain was chosen, no double spends would have happened since RPC interface on 0.7 clients was stopped and client showed a 'you might need to upgrade/downgrade' message.

At least for me, ironically it made me upgrade to 0.8, because i noticed it before the threads started.

[Fiyasko]

Oh my god... Do we blame the services? Or the client upgrade?

[Spaceman_Spiff]

Oh my god... Do we blame the services? Or the client upgrade?

That's easy : blame Canada !

http://www.youtube.com/watch?v=bOR38552MJA

[mccorvic]

Oh my god... Do we blame the services? Or the client upgrade?

That's easy : blame Canada !

http://www.youtube.com/watch?v=bOR38552MJA

I blame my mother.

[Piper67]

Oh my god... Do we blame the services? Or the client upgrade?

That's easy : blame Canada !

http://www.youtube.com/watch?v=bOR38552MJA

I blame my mother.

I blame Canada's mother!

[iCEBREAKER]

How does one detect a fork?

Run several versions of Bitcoin and if they start to mutually deviate over three blocks then that is either a fork or is, a least, sufficient reason for merchants to go into safe mode. Sounds to me like a business opportunity.

Actually, if this is implemented, it would make it much safer for alternative full Bitcoin clients to operate because we could detect the inevitable chain splits quickly and safely.

You're reading my mind bro. 

Any service handling serious amounts of BTC should be running a .7 (stable) and a .8 (beta) while checking the outputs for diff.

Also, why didn't .7's large block problem show up on testnet?

[jubalix]

well this is a fine mess one block chain goes sailing of into the abyss, but on the way people still think it ok and so cash in on int

it was the *other block chain* that was actually the real one, with no record of what is no longer the truth.

Oh look my moneys back!!!

Oh look i did not get those bitcoins!

[sd]

Also, why didn't .7's large block problem show up on testnet?

I suspect because nobody was testing for this problem.

Client testing needs to be radically improved if bitcoin is going to be a success.

[arklan]

Also, why didn't .7's large block problem show up on testnet?

I suspect because nobody was testing for this problem.

Client testing needs to be radically improved if bitcoin is going to be a success.

agreed. unfortunately it's kinda difficult. i mean, as someone involved in the testing of bitcoin, i have one pc, with one OS. the tests i can do are limited.

[makomk]

Excuse my non-technical, conceptual-only questions / thoughts / checking if I have this right in my head:


1)  There was no *permanent*  "double spend" right?   i.e. the transaction on the .8 fork would have eventually been orphaned / dumped / never credited to the address specified in the .8 block?

2)  The risk here (and reason why SwC shut off all deposits / withdraws for a few hrs until this was resolved) was that during this very odd period a sophisticated user could use multiple 0-conf services that would all at the same time believe they were getting the same coins, but after the blockchain becomes 1 again only 1 of the transactions would actually be included in the "real" blockchain?

In theory this didn't just affect 0-confirmation services. The transaction on the 0.8 side of the double-spend got 15 confirmations on that fork before it was invalidated by the conflicting transaction from the 0.7 side of the fork, which is more confirmations than anyone requires.

[oakpacific]

Excuse my non-technical, conceptual-only questions / thoughts / checking if I have this right in my head:


1)  There was no *permanent*  "double spend" right?   i.e. the transaction on the .8 fork would have eventually been orphaned / dumped / never credited to the address specified in the .8 block?

2)  The risk here (and reason why SwC shut off all deposits / withdraws for a few hrs until this was resolved) was that during this very odd period a sophisticated user could use multiple 0-conf services that would all at the same time believe they were getting the same coins, but after the blockchain becomes 1 again only 1 of the transactions would actually be included in the "real" blockchain?

In theory this didn't just affect 0-confirmation services. The transaction on the 0.8 side of the double-spend got 15 confirmations on that fork before it was invalidated by the conflicting transaction from the 0.7 side of the fork, which is more confirmations than anyone requires.

So I think the 6-confirmation recommendation should be changed: tell the merchants to wait for as many as possible if time permits, or if the amount is large.

[datafish]

So I think the 6-confirmation recommendation should be changed: tell the merchants to wait for as many as possible if time permits, or if the amount is large.

How about 6 confirmations plus the length of the longest fork difference, which is usually 0 or 1.  I think we need an automated fork monitoring system, with red alerts to the devs, miners and merchants when it gets greater than 2 blocks. 

[oakpacific]

So I think the 6-confirmation recommendation should be changed: tell the merchants to wait for as many as possible if time permits, or if the amount is large.

How about 6 confirmations plus the length of the longest fork difference, which is usually 0 or 1.  I think we need an automated fork monitoring system, with red alerts to the devs, miners and merchants when it gets greater than 2 blocks.  

There would be no way of knowing whether the longest one is the correct one, i.e., someone mined an orphan block and is immediately rejected, yet the main chain's difference to the orphaned chain will still continue to grow, but the monitoring system is a good idea, if both branches are detected to have two or more blocks, everyone gets an alert, and the merchant will keep requesting new confirmations until one branch stops growing for a certain period of time.

[DBordello]

Currently the merchant just polls the transaction for the number of confirmations.  The client never says "this one is non-reversible".  This kind of monitoring system would require the client to make a judgement call.

[iCEBREAKER]

Also, why didn't .7's large block problem show up on testnet?

I suspect because nobody was testing for this problem.

Client testing needs to be radically improved if bitcoin is going to be a success.

agreed. unfortunately it's kinda difficult. i mean, as someone involved in the testing of bitcoin, i have one pc, with one OS. the tests i can do are limited.

Really?  You can't emulate testnets-in-a-VM and try ranges of values for different parameters to see what breaks?

[oakpacific]

Currently the merchant just polls the transaction for the number of confirmations.  The client never says "this one is non-reversible".  This kind of monitoring system would require the client to make a judgement call.

I don't think you even need to change the client, it's about if the merchant delivers the final goods or not, so it could be an additional system that just checks the blockchain for possible forks.

[Maged]

So I think the 6-confirmation recommendation should be changed: tell the merchants to wait for as many as possible if time permits, or if the amount is large.

How about 6 confirmations plus the length of the longest fork difference, which is usually 0 or 1.  I think we need an automated fork monitoring system, with red alerts to the devs, miners and merchants when it gets greater than 2 blocks.  

There would be no way of knowing whether the longest one is the correct one, i.e., someone mined an orphan block and is immediately rejected, yet the main chain's difference to the orphaned chain will still continue to grow, but the monitoring system is a good idea, if both branches are detected to have two or more blocks, everyone gets an alert, and the merchant will keep requesting new confirmations until one branch stops growing for a certain period of time.

More importantly, if there is ever an orphan chain that appears to have more than a few percent of the hashing power, ALL clients should enter safe mode. The only time we wouldn't want that is if we did an intentional hard fork, but in that case the incompatible ("new") chain should have over 90% of the hashing power

[robbonz]

http://coincompare.com/themes/bitcompare/images/bitcoin-fork.jpg

[eldentyrell]

Merchants like this probably need to build something into their systems to automatically go into a safe mode if a 2-3 block fork is detected.

This might be an equally effective but less centralized approach. What do you think?

Yeah, this is probably the solution.

The Satoshi client already watches for long invalid chains -- a chain it thinks is malformed but yet has had more hashpower thrown at it than any other.  The 0.7 clients saw this happen; it's what triggered the "maybe you need to upgrade" message.  Unfortunately this never happened from the perspective of 0.8 clients.  From their perspective the whole network suddenly colluded to perform a massive 51% attack by orphaning a huge part of the chain.

Unfortunately it doesn't look like this can be ruled out in the future -- a situation where the miners have to choose between orphaning a huge branch vs permanently splitting the network (I hope they'll continue choosing the former).  So watching not only for longer invalid chains but also for unusually-long-but-not-longest branches is probably something that needs to happen.

How does one detect a fork?

Run several versions of Bitcoin

You can do it without having to run multiple clients.

Example: if there is a branch more than six blocks long and the forking point is less than 144 blocks (~24 hours) back, stay in safe mode until this is no longer true.  This will protect users on both sides of the fork so long as (a) nobody's trusting transactions with less than six confirmations and (b) the problem is discovered within 24 hours.

[jl2012]

Is this the only sucessful double spend yesterday? Has anyone made a detailed analysis?

[arklan]

Also, why didn't .7's large block problem show up on testnet?

I suspect because nobody was testing for this problem.

Client testing needs to be radically improved if bitcoin is going to be a success.

agreed. unfortunately it's kinda difficult. i mean, as someone involved in the testing of bitcoin, i have one pc, with one OS. the tests i can do are limited.

Really?  You can't emulate testnets-in-a-VM and try ranges of values for different parameters to see what breaks?

while obviously that's possible, i'm sad ot admit it's beyond my knowledge, currently.

[molecular]

So I think the 6-confirmation recommendation should be changed: tell the merchants to wait for as many as possible if time permits, or if the amount is large.

How about 6 confirmations plus the length of the longest fork difference, which is usually 0 or 1.  I think we need an automated fork monitoring system, with red alerts to the devs, miners and merchants when it gets greater than 2 blocks. 

+1

[molecular]

Currently the merchant just polls the transaction for the number of confirmations.  The client never says "this one is non-reversible".  This kind of monitoring system would require the client to make a judgement call.

I don't think you even need to change the client, it's about if the merchant delivers the final goods or not, so it could be an additional system that just checks the blockchain for possible forks.

it's not quite that simple: which blockchain?

[Prattler]

Well to be fair, OKpay did a pretty lousy job.

Cheap, fast, secure – you can only choose 2.

1. Cheap and secure: wait for a big number of confirmations, depending on the transaction size and business type.
2. Fast and secure: you need to have people and automatic warning systems to hold payments when something goes bad.
3. Cheap and fast: don't allow users to cash out big sums.

OKpay went with option 3. It's unfortunate it happened, they would have avoided trouble, had they gone with options 1 or 2.

I don't think this will seriously damage bitcoin. If anything, payment processors will implement warning systems that will automatically suspend payments and prevent this in the future.

[Raoul Duke]

You already have your 64.xxxx BTC. Unconfirmed yet. Now you can sleep 

[Grami]

http://blockchain.info/tx/cf7121622d6b208357b2a115cc75a9be283f8996dd7d6f612923b30f600bcadd

[OKPAY]

This is not the way to do customer service. If not this incident, I don't know when they start to care me and return me the 64.xxx BTC.

The payment has been successfully completed http://blockchain.info/tx/cf7121622d6b208357b2a115cc75a9be283f8996dd7d6f612923b30f600bcadd

Yup the "duplicate" payment has been payed out and thus the amount of 64 BTC was put on hold before user agreed to return the faulty payment. So everything could be solved much faster and easier by contacting support. I suppose it's always a tempting to try to get away with something that not belongs to you.

[S-Fattah]

Well done, guys.
One question: This guy has returned your money?

[OKPAY]

Yes we resolved this situation.

[oakpacific]

Currently the merchant just polls the transaction for the number of confirmations.  The client never says "this one is non-reversible".  This kind of monitoring system would require the client to make a judgement call.

I don't think you even need to change the client, it's about if the merchant delivers the final goods or not, so it could be an additional system that just checks the blockchain for possible forks.

it's not quite that simple: which blockchain?

The simplest way is to  just query a third party, e.g., blockchain.info's API, I am sure there are many other methods, to those who are on the orphaned chain, the main chain is just their orphaned chain, you only need to decide if there is a big difference between the two chains, no need to decide which one is right.

[luv2drnkbr]

Currently the merchant just polls the transaction for the number of confirmations.  The client never says "this one is non-reversible".  This kind of monitoring system would require the client to make a judgement call.

I don't think you even need to change the client, it's about if the merchant delivers the final goods or not, so it could be an additional system that just checks the blockchain for possible forks.

it's not quite that simple: which blockchain?

the longest one.  that's always the main chain.  that's why the two forks were called the 0.8 chain and the 0.7 chain.  chains that aren't the longest are the forks.  for a while the 0.7 chain was the fork until miners agreed with gavin that they should switch to it.  then it became the longest.

[AndrH]

Yes we resolved this situation.

Happy end =)

[matt4054]

I was wondering if this was the reason why OKPAY has still not resumed Bitcoin operations since the fork, while others already have?

I use OKPAY services including Bitcoin on a regular basis and I would hate that such events have long term consequences on OKPAY operations as well as other payment providers.

[allthingsluxury]

Yes we resolved this situation.

Good job.

[Piper67]

Yes we resolved this situation.

On behalf of those of us wanting to use OKPAY and Bitcoin, please don't let the wilful actions of one individual affect your operations too much. By his own admission, he attempted a double spend once he saw what was happening... clearly, a hard fork is not desirable, but the last time there was one was in 2010 (technically, orphaned blocks happen all the time and could be considered mini forks, but we're talking about one that went beyond six confirmations).

[paraipan]

Yes we resolved this situation.

Nice to hear that.

To prevent this happening again in the future consider implementing "bitcoind alerts", and do automatic emergency stop on all bitcoin transactions if an alert ever shows up. It's an elegant and easy way to be warned by the community, and with enough time to avoid being "double-spent".

[Stephen Gornick]

I later manually corrected the issue with another transaction, thus double spend.

For post-mortem analysis, would you mind sharing how that raw transaction was broadcast?  Through bitcoin-Qt/bitcoind I presume?   But was this just connecting to peers or had you actually explicitly connected to a mining pool or well connected node?

The question is that most peers would have rejected it because they should already have known of the original transaction (12814b8ad57ce5654ba69eb26a52ddae1bff42093ca20cef3ad96fe7fd85d195) and not relayed your double spend (762443f6373b7c8b3833d4ad23578fc3099cc29b86d1359d0c0565e3c8614f91).  

With all the nodes reverting to 0.7 and/or starting up with an empty memory pool it is not impossible for your node to have reached one of them with no knowledge of the original transaction and then because your double spend transaction was perfectly valid for that node to relay your transaction got relayed to other new nodes as well.  Eventually then there would be a path that would reach the miner who eventually mined 225466 (the block with the double spend) -- BTC Guild [Edit: Yes, confirmed BTC Guild, per the coinbase].

[JeromeS]

Been meaning to do this since The Night of the Two Chains, finally got the time to do it today.

A list of all transactions included in the .8 fork but not the (current) main chain. i.e. possible double spends.
270 transactions out of 11914 fit that description (2.26%).

• 1. 009d28163768ad497968c9506e23ded6f48e3b0a568a2725056f95ddf36d6688
• 2. 029c7e6daac01cb16dd53cd5c0aaa8cabc7a44c00835af444fa809e28a0689d4
• 3. 04382419df27e5731b58a60fad9e16e1b877e5a969ac0657a88e39ec2645c723
• 4. 04d3936eb4b1c116334b1654d1c5b0b0b5c164775a42250a497de9d0ea8ea106
• 5. 0513528f760aaefeefbb4a5e3216c5ee6d9073e01e6ae2ed820854767a86f7d4
• 6. 051420cf2180fe581c7a1ab0c84f0ab93b181f9033ba9967b4ef15d3a228507b
• 7. 05b934c1ca96bafc8319463bec3485eca38ceb0dce824cc78a983a4c53a1e243
• 8. 064f5d31bfa16507a7deb1619a99db0a5e0673e3e55159dcd62d20855c98b6fb
• 9. 06c5c36fce2621671ef823d147e7968d24382a10158e71a86fcf5737e0d6bfa0
• 10. 073079e5b4e542b88173e1f655a5d6fad22248f8cb8c56faa71f1b4c93190dcf
• 11. 07882ac060210af210be60f4b88f0aa5d4fc71e79dd97d75da1a92d722d45fa6
• 12. 085c940a3aeba128db3c6ec242d9d1577d56b1b120efd5a3fdb95e95c690d2f7
• 13. 09c82a2e5da9e6ebc82de88389e9a06b0d3df87a080ce65f8e3d016f0d192098
• 14. 0ac797ec1dba75d73fe5ae512a69f4fe46f4341ce0168f5840ffefdeb177d267
• 15. 0adf13be0beb55c570d144e1105a6bab5285eb1e9332ea39f5c7449f92f14e11
• 16. 0e145abe023f8eab34888026609f95ef6d374c9b0393c3b4fe518a7399effdda
• 17. 0e9750db78364909771895a3d923736e8118dc6a7eca3692ab0fdb467a4f15ad
• 18. 10247766e92ecb2cacd5625f7e8f3825dc3ab416cea897bf199452511185d52f
• 19. 112770007b01b1935a7216a2d37a2fce69c94768064eda2a3e6d0bbdec6a498d
• 20. 12814b8ad57ce5654ba69eb26a52ddae1bff42093ca20cef3ad96fe7fd85d195       <--- macbook-air's tx
• 21. 1340f4e7de3ccee7da773ea11a6b291bf123bea0b80ad0839dee5302f5e2c355
• 22. 138063e4bdb76feaa511f1e7f9c681eb468ef9140c141671741c965e503b84c6
• 23. 14de6833d13278dd727027ddfca4231669665d7a82bbbe4e062cb50ebc66d998
• 24. 1581e4d992d405cc4213bedc21f5ca43f18e8e21c064b1ef61aa94a9a3326ea2
• 25. 15eec298a273597a384edb2fa68b7c72a6fa3e0908a60199e68ecfc5a0f3b3b4
• 26. 165f8d8a2f5671e05684b9f9e48b1335e4aa5cd632a55789b68733f14533ea39
• 27. 177ce5cc2486cc3d0f0e09ceff000f32ccee1eef9eb42d95af6083ae90f84698
• 28. 19b8900015d974c7bc0c693a8513b5c318e68a58709c13788bbeff2bb747e27f
• 29. 19c97daee7aaaac8d1501347f0839d494d018c654cbb3ffe400b56ebd02a7207
• 30. 1a36677a420e9cbb12a525f9411bc6fb7e2ce7c265879915e03c69d21a5b95df
• 31. 1b98e621efc9b573511f46f4f1e841ddec98b43bb060b760f64115c0132b7c52
• 32. 1c2d0607fdd18325849f6d4776a12c1203166e11b93467818594b22382213708
• 33. 1e1a2fb5efacb20afa8f8ff19df5d0dd75311761c49d7b569232d65dc441dccb
• 34. 1e79e5e40994a272fd052608a0b21525746f468f6df7f79fca4be343a7f3663c
• 35. 1f237d8f3d06380e24ba886bb62454dda70037a50d39bb8408e574edf999d8af
• 36. 20ce6c45da3864a03568349ae60bcc3e4aa1879bd385692b04643a1b53cd51f5
• 37. 2194dfcdca5167c4d7ea37b29cefeb882bc29376efb471a07231a9df4f419ad5
• 38. 22617899df996bb3ec521cfe7a674864891ff7b3c337c3b2f0efde861fc80a4c
• 39. 2524fa5a6e0ba58e9cb888b80f4b7d9819bed1dd1766d34995047846c56cedb8
• 40. 2602d24f948c6bc5f5b5553be29476f1e1fdf2f23afe07219a800a35b4587e5f
• 41. 26190ad541fc58183c21f241827634622a00a3b025d28d34e01da426df878aeb
• 42. 2653e6406560d22b10c3bedc487b5029a8b35bdb405c21b84d79e42a00c4ce7e
• 43. 26a0e5ed1fb4e9d150962b827317c6678b346cc8a1dc53a7ab516d94a4b5bc4b
• 44. 2781584eaa44db6f5a022498854c3a3aff587a7a230f80d16d4cdc542da7dea9
• 45. 27ae60348ddaa4c4c3ca6b01a38150d02d8aeba4e216c282b1a40c6ceb2bb75f
• 46. 27d5d78369b745fbbaf8499c62520fa11929aa890069d300473dfbff0a4fa768
• 47. 27d9ba99889c853a0a3f9d8096b7c06ea7f124f530d334cb33191859d09aab9b
• 48. 2a68a6c37ed74d9c1fcf3c78468990635446c5b53f4eae84bcb06d3092c12e37
• 49. 2b404ef85a22feaceb383b150904418a3ee7b066f58438163351d5185110236f
• 50. 2d798beb98f44026d9da346be7701429c3f0c0e2f80ccd8d0f98a1d97686ede0
• 51. 2fb9feba943a29eafdb5b592514343144888b02bc30a0ac69dd6593dbc2ae965
• 52. 2fdbeaddcaa03e4650a991cb71bcb60f00904fa4d9545ec0597ec2b17450dbe1
• 53. 30814f8c6bb0468af20feeb1d87d131cbd0b721b7c3d6593244c848bdef95ca5
• 54. 33fab1b4a287059b014b1149c47158f65b9beb5413c75b8039e14ab700c88b75
• 55. 342ddd82fbbfb49b3cb6d76f18ad0f14b2b8375e211176df540540724ab6f958
• 56. 346c0022ddef464c1da3555c6d298895d1859246408225d77339c9b9eabd9fc4
• 57. 355d4ea51c3b780cf0b10e8099a06a31484e0060bc140b63f3d6e5fb713ace5e
• 58. 360b30d19ff502a48e3961809ff4d5271dfa8ede62ec32c81b5f8222711dbb8b
• 59. 36fc472aed5bba616bf8c6fc2f26bfbc2fefb6c9fdccbc9c47fc26f060fc0c18
• 60. 37200b5b66a297a40adb7b86d2e9f4ea8f12c85f8340cd985d304e69b6ad022a
• 61. 38aa8913899a6c4022a6dc798d68f49924478f41f7abdadbe2105e370c408f7b
• 62. 3969bcf496835897f07596a1e2cbdf4d6d923e11916b6b5d20a905b8ab934fbf
• 63. 3a67b7b60d18879173d16ace758181d5d5c3eae25d8918516f0156f97bd761d1
• 64. 3ab845e4792130000bd5f6b7d84e657674de2d8d43970295d3c79a3baa6d40cf
• 65. 3bfe72074f66bbf4ecbcd0d0ababa819bc8a4a68c5a46183ffc5b43e210e31d8
• 66. 3d50468c930ad81113f4cb5c6d4533db0de4d9ade3f69ba7e53a5114faa3ebd2
• 67. 3d81bb28669acc76823508e5c7958b2c0829d891d4897c56af30451ff8660ddc
• 68. 3f09d0a74c7b2043512d869c63b3b4a7a9b92741c8918887e2fc903a17b85140
• 69. 405a08daafaadb141f71128cb9c75cfa41f640560226f576c0ef516faa988759
• 70. 4133f75f0f2288fa47d0bcb873de1a44417dc81941b537a95c612ddf36fb4105
• 71. 42372990b92632b9d1f62c119e6385fad5cce404dc5b50b84d3a57ce34ed7dff
• 72. 43c05742191a9ab0eba71896f707b50bddb947706a5e54dddc1fcfce29879f8a
• 73. 453c25df56ff465bd60ac078bc6b542e3e5b862ca600a2c54e665afb4c04b410
• 74. 45c557203dc1b8dd826e209df4925cfd178af0111fee0cd02afd08217a1b4e7c
• 75. 45ca1399e020cb30753bf8c75f713698d2c0439c76143af47c7c11a9cac4d844
• 76. 46551bc2118f9e06a6c1765011b9006b8563b93cad82d493a0741d79b3adfcd8
• 77. 465d41009f7c74124437305b584c4ef578a9e4738d4ff0860cb9ca81d24f16e6
• 78. 46d624f184419dcac5e94616e94b59440c84aa58fa354a99afc678b16a61036a
• 79. 471e152f1fbbf9a73cacf1388a7dc17806f76e2520de7c238913418982fca36e
• 80. 47ef2d33bf2e690e17ba02fdb862206b3c51c7d844febd27bb59126b13dc32e1
• 81. 4a2ec3cbf9d8e73f743b949dea1e1ae7b7461f08b3469fb534c0a776f9ec1ef6
• 82. 4a4f73e73beb3b8042f8cfa63be9fef87822aa93125efd698b9424aceab5d850
• 83. 4b5a8c11b9792ff695a2caae47312096b0ce3d40cdb34da5ed50818ca983780f
• 84. 4b8eb9a9a11131b9ac68ba686d6c6d720aac0280e2bd0114e2e2bdf5523fe0da
• 85. 4bb2c2fa73e92e49bc3c08020e77175bff6626920a546c474f4c952a1cf8da22
• 86. 4c80edb97ca07ed1ec03a8c88ad618171bef49ebf10a5b3e71a011cdd4c1a447
• 87. 4e6e657fe49e4bef30167543b0a9d83bcfddc1492ef3b68d52531602f3ee8bb2
• 88. 510f3f3ccc80d64f315cb2f7cda1f40064633aeb3b7218592a638e7de10bc20b
• 89. 51c5ab5c17a790658d3aca34a5031f11accafc403e8c364404a3b5201564a507
• 90. 51fb9b64103152da62355aa066a2d5211ecc516255b3f9faabaf2c8b7b08d972
• 91. 542dd6794fb05c12395e0d44aaeada7a814e0aa196926bd12377fc2927c6dc23
• 92. 556f794187893daf37d700839e3f4a0f5912e291d73c879b3d6cd9a173535dea
• 93. 57f5ae64f4dc23c40993d37d506794ab638e716a1364a0c1314a76f7743dfaa5
• 94. 58741106bed53fc6909af1785f98bf90681c68008986fc21238d15d0de3ca215
• 95. 5878452d208b21d0605f600b779595b1e9ee3a6a795be4bce18a155ac57b9fbb
• 96. 593e136b327e9f1090eafb519ffb87997933dd49b5c4a50b6db985ac2d80c4bb
• 97. 59624c4bc669e5fe07ccc41254ab7c734a809930524c6fc467ae2b5e9c732545
• 98. 5a7f144af9070702a985a765d1cf39847176521802c9c049885530007867f955
• 99. 5ba94f09a69404eae834e69113aef9aaa4bab29d76097e2801f4258e4ddbd98a
• 100. 5c0c7b5d750f9844a7263ad5fb5bf92424a574169fe95b3501b72ac99a2d8777
• 101. 5d5484a45e3998dd480466563b6a76c73397b391194a536d0a63691c3dde4e0f
• 102. 5d8441dc6aba6ee26478919017ebaaacbf64fb6e836f9650c6be994a99b49b58
• 103. 5dc0929c0f17d1c66a317701a5ca0311de0a42225b6fce9efc029c71fe6b7e18
• 104. 5e1177f69a2ee2136d71ed83e13e7961b8929d61516ceee6ad5d68030d462cc7
• 105. 5f01038e79494187b152962bfae39b81cae03b2ea362ef363fecd2adcff3f1fe
• 106. 5f2eb1e10878840b462580c90f115ce5595359babbfc2f5902cdef07d3455ea3
• 107. 60148aaa14cf0d6479b16ef6c7effd081803138ea722318e65c67fe4f53e254d
• 108. 694b438c8c3f7ebf11a915de86ff0eefe40e9215537412c2c22ea9032acf386f
• 109. 6a3e6afbf968eaae82fb9319ce0763e54f96bd5140510f286edc680ded27a029
• 110. 6b00c57fbf7fb8c83f59790a5382d6d2622c440948940ac844f51f5b56a54944
• 111. 6bca6f29fb74a23bbf1ef0a5395dc513522b9f92e3d7a41e60bd11a21808495b
• 112. 6ca33644948976b01bfb48fc320905eae67a79a1c67b7a5807af44c3666873a8
• 113. 6cbfaed2ce6bb1aaf0cd9a9b21b5c5864df91586be2d2f44bf2d3acfaf02ed2a
• 114. 6f30944238dd9d7a26e634d5202b8cf16d85983ec5360eb0367b05a7e2772ed7
• 115. 6fd1910aed74fe42904860ed7721e8f95741568c118876ad8c27c8cc6f84d665
• 116. 7004d9867544b8b21b2d0c11c98516d72504f335749e6c41b0746f2a8475e8c9
• 117. 712d4953fcff2ef0bf762a2d16b92894925601197d77172a0a305098c054b32d
• 118. 7192807f952b252081d0db0aa7575c4695b945820adaf7776b7189e6b3d86f96
• 119. 7286090c53a7f9d3ad0c46f45daa0ff052ca84a0cf13a6f99520e39790172c88
• 120. 7364203f3e3091aa38748c310201c3a659f85653bb191c444cc075c78dc0a41a
• 121. 73ac8837c2b48bfb1e780e3907ddd214459204bc982c9aa9ed0b22f603dcf1fd
• 122. 749f039022a4d3628795fddba4d5106ac80d431deba3f965857b3d309092e43f
• 123. 79f316a0d1fdbe2cd80c8447797891878e44a250f369ca4a132af300c001198c
• 124. 7a38e6abb5edef03d0f99f4170ad645c2fb1123e991c5532b197ef66223faa9e
• 125. 7c0c1b72936dd98c31d3d97c11c4d930c2167827dce2fc70c7ec157f8b9ca9c5
• 126. 7cb0ea9884bb692c69bfb481e1daf5d5615a31e7c90db46a85f88835fffc28f4
• 127. 7ced1ca13ae92396f9c1d6fbae65bca55952d0111d4b440cf294cbcf00133e88
• 128. 7dd1f1580be103b045990859dbcd2e3153e823a4077fc15f71c63a1ada61ba0b
• 129. 7e13fd12d47a50dcae62d3f9e7f925e7d77d4b0efb49bc4c1ea38cf593c31c0b
• 130. 7ed48ace27a43387773749ffc535da13eca316c32c352665d30b02fccdbfb9d2
• 131. 7f2be3879c28c6dd1ae8c336ba7477821d0df369c15382e854949e6c35c2b9e3
• 132. 7f3c7b0169c2bd8859906cf09f8366762da50cb72261b79eb82cc3514e2382d4
• 133. 7f7b236853afe0b97584f3f3c950db47579955efe9a7a088dfa6ab00c25a9417
• 134. 7fca08836d79c2603df99e9ea776fd7fbaecb237aaa984c6f5369901916c37d7
• 135. 7fd18b4ebc6d14153b8bed763ac5c439e2d80fb741fb10cd79d5c9f3f887fee8
• 136. 8033136a1efa4575beb6ffed603ea01aa871cae78155991e4eea6f06b653932b
• 137. 8141d4d75a74f7aab4ef7e517642b924338fdb9d838a1f4b8064653cbfd0d303
• 138. 83a69dc0661bc16cebcbb92af5d6cbf37d8abb0f51b4b3bfa91036fcdd77af9b
• 139. 83df282a8580fbd0799f25a4a833b350aa6719394fdd3b66389fd96425e17b14
• 140. 84aa0952ce49f7ecbedb7f5b433ce41661389b9cdf86dfad6c7d8581ba7b9a32
• 141. 84c71f44560cd16523755f8956cf38eb7f6f6b386001e661963dcb1e961d2da9
• 142. 86a8e4bedbde7c6895742ed3a95901c8a501f07691393c5ebe3ac76038502c38
• 143. 86a96c7c580e78fb9595aa0cf69b0901c0d74eae8006c492fc03e24fbc5e0276
• 144. 87073d73fec85036e7b7526e8094d3336dfa93d10e2e50e4aebba73acf13e286
• 145. 87284b724667f220ac4db1e575f460549bbb7be6e03c88cbd3766548e8501159
• 146. 8795cece9b25a2b9dc9bb7849f3ca620405ef79c1851c89e33961c21fbf022b2
• 147. 8876207ff497db810cddf42a9d8b0eab6e9ea0c05f9f6e576d60de1bf50e6e76
• 148. 8b563b12f2fef27a1cbd44b2dd36f46126cdf2b7dd2c66c2963c40e574b7fe1f
• 149. 8e32d277c61da6dcdd013ade0501d75785201a36d172a5f79339791c711c4a48
• 150. 8f47bb34a597588483589f0b12bd5799e0d15690d2ad0997bba9a5e8f7dc749a
• 151. 901a1d5d3148e56cc5e7b6563e610b13d5edb752ecfa2ed1ace4702c1984cbef
• 152. 908cbbd3b36382e63d5ee9d9a4d822022fb1da1c1c81c0090f6ebb0c464c4c78
• 153. 9250f64609c68d32a40d7748d36fac9234f04a3feebc9f7139959cb8b289e906
• 154. 93409a5796d4ddc00be5a255971123d375d2d96e893ae429b787257d2ce51aae
• 155. 940670d0a1b9784c3d83134922a1d7c1dc7805363dbaca4e5a8f61a19130ff83
• 156. 9516fcc407e7b9e01384a98d4ad9ce426cbd49dc34801837c807e26552eddbf9
• 157. 960550e4d943db66ca30d863c429151f206cc81d8d14b7261726188f85fc1581
• 158. 97536cb9dd68ba25bfe66caa480abeaaad2355dedf5cfda64dec34104615e221
• 159. 9812bce46dd875c329434f2910364535a89d52d665a9b02337107161e8467859
• 160. 99fdfe009f53ea5a4a0240f94bf5db72c9bec1a1794411ba36de997c696986ef
• 161. 9a018f587ae3b1172e63a45ae5ea791a0848803892b7402d06ffd5deaedf5533
• 162. 9bfae0ea23bc15db2b0cfba42926e32a396103ca080baa73349936f38c6ab7b0
• 163. 9c5a8f12602337b009c401b34c953d98d4b04b906d98076a6bc796f0dc3d3e83
• 164. 9d39389811da060fc25270dce0d51ca43547ee1f39edd437168870c682c32b48
• 165. 9da8cdde46cf635be8baab94d3980155c653f1d305ce815bce30131e1790ccb2
• 166. 9daf43ff61d2adec45c6728d151ea1041e0d632116851c28ce1d167e6dc6bb56
• 167. 9f84fbb5d6d09b47d210490f6a624d1f4c0588bbe44ee1155122ddccca67afd2
• 168. a0242a7508a34a823af1b0ffe43f732b3b3ffdbddea019b9a1826bc7b1cb1142
• 169. a0664cae54a314e62c414c3d35d04138ace6d52c26c852775506653bc11b4549
• 170. a091d381200c72eda6b297b24c4b85f22d706cefd8dae1f9d07569a682b73020
• 171. a10bd194cdbf9aa4c12eb0b120056998a081a9b0d93d70570edff24dec831f90
• 172. a15be59001b48449d7bcb4f650428ac1b070b0a4d500839eaba223b8703a8e1e
• 173. a180ca116ea867c1847ac99f6f027a943f35292f634955910c47f6f91a3b7b74
• 174. a2bfcce2a518d5ce830751dd739962e556c11da565b17c8dfb75d69a51a80fb3
• 175. a49431850f3ebf467872d4b763a5a6c565cfe306f76fb96ed83335190c181068
• 176. a6addf63b82c350558dacd27e0eb69d6ae5710edf3564fc6c9cdd82eeec60518
• 177. a6b0a8205b21bc9353d72014d9ad541cb8016cd3705562d506dcafbeefca1ada
• 178. a6c67e023f5901333e0fca125a9350980e16102c056726bab752ecd2c95317ef
• 179. a6e3cc4a8fb3bedc1071c8bb2a51e7b55824e2344cf94612be0e61cd4fa4caf5
• 180. a866594c74633d54033126c0c351155c6124092e85aa42326858eecc792b0389
• 181. a910028bbf27904707bbf1beb1800610df62de7b00c98664ab4967b5fc14ff4c
• 182. a94edc5b599adc0ea3693b5b7f1df1b7f3d5fc0ff86ec127864d7d18e4bf5be2
• 183. aa6c404ee0e10b80761d547f6c5a8538c5ccfa22d236632c1a8ea3f736ab9259
• 184. ab355450ee2bcea9bf4a6d5e47fcfd35a78c2db4010cc549f66f3b2c81722bbf
• 185. ab64884a11fad661aef4819e8f59ac3f64aba2e6d5831dd0b8d52e86968f4124
• 186. ad7fbb58fc5387ce304f5428a9d0b1f2171215bda6d813341ec254748e3e1ae6
• 187. aeb509272452a167f223ab56e31dc28593d63714f942c0334fc2c3de22dce6b1
• 188. aec0463d72e5f309b6c200acb7f273c5a1b198abe40bd722220c1efee7e4ddd2
• 189. af07a50daa5a5c296682d817d54f07c4572c95b801071400fc607cfcce74ef9b
• 190. b16ba8366423f44511f55fde547fcf96ef90c8a5a8c0965c46f1992bb63aa2b5
• 191. b1fac3814cc9027e9b68b43d04a4fe33083916c165183c1ce731225da12ed7b9
• 192. b2347288a2b172a3e58fc669a3db8597bc702ff17ed6658080703f12b3872611
• 193. b2d2b07a37b9a9ea17cf3bef01ebb91504d4dd7348c7839eaf648138acc9e79d
• 194. b4cb1cee759f02853f5c50ac081ccdfdd40fed742b987a0a1d456e5c077283b0
• 195. b693cd6c08c0133dd4d872ed5317ebfd0bc368b636a095d2b87764d23077b019
• 196. b7497254ec76d5ad83727c8f8c39c2d1c5b10bd3fa11423ae30eaa3917cfa1ab
• 197. b86b783b63aee0fd50344a59065ce2ec64ede20d4a247fd3be2cba3060fed182
• 198. b8f0742b7cc8fe5385c7946e977ff76c172ace729ea42ed6ccef8f29424a271f
• 199. b961bc0c663a46893afd3166a604e7e2639533522d9fec61fdb95eb665e86f5a
• 200. b962b3b2969619658e18660d60684fcb303dc03d9ac1072e44d38f3ccf32f8c3
• 201. b9d3b8bd1d7e5ef2ab76f244cc0ea09764f3c7a21e324f0ef2e894ad247b72d3
• 202. bac6e7e8fa2879ab3fe401ad879c501023a0f495d333820bdc7d6defb6d123e4
• 203. bcbc6943489afc9e2bf941cb6aeb89b4e71ab132fc12526460171fdcecd4549d
• 204. bedaf20cc60e23acb08d5be647d1209a5bb80a00fc3d603ff6e15cbcba14c603
• 205. bf0fbfe87482ba48668813196e000811b6d84ab6a11f621b2b31f73862b89af8
• 206. bfa06e523546934e3f84f6aa2ee0fea6d1f1508331afe47812ea614714e8d5fb
• 207. bfc7c91ae9d31510a66e990c7b4f30c6e3c53bd26775bb50e27224f1582f31e1
• 208. c7622f1b1a78f6be4459812beb9ce802efcae2fe22a3506174b4e1049db56491
• 209. c7797439e4b7403fb4ce79299e6d27dfd5639da33d38092b12e1102eb357109c
• 210. c7d6424598b829281fb75ff4c89ef08312f75536682678edb8c3221db0d437f0
• 211. c99d44af2dd50307ed572dc9f19690c4f39de094a7f15c60ff853dbea46377c5
• 212. cb10dcf16223562788d82ed3f81e551233d9a09849b593c01c1710a3d88e5401
• 213. cb36ba33b3ecd4d3177d786209670c9e6cdf95eb62be54986f0b49ca292714af
• 214. cbf5928cafd0ce423065d29fa90aa39a114a06d431c140c2a2e4b0b724cbb90a
• 215. cde0f841ae62bc4facaa16a853f30b8107bf39cf7d4fce6586ecbb731a00a134
• 216. ce9e6a48c4f3e02564d398751a68721b1c7644fdb985aed9375761c9567770b9
• 217. d052315ce2d4f0310c01142cea33621bca0b90d0c8a4e7eabcd83929da34ed7f
• 218. d0908c19e8d4794fa511439e62acf64100c2f275970470da0cb8d5baca0c4ba8
• 219. d0d7622f76c560a0f7b44ac53f808c322efd6eac7e8607bdd3d54a8941fafb1c
• 220. d1229434fc8104c839ac15977137b3a3a1adeca5b68a4bc9d22c9a9232ff1cc0
• 221. d2dceacdf4ff9854fc37bc0e669106a78c4ab85d7d18d21badb59c1f1e34d663
• 222. d32839413d0d0f67da93929a78dec0132a8f76e03ef991d6c33089dc65e57f85
• 223. d4b0aa6ba8fde6a37f713210de01b09fdda3764164bfd149e6a978f41f3ef91d
• 224. d4fd5e4536d8f5192a4518b0da40bc3a39ccc0dd99eceb0123f995881fe98748
• 225. d5caba9c569e591de2224a98b7e31c9165eb7fb4eaec5be39a63463d0e0b7a65
• 226. d7ea59a35799b1625de066371aac5cae21c20b8a3547d6428e9dd74100107a07
• 227. d828eff1e679f5276589a5de958c4e00722d3349ce08b6a58f184865e75568f1
• 228. d86ddfac614545690d59fecb031b93cb5961b24656e4ee91963a4810ea51044f
• 229. da8ee02ece2ac90727453f4c0ed353c87938b9a81a02575f22ac26f0177b30eb
• 230. dabb4164dbe464d3889fd5da02e6ed9e99d7a248d0acd26a02334948b75895bb
• 231. dcb13665f9471393f36d6739fa235d5b90d25db0bb8208d33af59669859a2bcd
• 232. dd49bb875253771ecf6d65f09e30bf8d7342107ccbeb2509f8e9f6c131865e94
• 233. de3fcd1928d8b6355543157ea2af04e6243bb90bbd5dd581f4b0db74f9a27d4d
• 234. de7e2217d4539b8fabfddb9b541af4ed0b9ff5bdccf715bf0ee3519f83864636
• 235. dee3064b7db0e50f9fe3e51c2cdeef326b746b00db206d020953a0ad091c41ac
• 236. df057a747b460397f8d5914d51ea687aba5a21b93ec20580bd3696ec5f274aa5
• 237. df6dca4f47154c5247f26cda2e5d7d59869822f72127f1785feb12085f3218aa
• 238. e099ab052d6362b76f0ba11e531a2e9d256329eb0904a1581de6c972a854f8e7
• 239. e171fb5cc3acf9296862477c339aa293e649b02f2ad6da9295ae9a04b40d14e3
• 240. e1ef547f838ba5e77f624465773020992c96ff2fd49b3c811ff79603e8b500e6
• 241. e20f0a0d9b5deb10556ad10b7269d0a926866762d6aa3b7d619d27313449ddd0
• 242. e3cc053e67d14e6e22dcfa578ab7cd42aed7259b0ce88f7731d02e36e29b8739
• 243. e580a7fea1965a7d372302beb3adfff956a077f68f1fce70d8e5dfbd16d3a174
• 244. e60661ae318c5501233a3ecaebb4ea853594934fd8f2dde2c63cf8e63dddd432
• 245. e665e91cb0bbda4d9296251bcd4adebe779a326b3f5a6b40b6c06b5c07c5e8e3
• 246. e78ce27c5a18d1b2f76b824f3610f8ff9986aa30c924ded51a8ea0df162b039a
• 247. e8bb8e1a08399fb2b5d4a469dcd54b0e8194cac82123800031f7de8d6ab315af
• 248. ea590d12d5c166f62fa37ce9cb0fcdb190f0fea8a1e0cee55d52b537c235ce05
• 249. edb8f236a2a368a1ef5781937bcfeb0902a4da8daec189fbc2e11c6facdd629c
• 250. edf604b55a775fd4ef3362b8857906997ce16790c8129a30a9061570872e88e3
• 251. ee75ea4a67e5ad29782f6d219f536925834565d9fcedb603d6e92979916a9955
• 252. efa05b94e5962616dea19132f414b375cef2a19466554078618aa96d07409abd
• 253. f07f649872ef008ae54dcb6f612676bb7b68debbd4411904d043f9d99f3b9560
• 254. f087ed64ee8b926175a72c7a379881992c312e4e7da30560adbd7bedff49c7bb
• 255. f0fc5520ff4085784de926bbe78404d30803cf4c91f3f083d416ac740a39b92e
• 256. f1fd07490be10a6186235bca33d9102ea918ff3071a898ceb7570ca0bfd66c80
• 257. f25da320fda8db7bdb53fda4c3c4830fa3a36233c6004fb2734664d915bb52bf
• 258. f35eebbc712ec0258ac578876a13e2f9184f63e6b4b3209e803b4c26a32fdf31
• 259. f37ae3cda4cfc8ce549a9c6d48c8682b3207b186469d90753937323dd35a603e
• 260. f3eedc69ed11c1ed5b72be795be832348597618b2aec71dfcf2d9c98f6722a2f
• 261. f7825be12446fcec207cbe203f1b201f09aa2744769271c395a46c759c24c310
• 262. f7d7d5e59c9e1463f774094a599272164ee1345b3fb3e40fd2a12c16f1f82ade
• 263. f8423cc9361a8ba9c2e551ac366e59180f6d947480d34c8eddab57312bdb2a35
• 264. fafa95c744090eecd9b16dbd503b9e53904218600c86df049e7ff22d4ce2b16a
• 265. fafe9de4309001411d5147cbda8b05511fb330573a650d672d127e5205909f12
• 266. fc7b6bea720bc18945151b01d7f1708c69e6fded74b0ebbe8da0b71f74824569
• 267. fcfa879a7d4156f280279584e5bca4629289b1fd7340f239197212764220b33a
• 268. fdae295cdd305a6c777f3d8c2eb4e76770d95aaf667a64fd2388342a76e396a1
• 269. fe54c17a2d69a8b3c40b66c9f9ad9cdcbf525083665227b2c628ff4f03a1f5cc
• 270. fe8c85f8811c4bb9f14575bac01ec1ef11ebf31ab396a990525debe18bbec5cd

(got the data from here, for some reason that page doesn't work anymore)

I might go through all of these later and try to find any "special" ones. For now though I have to study... : /


p.s. sorry for the huge post. [spoiler] tag not working. how do I spoiler stuff on this forum ?

[macbook-air]

I later manually corrected the issue with another transaction, thus double spend.

For post-mortem analysis, would you mind sharing how that raw transaction was broadcast?  Through bitcoin-Qt/bitcoind I presume?   But was this just connecting to peers or had you actually explicitly connected to a mining pool or well connected node?

The question is that most peers would have rejected it because they should already have known of the original transaction (12814b8ad57ce5654ba69eb26a52ddae1bff42093ca20cef3ad96fe7fd85d195) and not relayed your double spend (762443f6373b7c8b3833d4ad23578fc3099cc29b86d1359d0c0565e3c8614f91).  

With all the nodes reverting to 0.7 and/or starting up with an empty memory pool it is not impossible for your node to have reached one of them with no knowledge of the original transaction and then because your double spend transaction was perfectly valid for that node to relay your transaction got relayed to other new nodes as well.  Eventually then there would be a path that would reach the miner who eventually mined 225466 (the block with the double spend) -- BTC Guild [Edit: Yes, confirmed BTC Guild, per the coinbase].

Code:

$ git diff src/rpcrawtransaction.cpp
diff --git a/src/rpcrawtransaction.cpp b/src/rpcrawtransaction.cpp
index e634ed7..1f045e8 100644
--- a/src/rpcrawtransaction.cpp
+++ b/src/rpcrawtransaction.cpp
@@ -164,8 +164,6 @@ Value listunspent(const Array& params, bool fHelp)
             CBitcoinAddress address(input.get_str());
             if (!address.IsValid())
                 throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, string("Invalid Bitcoin address: ")+input.get_str());
-            if (setAddress.count(address))
-                throw JSONRPCError(RPC_INVALID_PARAMETER, string("Invalid parameter, duplicated address: ")+input.get_str());
            setAddress.insert(address);
         }
     }
@@ -520,9 +518,11 @@ Value sendrawtransaction(const Array& params, bool fHelp)
     {
         // push to local node
         CTxDB txdb("r");
-        if (!tx.AcceptToMemoryPool(txdb))
-            throw JSONRPCError(RPC_DESERIALIZATION_ERROR, "TX rejected");
-
+        if (!tx.AcceptToMemoryPool(txdb)) {
+            SyncWithWallets(tx, NULL, true);
+            RelayMessage(CInv(MSG_TX, hashTx), tx);
+            throw JSONRPCError(RPC_DESERIALIZATION_ERROR, string("TX rejected, relay anyway ")+hashTx.GetHex());
+        }
         SyncWithWallets(tx, NULL, true);
     }
     RelayMessage(CInv(MSG_TX, hashTx), tx);
$ bitcoin createrawtransaction '[{"txid": "e6ee06cec15f8fab2c1aa81037e6ba9b1d7c7c4073a2795e847aa9ca5b84daf1", "vout": 1}]' '{"1J1F3U7gHrCjsEsRimDJ3oYBiV24wA8FuV": 137.35368622}'
0100000001f1da845bcaa97a845e79a273407c7c1d9bbae63710a81a2cab8f5fc1ce06eee60100000000ffffffff01ae17b132030000001976a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac00000000
$ bitcoin signrawtransaction 0100000001f1da845bcaa97a845e79a273407c7c1d9bbae63710a81a2cab8f5fc1ce06eee60100000000ffffffff01ae17b132030000001976a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac00000000 '[{"txid": "e6ee06cec15f8fab2c1aa81037e6ba9b1d7c7c4073a2795e847aa9ca5b84daf1", "vout": 1, "scriptPubKey": "76a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac"}]'
{
    "hex" : "0100000001f1da845bcaa97a845e79a273407c7c1d9bbae63710a81a2cab8f5fc1ce06eee6010000008c493046022100d6e0dec64778c669a4869a73dc057959ac6f8c6fa390e9ceb03bb040d3eb0933022100f3dc55dccbde883e7aadf737a5d8531847fdab1cc2b80a2357ce5dc5378a8485014104054f0d458c4dc3d0a4bb35cd17e624eb76ea99b702561bd0bea0aefe34c6c860fafca4faae957bb009e0ece9cba07cccd30dc008512a9a3021dbcfd2127938fcffffffff01ae17b132030000001976a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac00000000",
    "complete" : true
}
$ bitcoin createrawtransaction '[{"txid": "b53cbbf160f1a5c470dba13672338908c5c9099985f6fdc37d44a5facae957df", "vout": 1}]' '{"1J1F3U7gHrCjsEsRimDJ3oYBiV24wA8FuV": 62.64371378}'
0100000001df57e9cafaa5447dc3fdf6859909c9c50889337236a1db70c4a5f160f1bb3cb50100000000ffffffff01b2b86275010000001976a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac00000000
$ bitcoin signrawtransaction 0100000001df57e9cafaa5447dc3fdf6859909c9c50889337236a1db70c4a5f160f1bb3cb50100000000ffffffff01b2b86275010000001976a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac00000000 '[{"txid": "b53cbbf160f1a5c470dba13672338908c5c9099985f6fdc37d44a5facae957df", "vout": 1, "scriptPubKey": "76a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac"}]'
{
    "hex" : "0100000001f1da845bcaa97a845e79a273407c7c1d9bbae63710a81a2cab8f5fc1ce06eee6010000008c493046022100d6e0dec64778c669a4869a73dc057959ac6f8c6fa390e9ceb03bb040d3eb0933022100f3dc55dccbde883e7aadf737a5d8531847fdab1cc2b80a2357ce5dc5378a8485014104054f0d458c4dc3d0a4bb35cd17e624eb76ea99b702561bd0bea0aefe34c6c860fafca4faae957bb009e0ece9cba07cccd30dc008512a9a3021dbcfd2127938fcffffffff01ae17b132030000001976a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac00000000",
    "complete" : true
}
$ while true; do bitcoin sendrawtransaction 0100000001f1da845bcaa97a845e79a273407c7c1d9bbae63710a81a2cab8f5fc1ce06eee6010000008c493046022100d6e0dec64778c669a4869a73dc057959ac6f8c6fa390e9ceb03bb040d3eb0933022100f3dc55dccbde883e7aadf737a5d8531847fdab1cc2b80a2357ce5dc5378a8485014104054f0d458c4dc3d0a4bb35cd17e624eb76ea99b702561bd0bea0aefe34c6c860fafca4faae957bb009e0ece9cba07cccd30dc008512a9a3021dbcfd2127938fcffffffff01ae17b132030000001976a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac00000000; sleep 10; bitcoin sendrawtransaction 0100000001f1da845bcaa97a845e79a273407c7c1d9bbae63710a81a2cab8f5fc1ce06eee6010000008c493046022100d6e0dec64778c669a4869a73dc057959ac6f8c6fa390e9ceb03bb040d3eb0933022100f3dc55dccbde883e7aadf737a5d8531847fdab1cc2b80a2357ce5dc5378a8485014104054f0d458c4dc3d0a4bb35cd17e624eb76ea99b702561bd0bea0aefe34c6c860fafca4faae957bb009e0ece9cba07cccd30dc008512a9a3021dbcfd2127938fcffffffff01ae17b132030000001976a914ba85dc466140d3ef2eac3f7e42088ec501315a4b88ac00000000; sleep 10; done
15e8729f4632f9542790e12e7e6891ac42ca9d109c38656e7259b58ee9133d2e
22b85b88ac4f18afaab9c06ee70b47df7da23201bd2ad646ca0b202d775aa999
15e8729f4632f9542790e12e7e6891ac42ca9d109c38656e7259b58ee9133d2e
22b85b88ac4f18afaab9c06ee70b47df7da23201bd2ad646ca0b202d775aa999
15e8729f4632f9542790e12e7e6891ac42ca9d109c38656e7259b58ee9133d2e
22b85b88ac4f18afaab9c06ee70b47df7da23201bd2ad646ca0b202d775aa999
......

(I cannot remember whether the last command returns simply txid or "error: {"code":-22,"message":"TX rejected, relay anyway txid"})

No. My fully synced bitcoind refused to execute any commands except getinfo. It simply reports error message: "Warning: Displayed transactions may not be correct! You may need to upgrade, or other nodes may need to upgrade." and exit.

So I had to set up a new bitcoind in Parallels Desktop's Ubuntu virtual machine and this bitcoind was not yet fully synced with the network. And it shouldn't have many connections as it was behind NAT.

[Yurock]

Maybe the problem it that there is one implementation (bitcoind / bitcoin-qt) that decides what is right and what is not for the majority of the network. Bitcoin relies on a state when the vast majority of the network agree on the rules. That may not be the case when a single bug affects a large portion of the network. Under these conditions the whole proof of work mechanism becomes unreliable.

If we had a variety of implementations, a bug in one of them would have much smaller impact. Only users of the buggy client would be required to upgrade. We would not have to take the generally wrong way of downgrading to a buggy version.

[arklan]

Maybe the problem it that there is one implementation (bitcoind / bitcoin-qt) that decides what is right and what is not for the majority of the network. Bitcoin relies on a state when the vast majority of the network agree on the rules. That may not be the case when a single bug affects a large portion of the network. Under these conditions the whole proof of work mechanism becomes unreliable.

If we had a variety of implementations, a bug in one of them would have much smaller impact. Only users of the buggy client would be required to upgrade. We would not have to take the generally wrong way of downgrading to a buggy version.

this is why the discussion of a "bitcoin spec" is ongoing and of worth. check the dev sub forum.

[Bitcoinpro]

Just received a telephone call from OKPAY. The double spend money has been refunded to them. They promise me to send me my 64.xxx BTC within 15 minutes after they receive my refund. Please see if they respect their promise: 1J1F3U7gHrCjsEsRimDJ3oYBiV24wA8FuV This address should receive 64.xxx BTC before 09:30 UTC.

This is not the way to do customer service. If not this incident, I don't know when they start to care me and return me the 64.xxx BTC.

why is ok support using a comma and decimal point in the same post

[Raoul Duke]

why is ok support using a comma and decimal point in the same post

Probably both values copy pasted from previous messages or one of them c/p and the other written by the support person.

[jl2012]

It has been more than a year. I think this is the only instance of successful double spending of a 6-confirmation transaction. Have anything been done to address this problem? Comparing with those fancy applications, this involves the very fundamental principle: transaction irreversibility. If it is not fixed, I am sure it will happen again when we have another unexpected fork and the harm could be much more serious.

[arklan]

It has been more than a year. I think this is the only instance of successful double spending of a 6-confirmation transaction. Have anything been done to address this problem? Comparing with those fancy applications, this involves the very fundamental principle: transaction irreversibility. If it is not fixed, I am sure it will happen again when we have another unexpected fork and the harm could be much more serious.

Because this required a fork, I don't think there's anything that can be done. I mean, the entire system of bitcoin relies on forks not occurring. I'm no Dev though, so don't take my word as law.

[drawingthesun]

It has been more than a year. I think this is the only instance of successful double spending of a 6-confirmation transaction. Have anything been done to address this problem? Comparing with those fancy applications, this involves the very fundamental principle: transaction irreversibility. If it is not fixed, I am sure it will happen again when we have another unexpected fork and the harm could be much more serious.

I wonder how it can be fixed?

Perhaps the client once seeing two similar chains is put into safe mode until an obvious longer chain emerges. Once one chain is perhaps 6 confirm ahead then automatically go out of safe mode and resume using the longest chain.

This solution would require no central control and can be automated by the nodes.

The important aspect would be the detection mechanism and watching two chains until there is a clear winner.

[franky1]

It has been more than a year. I think this is the only instance of successful double spending of a 6-confirmation transaction. Have anything been done to address this problem? Comparing with those fancy applications, this involves the very fundamental principle: transaction irreversibility. If it is not fixed, I am sure it will happen again when we have another unexpected fork and the harm could be much more serious.

technically its not a double spend.. there is only one "receipt" of BTC that got confirmed on this main blockchain, meaning there is not 2 bunches of bitcoins that originated from one single bitcoin source on this main blockchain.

what happened is that businesses did not update the clients to be using the correct blockchain. thus the business does not see the correct transaction and ends out paying FIAT twice.

what needs to be done is to ensure forks dont happen, and in the small chance that a fork does occur, that businesses are not lazy about updating their clients, or double checking transactions before releasing different forms of funds or goods. (which would have prevented malleability issues if businesses had proper double checking methods in place)

there is not one single extra bitcoin on the blockchain that was cloned or not originate from a verified block reward. ill say it again a double spend is where 1 btc can be used twice. no coin has been used twice. all that has happened is that businesses have not had adequate checking processes in place. the bitcoin protocol and the blockchain does not show more confirmed coins then what has been released through blockchain rewards

[zolace]

All time UTC+08:00:

08:08 – Well before I knew what later have happened, I deposited $10000-worth Bitcoins to BTC-e over OKPAY's Bitcoin payment, I paid OKPAY address 12z2n8YCJw1BEsJhhQPLCTuLqwH341nKnE 211.9093 BTC and 0.0005 BTC as transaction fee.
09:30 – The transaction was included in version 0.8's fork, block 225446
10:08 – Deposit completed, $9800 credited to my BTC-e account
12:53 – After some study, I recognized, the transaction, though included in version 0.8's fork, was never confirmed by the pre-0.8 fork, so I decided to make two double spend transactions on two of the vins of the OKPAY transaction, and broadcasted them with the raw transaction API, 0.001 BTC transaction fee included in each transaction.
13:01 – The double spend transaction was included in pre-0.8 fork block 225446

You should know what happens next...

I bet merchants would think twice before they decide to accept Bitcoins after the incident.

well this is why there should be inventory watchers, and this is why there has to be someone to over look these transactions as this can be avoided.   

[RodeoX]

There has never been a double spend.

[Peter R]

It has been more than a year. I think this is the only instance of successful double spending of a 6-confirmation transaction. Have anything been done to address this problem? Comparing with those fancy applications, this involves the very fundamental principle: transaction irreversibility. If it is not fixed, I am sure it will happen again when we have another unexpected fork and the harm could be much more serious.

Like others have said, the same coin has never been spent twice in the history of bitcoin; when we say "double spend," we mean that someone thought they got paid but later found out that they didn't.  

There are many ways one can protect himself against such events.  In the case described in the OP, the event took place during a planned hard fork of the protocol.  If we feel it necessary to fork the code again (and we likely will), I think it would be wise for exchanges to require a full day of confirmations for large deposits.  Once it is clear that the fork was handled properly, services can reduce the confirmation requirements back to normal.  

Non-planned forks (large orphans) are rare. It is rarer still that the funds confirmed in the now-orphaned chain wouldn't also be confirmed in the new longest chain (it would strongly imply malicious intent, careful planning, and a large financial investment).  But one should still dynamically track confirmed coins and in the very very unlikely event that coins are not confirmed in the new chain, the service should immediately freeze the accounts in question.    

There is also nothing special about the number "6": personally, if I ran a large exchange I would likely require less than 6 confirmations for most deposits, but more than 6 for, say, 10 000 BTC.  

[00ph8al]

There has never been a double spend.

It seems like many have to believe this for their world to continue spinning but it is patently ludicrous to put your faith in a negative statement that by definition cannot be proven.

There HAVE been double spends and there will continue to be double spends.

If we truly want this experiment to survive we must identify these edge cases and provide solutions instead of burying our heads in the sand afraid of the bad publicity.

[jl2012]

ok, if you guys think it is not "double spend", let's call it "reversed transaction". That's not the main point here.

what happened is that businesses did not update the clients

Actually the opposite is true. OKPAY got cheated because they were using the latest client (0.. If they did not update, they would never see the original transaction confirmed.

[RodeoX]

There has never been a double spend.

It seems like many have to believe this for their world to continue spinning but it is patently ludicrous to put your faith in a negative statement that by definition cannot be proven.

There HAVE been double spends and there will continue to be double spends.

If we truly want this experiment to survive we must identify these edge cases and provide solutions instead of burying our heads in the sand afraid of the bad publicity.

Maybe your right? Please point me to a double spend.

[Syke]

Because this required a fork, I don't think there's anything that can be done. I mean, the entire system of bitcoin relies on forks not occurring. I'm no Dev though, so don't take my word as law.

Good thing you added that last line, because you are dead wrong. Forks are expected. They will always happen. That is a core feature in how Bitcoin is designed.

[Joshuar]

Correct me if I'm wrong, but forks are only possible with a certain amount of hash power, and to gain enough hash to fork the Bitcoin chain would be very expensive and thus impractical.

[skooter]

Correct me if I'm wrong, but forks are only possible with a certain amount of hash power, and to gain enough hash to fork the Bitcoin chain would be very expensive and thus impractical.

yeah you're wrong.

Forks happen anytime 2 miners solve a block referencing the same previous block.

[b¡tco¡n]

Correct me if I'm wrong, but forks are only possible with a certain amount of hash power, and to gain enough hash to fork the Bitcoin chain would be very expensive and thus impractical.

yeah you're wrong.

Forks happen anytime 2 miners solve a block referencing the same previous block.

But usually one of the forks gets chosen by consensus right? And after 6 of so confirmations it is extremely unlikely to change.

When this thread was opened there was a bug (or bug fixed) with the latest version of the software which means that the new version of the software chose a different branch to the old, which is a very different and horrible problem. IIRC the developers told people to downgrade to the old version and the associated fork became the chosen fork, and thus losing transactions made in the unchosen fork.

Hammer me if this isn't 100% accurate but I think it is the gist.

Usually forks are OK and get resolved. This thread is about a different 'hard' fork cause by software issues.

[Meuh6879]

How serious is this please. Double spends  I thought were impossible .

it's impossible.
when you wait the 1 confirmation (1 block) ... anyway. 

[Maged]

It has been more than a year. I think this is the only instance of successful double spending of a 6-confirmation transaction. Have anything been done to address this problem?

Yes, the failures in the automated alert systems have been addressed, so services should be able to disable themselves when a long fork is detected:
https://github.com/bitcoin/bitcoin/pull/2658

[Bit_Happy]

Your .gif was not accurate.
2013 turned out to be a great year for BTC. 

[00ph8al]

There has never been a double spend.

It seems like many have to believe this for their world to continue spinning but it is patently ludicrous to put your faith in a negative statement that by definition cannot be proven.

There HAVE been double spends and there will continue to be double spends.

If we truly want this experiment to survive we must identify these edge cases and provide solutions instead of burying our heads in the sand afraid of the bad publicity.

Maybe your right? Please point me to a double spend.

What is your definition of double spend?

Are you including operator error and self-inflicted (ie. allowing 0-confirmations)?  The network (and access) isn't always fast enough for every use case either.

[RodeoX]

What is your definition of double spend?

Are you including operator error and self-inflicted (ie. allowing 0-confirmations)?  The network (and access) isn't always fast enough for every use case either.

I think of a double spend as a spend with confirmation followed by another spend of the same balance. Without confirmation it seems to me that the protocol is behaving as designed and, as you mention, it's an operator error. I also don't think that it counts if it happened on the test net.
A true double spend in the wild would spook me. If a method were found to do that, it would be the most serious threat to bitcoin so far.

[Stephen Gornick]

I think of a double spend as a spend with confirmation followed by another spend of the same balance.

Which is what happened in March, 2013.  If you were using the v0.8 client there was a transaction that had more than six confirmations but later the fork which pre-v0.8 clients were mining surpassed the one the v0.8 were using and that transaction became invalid.  What happened was a user noticed the fork, created a new transaction that spent the same funds, used a custom client that repeatedly broadcast the double spend attempt and eventually that new transaction got included in the pre-v0.8 blockchain fork.

So yes, a double spend of a transaction that had six or more confirmations on what was at the time the longest chain is something that has occurred.

The gist of the problem was that miners starting up with the v0.7 client had an empty memory pool and thus no knowledge of the transactions that had already confirmed in the blockchain fork mined by the v0.8 clients.  So then it simply is a race -- whichever transaction (of the two that were spending the same coin) that reached those miners first is the one that was being hashed.   For that specific weakness, there are solutions that had been discussed but not implemented as far as I know.  For instance, when the miner is shut down the unconfirmed transactions in the memory pool could be written to the filesystem and then those transactions get loaded first when the miner starts back up.   That would likely have made the result of that March 2013 fork be that no double spends had occurred.