Bitcoin Forum
May 14, 2024, 04:02:57 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: 1 2 3 [All]
  Print  
Author Topic: Security update: duplicate transaction vulnerability fix  (Read 14504 times)
Pieter Wuille (OP)
Legendary
*
Offline Offline

Activity: 1072
Merit: 1174


View Profile WWW
March 07, 2012, 03:53:52 PM
Last edit: March 15, 2012, 04:37:59 PM by Pieter Wuille
 #1

UPDATE: 0.5.3 was released with BIP30 support

Hello all,

as some of you already know, a vulnerability was found in the way bitcoin currently handles duplicate transactions. Even though we were able to demonstrate a potential attack on testnet, there is no cause for alarm because such an attack requires significant hashing power, is rather complex, and even if carried out does not allow financial gain for an attacker. Still, it is a vulnerability that can cause problems and we want to fix it as soon as possible.

But before we roll out the fix we wanted to inform you, the community, exactly what the problem is and how we are going to fix it. Essentially we want everyone to be on the same page to make the fix roll-out as smooth and uneventful as possible.

For most problems, this is not required, as we can just release an updated client. However, the nature of the problem dictates a fix that will introduce a new stricter block-validity rule. This means we have to actually change the rules by which blocks are considered to be valid or invalid. This in and of itself poses a risk, because an attacker violating this new rule while only a minority has upgraded their clients could cause a block chain fork. This is a bad thing because it would allow any transaction output that existed before the fork to be spent twice: once in each branch of the fork. However, if a majority enforces the stricter rules, we are sure that any fork will be resolved quickly and not be permanent. This is why we need your support.

The fix, described in BIP 30, is noncontroversial. There was a long discussion on the mailing list, and several developers have expressed their support. A majority of mining pools (in terms of hashing power) have also confirmed they will be updating their nodes. BIP 30 is backward-compatible: only what was previously considered a valid block can become invalid and not the other way around. This means that old software will continue to accept blocks "mined" by clients following the new rule, and keep building upon them.

A bit more technical, the actual bug is this: the bitcoin software was written with the assumption that it is impossible to create a transaction with a hash that is identical to that of a previous transaction. This is however not entirely true. It requires buggy software, or malicious intent, but one can create a coinbase transaction that is identical to a previous coinbase, implying it has the same hash. Furthermore, by recreating transactions that use these duplicated coinbase(s), those can be duplicated as well. Now here is how the bitcoin software currently deals with this: it does not check whether that previous hash already exists but simply overwrites it in its transaction index database. Even worse, when a block that contained such a duplicate is reverted (during a reorganisation), the index entry is deleted entirely. If the original transaction was not yet spent, it has now become unspendable.

The way to fix this is simple: simply disallow blocks to contain such duplicated transactions. In order not to make pruning impossible in the future, one exception is added: if the original transaction was already completely spent before a block, it is (for now) still allowed to contain a duplicate. Without this exception, every full node in the network would be required to keep an index of all transactions ever. This was implemented, the change is merged in git master (so it will be included in the 0.6.0 release) as well as in several backports. It was tested by roconnor (who demonstrated the original attack on testnet) to prevent his attack, and it was tested to still accept the current longest block chain on the real network.


To conclude: We want to fix a newly found vulnerability, we need majority of miners to support the fix in order to prevent a potential permanent fork, the fix was thoroughly discussed on the mailing list, the developers support it and it is noncontroversial. With majority support, if all goes well, this change will be activated on march 15th 2012, 0:00 UTC.

PS: thanks to Hazek for proofreading

I do Bitcoin stuff.
1715659377
Hero Member
*
Offline Offline

Posts: 1715659377

View Profile Personal Message (Offline)

Ignore
1715659377
Reply with quote  #2

1715659377
Report to moderator
1715659377
Hero Member
*
Offline Offline

Posts: 1715659377

View Profile Personal Message (Offline)

Ignore
1715659377
Reply with quote  #2

1715659377
Report to moderator
The trust scores you see are subjective; they will change depending on who you have in your trust list.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1715659377
Hero Member
*
Offline Offline

Posts: 1715659377

View Profile Personal Message (Offline)

Ignore
1715659377
Reply with quote  #2

1715659377
Report to moderator
1715659377
Hero Member
*
Offline Offline

Posts: 1715659377

View Profile Personal Message (Offline)

Ignore
1715659377
Reply with quote  #2

1715659377
Report to moderator
Pieter Wuille (OP)
Legendary
*
Offline Offline

Activity: 1072
Merit: 1174


View Profile WWW
March 07, 2012, 03:54:54 PM
Last edit: March 15, 2012, 01:23:05 PM by Pieter Wuille
 #2

These pools have confirmed that they have deployed BIP 30 updates:
* deepbit (3400 GH/s)
* btcguild (1300 GH/s)
* mining.bitcoin.cz (1400 GH/s)
* eclipsemc (520 GH/s)
* eligius (290 GH/s)
* ozcoin (600 GH/s)
* bitclockers (175 GH/s)
* bitlc (170 GH/s)
* btcmp.com (120 GH/s)
* btcmine (620 GH/s)

(total over 8 TH/s)

I do Bitcoin stuff.
Cryptoman
Hero Member
*****
Offline Offline

Activity: 726
Merit: 500



View Profile
March 07, 2012, 05:13:12 PM
 #3

To conclude: We want to fix a newly found vulnerability, we need majority of miners to support the fix in order to prevent a potential permanent fork, the fix was thoroughly discussed on the mailing list, the developers support it and it is noncontroversial. With majority support, if all goes well, this change will be activated on march 15th 2012, 0:00 UTC.

So the strategy is to post updated code on Sourceforge before March 15 and have miners switch at 0:00 UTC?  I assume this version will not include any form of multisig, in order to avoid controversy and resistance to immediate adoption.  

"A small body of determined spirits fired by an unquenchable faith in their mission can alter the course of history." --Gandhi
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1025



View Profile
March 07, 2012, 05:15:42 PM
 #4

Do we have an ETA on the patch getting merged and included in rc3 or whatever?

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
the founder
Sr. Member
****
Offline Offline

Activity: 448
Merit: 251


Bitcoin


View Profile WWW
March 07, 2012, 05:16:56 PM
 #5

If anything will ever go wrong with bitcoin... it'll be now....  the whole idea of a possible fork is somewhat scary.


Bitcoin RSS App / Bitcoin Android App / Bitcoin Webapp http://www.ounce.me  Say thank you here:  1HByHZQ44LUCxxpnqtXDuJVmrSdrGK6Q2f
Hawkix
Hero Member
*****
Offline Offline

Activity: 531
Merit: 505



View Profile WWW
March 07, 2012, 05:23:37 PM
 #6

A technical question - why the patch does not simply discard the block with the duplicate hash, regardless of its contents?

Donations: 1Hawkix7GHym6SM98ii5vSHHShA3FUgpV6
http://btcportal.net/ - All about Bitcoin - coming soon!
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1025



View Profile
March 07, 2012, 05:23:56 PM
 #7

If anything will ever go wrong with bitcoin... it'll be now....  the whole idea of a possible fork is somewhat scary.

Very unlikely.  Someone would need to plan the attack very carefully, have huge amounts of hashing power hiding offline, and gain essentially nothing from it.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
notme
Legendary
*
Offline Offline

Activity: 1904
Merit: 1002


View Profile
March 07, 2012, 05:32:00 PM
 #8

If anything will ever go wrong with bitcoin... it'll be now....  the whole idea of a possible fork is somewhat scary.

Very unlikely.  Someone would need to plan the attack very carefully, have huge amounts of hashing power hiding offline, and gain essentially nothing from it.

+1

https://www.bitcoin.org/bitcoin.pdf
While no idea is perfect, some ideas are useful.
piuk
Hero Member
*****
Offline Offline

Activity: 910
Merit: 1005



View Profile WWW
March 07, 2012, 05:43:38 PM
 #9

Very unlikely.  Someone would need to plan the attack very carefully, have huge amounts of hashing power hiding offline, and gain essentially nothing from it.

Not really, during this protocol change they only need to mine one valid block.

But as along as the majority of hashing power is supporting BIP30 the difficulty on the forked chain would be left so high that it would probably die fairly quickly and would not be able to produce the 6 valid blocks needed for transaction reversal.

kronosvl
Full Member
***
Offline Offline

Activity: 134
Merit: 100


View Profile
March 07, 2012, 05:44:11 PM
 #10

I'm not familiar with how the blockchain is stored in nodes so I have one question.
Can this replacement of a transaction with another one with identical hash break a trace if someone is following the coins? if yes then whoever wants to do this will have to implement a different storing mechanism to allow multiple transactions with identical hash to coexists. Is this true or I'm too noob to understand?

Donations are accepted @: 19Uk8zVhdgfrRo5Z6wH9yghWxZUtdiNtX9
OTC: http://bitcoin-otc.com/viewgpg.php?nick=kronosvl
notme
Legendary
*
Offline Offline

Activity: 1904
Merit: 1002


View Profile
March 07, 2012, 06:00:49 PM
 #11

Very unlikely.  Someone would need to plan the attack very carefully, have huge amounts of hashing power hiding offline, and gain essentially nothing from it.

Not really, during this protocol change they only need to mine one valid block.

But as along as the majority of hashing power is supporting BIP30 the difficulty on the forked chain would be left so high that it would probably die fairly quickly and would not be able to produce the 6 valid blocks needed for transaction reversal.

One block forks happen all the time.   Meh.

https://www.bitcoin.org/bitcoin.pdf
While no idea is perfect, some ideas are useful.
Daily Anarchist
Hero Member
*****
Offline Offline

Activity: 614
Merit: 500



View Profile WWW
March 07, 2012, 06:46:43 PM
 #12

All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.

It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.

Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:

Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.

I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?

Discover anarcho-capitalism today!
hazek
Legendary
*
Offline Offline

Activity: 1078
Merit: 1002


View Profile
March 07, 2012, 06:55:22 PM
 #13

All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.

It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.

Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:

Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.

I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?

Simple. As long as the process is completely transparent and open there's really nothing anyone could "sneak" past everyone else in order to cause damage. And if you checked the OP, it pretty much doesn't get any more transparent then how they're going about with this fix. I mean you can even read the mailing list how the devs formulated the fix..

My personality type: INTJ - please forgive my weaknesses (Not naturally in tune with others feelings; may be insensitive at times, tend to respond to conflict with logic and reason, tend to believe I'm always right)

If however you enjoyed my post: 15j781DjuJeVsZgYbDVt2NZsGrWKRWFHpp
casascius
Mike Caldwell
VIP
Legendary
*
Offline Offline

Activity: 1386
Merit: 1136


The Casascius 1oz 10BTC Silver Round (w/ Gold B)


View Profile WWW
March 07, 2012, 06:57:19 PM
 #14

My understanding is that there already exists such duplicate coinbase transactions in the production block chain.  (I first learned of this issue when someone came on the forums and pointed out how they "whoops" sent 50 BTC to never never land by mining a duplicate coinbase).  If I find that reference, I'll come back, but I'd like to think this was around block 90000-100000 or so.

EDIT: to include the specific block numbers: 91842 has a duplicate coinbase of 91812.
http://blockexplorer.com/b/91812
http://blockexplorer.com/b/91842

Can the proposal be amended to explain exactly what will happen to these transactions?  Or at the very least acknowledge such transactions exist (assuming I'm not mistaken)?  I assume they must remain valid (so as to not invalidate the entire block chain coming after them).  Anyone reimplementing the Bitcoin protocol will have to account for the fact that these transactions are valid and later ones just like them are not, and there will need to be very clear rules to determine what constitutes a valid transaction.

Companies claiming they got hacked and lost your coins sounds like fraud so perfect it could be called fashionable.  I never believe them.  If I ever experience the misfortune of a real intrusion, I declare I have been honest about the way I have managed the keys in Casascius Coins.  I maintain no ability to recover or reproduce the keys, not even under limitless duress or total intrusion.  Remember that trusting strangers with your coins without any recourse is, as a matter of principle, not a best practice.  Don't keep coins online. Use paper or hardware wallets instead.
Daily Anarchist
Hero Member
*****
Offline Offline

Activity: 614
Merit: 500



View Profile WWW
March 07, 2012, 06:57:34 PM
 #15

All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.

It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.

Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:

Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.

I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?

Simple. As long as the process is completely transparent and open there's really nothing anyone could "sneak" past everyone else in order to cause damage. And if you checked the OP, it pretty much doesn't get any more transparent then how they're going about with this fix. I mean you can even read the mailing list how the devs formulated the fix..

Two more questions then:

Is 8 days really enough time to vet a potentially nefarious plan?

What if a few people DO see something nefarious in it, will they be listened to, or ignored?

Discover anarcho-capitalism today!
theymos
Administrator
Legendary
*
Offline Offline

Activity: 5194
Merit: 12985


View Profile
March 07, 2012, 07:00:59 PM
 #16

I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?

Anyone can read the code and verify that it really does what it claims. It's like 10 lines of code.

Can the proposal be amended to explain exactly what will happen to these transactions?

They won't be touched. The new rules only apply to blocks with a time after March 15.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
hazek
Legendary
*
Offline Offline

Activity: 1078
Merit: 1002


View Profile
March 07, 2012, 07:10:12 PM
 #17

Is 8 days really enough time to vet a potentially nefarious plan?

It's not just 8 days. The fix was developed for some time now and the process was open so it already was under scrutiny. But even if it was just 8 days it's still enough.

What if a few people DO see something nefarious in it, will they be listened to, or ignored?

No one is forced to download the updated client. That's the fialsafe in Bitcoin in general.

My personality type: INTJ - please forgive my weaknesses (Not naturally in tune with others feelings; may be insensitive at times, tend to respond to conflict with logic and reason, tend to believe I'm always right)

If however you enjoyed my post: 15j781DjuJeVsZgYbDVt2NZsGrWKRWFHpp
Daily Anarchist
Hero Member
*****
Offline Offline

Activity: 614
Merit: 500



View Profile WWW
March 07, 2012, 07:14:02 PM
 #18

Is 8 days really enough time to vet a potentially nefarious plan?

It's not just 8 days. The fix was developed for some time now and the process was open so it already was under scrutiny. But even if it was just 8 days it's still enough.

What if a few people DO see something nefarious in it, will they be listened to, or ignored?

No one is forced to download the updated client. That's the fialsafe in Bitcoin in general.

This is true, no one is forced. But people have a tendency to lose their way.

I just hope Bitcoin doesn't lose its way over time.


Discover anarcho-capitalism today!
casascius
Mike Caldwell
VIP
Legendary
*
Offline Offline

Activity: 1386
Merit: 1136


The Casascius 1oz 10BTC Silver Round (w/ Gold B)


View Profile WWW
March 07, 2012, 07:21:48 PM
 #19

The bitcoins in block 91812 coinbase have not been spent.  My understanding is the current client will only allow them to be spent once.  If the block chain spec is being amended to say that duplicate coinbases are valid prior to March 15 2012, it should also explicitly address the bitcoins in the duplicate block 91842 and formally declare them as a valid block containing an invalid/unspendable transaction, otherwise there is an ambiguity that may be implemented by future clients/utilities the wrong way.

Companies claiming they got hacked and lost your coins sounds like fraud so perfect it could be called fashionable.  I never believe them.  If I ever experience the misfortune of a real intrusion, I declare I have been honest about the way I have managed the keys in Casascius Coins.  I maintain no ability to recover or reproduce the keys, not even under limitless duress or total intrusion.  Remember that trusting strangers with your coins without any recourse is, as a matter of principle, not a best practice.  Don't keep coins online. Use paper or hardware wallets instead.
istar
Hero Member
*****
Offline Offline

Activity: 523
Merit: 500


View Profile
March 07, 2012, 08:05:05 PM
 #20

All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.

It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.

Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:

Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.

I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?

Simple. As long as the process is completely transparent and open there's really nothing anyone could "sneak" past everyone else in order to cause damage. And if you checked the OP, it pretty much doesn't get any more transparent then how they're going about with this fix. I mean you can even read the mailing list how the devs formulated the fix..

Two more questions then:

Is 8 days really enough time to vet a potentially nefarious plan?

What if a few people DO see something nefarious in it, will they be listened to, or ignored?

Also (all?) of the coders are known by real name. If they f*ckup bitcoin, they will be known by everyone as the person who screwed things up.
That would be the same as destroying many peoples money, and time invested and many companies and that is not like something you would like to do, ofcourse there is allways the possibility that they are threatened by jail or whatever unless they do this. But in this case its far from very likely.

Bitcoins - Because we should not pay to use our money
Daily Anarchist
Hero Member
*****
Offline Offline

Activity: 614
Merit: 500



View Profile WWW
March 07, 2012, 08:28:22 PM
 #21

All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.

It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.

Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:

Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.

I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?

Simple. As long as the process is completely transparent and open there's really nothing anyone could "sneak" past everyone else in order to cause damage. And if you checked the OP, it pretty much doesn't get any more transparent then how they're going about with this fix. I mean you can even read the mailing list how the devs formulated the fix..

Two more questions then:

Is 8 days really enough time to vet a potentially nefarious plan?

What if a few people DO see something nefarious in it, will they be listened to, or ignored?

Also (all?) of the coders are known by real name. If they f*ckup bitcoin, they will be known by everyone as the person who screwed things up.
That would be the same as destroying many peoples money, and time invested and many companies and that is not like something you would like to do, ofcourse there is allways the possibility that they are threatened by jail or whatever unless they do this. But in this case its far from very likely.


I don't see very much as unlikely when you're talking about challenging the global banking cartel.

Discover anarcho-capitalism today!
casascius
Mike Caldwell
VIP
Legendary
*
Offline Offline

Activity: 1386
Merit: 1136


The Casascius 1oz 10BTC Silver Round (w/ Gold B)


View Profile WWW
March 07, 2012, 11:55:12 PM
 #22

A technical question - why the patch does not simply discard the block with the duplicate hash, regardless of its contents?

My guess is because of duplicate transactions already on the block chain.  Discarding the old blocks would discard half the block chain and half of the bitcoins in existence with it.

Companies claiming they got hacked and lost your coins sounds like fraud so perfect it could be called fashionable.  I never believe them.  If I ever experience the misfortune of a real intrusion, I declare I have been honest about the way I have managed the keys in Casascius Coins.  I maintain no ability to recover or reproduce the keys, not even under limitless duress or total intrusion.  Remember that trusting strangers with your coins without any recourse is, as a matter of principle, not a best practice.  Don't keep coins online. Use paper or hardware wallets instead.
Atheros
Sr. Member
****
Offline Offline

Activity: 249
Merit: 251



View Profile WWW
March 08, 2012, 01:39:55 AM
 #23

For all the paranoid people, here is the patch for everyone to read. The red lines came out and the green lines were added.

https://github.com/sipa/bitcoin/commit/a206b0ea12eb4606b93323268fc81a4f1f952531

BM-GteJMPqvHRUdUHHa1u7dtYnfDaH5ogeY
Bitmessage.org - Decentralized, trustless, encrypted, authenticated messaging protocol and client.
Pieter Wuille (OP)
Legendary
*
Offline Offline

Activity: 1072
Merit: 1174


View Profile WWW
March 08, 2012, 02:03:03 AM
Last edit: March 08, 2012, 02:46:25 AM by Pieter Wuille
 #24

A technical question - why the patch does not simply discard the block with the duplicate hash, regardless of its contents?

That is basically what happens: a block with a duplicate transaction will become invalid, and thus will be ignored.

EDIT: obviously, this does not happen to blocks with existing duplicate transactions, as that would require invalidating the entire chain since then.

@casascius: you are right that technically a specification is needed for what to do with duplicate transactions in blocks before the switchover point. However, there is no formal specification of bitcoin's network rules currently either (only the code...), and the BIP only specifies the changes to the current rules. The current rules are: the transaction index is overwritten for a duplicate transaction, and a reorganisation removes the relevant transaction index entries.

I do Bitcoin stuff.
Hawkix
Hero Member
*****
Offline Offline

Activity: 531
Merit: 505



View Profile WWW
March 08, 2012, 06:03:09 AM
 #25

A technical question - why the patch does not simply discard the block with the duplicate hash, regardless of its contents?

That is basically what happens: a block with a duplicate transaction will become invalid, and thus will be ignored.

EDIT: obviously, this does not happen to blocks with existing duplicate transactions, as that would require invalidating the entire chain since then.

I understand the "IF NEWER THAN SOMEDATE" in code, this is to prevent invalidating the entire chain since when first duplicate transaction appeared.

But, with this date switch implemented, why to check for spent/unspent transactions, instead of simply comparing the hash against all previous (just for blocks after the switch)?

Donations: 1Hawkix7GHym6SM98ii5vSHHShA3FUgpV6
http://btcportal.net/ - All about Bitcoin - coming soon!
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1025



View Profile
March 08, 2012, 06:29:02 AM
 #26

A technical question - why the patch does not simply discard the block with the duplicate hash, regardless of its contents?

That is basically what happens: a block with a duplicate transaction will become invalid, and thus will be ignored.

EDIT: obviously, this does not happen to blocks with existing duplicate transactions, as that would require invalidating the entire chain since then.

I understand the "IF NEWER THAN SOMEDATE" in code, this is to prevent invalidating the entire chain since when first duplicate transaction appeared.

But, with this date switch implemented, why to check for spent/unspent transactions, instead of simply comparing the hash against all previous (just for blocks after the switch)?

It has been known for quite a while that a new coinbase with the same hash as an old coinbase was possible and not handled at all.  But since they are referenced by their (identical) hashes, there was no way to specify which one you meant to redeem, so redeeming one would redeem all of them.  This is not in any way a security problem, and can only be used to lose money.  There are faster and easier ways to lose money.

What was only recently discovered was that if you spend a coinbase, and then generate another identical one, you can then spend the second one too.  Spending a transaction doesn't mark that hash invalid for all time, it just removes it from the list of unspent transactions.  Another one can show up and be accepted.  This is, again, not a security problem, and doesn't, by itself, make it possible for people to lose coins unintentionally.

The list of all used transactions isn't readily available, and once pruning shows up, it might not even exist at all.  So, it only makes sense to compare the new coinbase to the list of transaction hashes that are unspent at the time, which aren't subject to pruning.  This also preserves the old behavior of being able to generate and spend multiple identical coinbases, so long as they are spent before the next one shows up.  And finally, it gives us a really short and simple patch, that anyone can read and understand without having to worry that a new side effect has snuck in.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
Hawkix
Hero Member
*****
Offline Offline

Activity: 531
Merit: 505



View Profile WWW
March 08, 2012, 06:43:56 AM
 #27

Nice explained. Thanks!

Donations: 1Hawkix7GHym6SM98ii5vSHHShA3FUgpV6
http://btcportal.net/ - All about Bitcoin - coming soon!
eleuthria
Legendary
*
Offline Offline

Activity: 1750
Merit: 1007



View Profile
March 08, 2012, 02:52:13 PM
 #28

This thread really needs to be more visible:  Perhaps a giant bold red link at the top of the forum?

BTC Guild will be pushing the patch tomorrow, and my understanding is both Slush & Deepbit will be shortly as well, which means the chain will have majority support.  That doesn't solve the problem of the other 30-40% of bitcoin miners that may not even know about the issue due to lack of visibility.

RIP BTC Guild, April 2011 - June 2015
Pieter Wuille (OP)
Legendary
*
Offline Offline

Activity: 1072
Merit: 1174


View Profile WWW
March 08, 2012, 03:19:18 PM
 #29

BTC Guild will be pushing the patch tomorrow, and my understanding is both Slush & Deepbit will be shortly as well, which means the chain will have majority support.  That doesn't solve the problem of the other 30-40% of bitcoin miners that may not even know about the issue due to lack of visibility.

I have updated the second post with a list of pools from whom I have received confirmation they will update. I will keep that list up-to-date as I receive more confirmations.

I do Bitcoin stuff.
hazek
Legendary
*
Offline Offline

Activity: 1078
Merit: 1002


View Profile
March 08, 2012, 03:35:47 PM
 #30

Maybe the person who owns http://bitcoinwatch.com/ could post this thread in the "Latest news" section of their website?

My personality type: INTJ - please forgive my weaknesses (Not naturally in tune with others feelings; may be insensitive at times, tend to respond to conflict with logic and reason, tend to believe I'm always right)

If however you enjoyed my post: 15j781DjuJeVsZgYbDVt2NZsGrWKRWFHpp
cypherdoc
Legendary
*
Offline Offline

Activity: 1764
Merit: 1002



View Profile
March 08, 2012, 03:37:10 PM
 #31

IMO, you haven't made clear whether or not ALL users, or just miners, NEED to upgrade to 0.6 to facilitate/support this transition!

edit: by "support" you mean upgrade to 0.6?
Pieter Wuille (OP)
Legendary
*
Offline Offline

Activity: 1072
Merit: 1174


View Profile WWW
March 08, 2012, 03:52:42 PM
 #32

IMO, you haven't made clear whether or not all users NEED to upgrade to 0.6 to facilitate/support this transition!

edit: by "support" you mean upgrade to 0.6?

Support was meant in a general sense: such a network rule change can only happen when the community agrees to it.

If you are a miner or pool operator: yes, please upgrade quickly, as the higher the percentage enforcing the new rules, the less chance for (temporary and short) block chain splits one could try to create.

Apart from that, if all goes well, eventually everyone will upgrade, but as a normal user, there is relatively little risk in keeping an old version. The more people running a version that enforces the new rule, the smaller the effects of an attacker can be. 0.6.0 will still take some time to be released, but there will be a bugfix-only 0.5.3 soon.

I do Bitcoin stuff.
cypherdoc
Legendary
*
Offline Offline

Activity: 1764
Merit: 1002



View Profile
March 08, 2012, 03:59:28 PM
 #33

IMO, you haven't made clear whether or not all users NEED to upgrade to 0.6 to facilitate/support this transition!

edit: by "support" you mean upgrade to 0.6?

Support was meant in a general sense: such a network rule change can only happen when the community agrees to it.

If you are a miner or pool operator: yes, please upgrade quickly, as the higher the percentage enforcing the new rules, the less chance for (temporary and short) block chain splits one could try to create.

Apart from that, if all goes well, eventually everyone will upgrade, but as a normal user, there is relatively little risk in keeping an old version. The more people running a version that enforces the new rule, the smaller the effects of an attacker can be. 0.6.0 will still take some time to be released, but there will be a bugfix-only 0.5.3 soon.


thats a little better.  let me repeat and if i'm wrong please correct:

Anyone who mines needs to upgrade to 0.5.3 as soon as it comes out to minimize the chances of a blockchain fork.

i'm not trolling; seriously it needs to be spelled out.
Hawkix
Hero Member
*****
Offline Offline

Activity: 531
Merit: 505



View Profile WWW
March 08, 2012, 04:21:22 PM
 #34

Is there an official Bitcoin.org Twitter account? This would be great place to post notifications, warning, new version release, and so on.

Donations: 1Hawkix7GHym6SM98ii5vSHHShA3FUgpV6
http://btcportal.net/ - All about Bitcoin - coming soon!
finway
Hero Member
*****
Offline Offline

Activity: 714
Merit: 500


View Profile
March 08, 2012, 04:47:07 PM
 #35

Glad to see you guys killing this annoying bug.

Wandering Albatross
Member
**
Offline Offline

Activity: 70
Merit: 10



View Profile
March 08, 2012, 05:29:42 PM
 #36

Quote from: Daily Anarchist
All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.

It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.

Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:

Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.

I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?

You might as well throw out the label  "conspiracy theory".  History, long past, and recent, has shown that discerning minds often reveal the truths behind events. So don't let lesser minds diminish accurate perception.
What inspired BTC? Bored geeks?  Current world events?

Recall the debian ssl "issue". It can happen here too. What did the spooks talk about with Andresen when he went to visit them?
Oh and have a look at github notification about recent security breach.  Isn't bitcoin code stored there?
The question is not if but when. The spooks always have a control point. They don't rest until they do and even then they hedge because that's the game.

BTC: 1JgPAC8RVeh7RXqzmeL8xt3fvYahRXL3fP
Wandering Albatross
Member
**
Offline Offline

Activity: 70
Merit: 10



View Profile
March 08, 2012, 05:40:10 PM
 #37

Did you all choose sf.net for mailing list because no one likes it? I'm not clear on the technical details of this but from a higher-level view it seems that the devs should, if not already, think about how to push updates out.  i.e. decentralize the software

I'm not sure how to do what I'm getting at but having everyone update, in the way that's needed to implement this change, isn't going to scale into the future.
Has this been discussed already? Some kind of enforcement that would prevent unsupported software access to the network? Am I making any sense at all?

I realize pushing updates out automatically would be unpopular so another model would be needed. Not sure what model would work.

BTC: 1JgPAC8RVeh7RXqzmeL8xt3fvYahRXL3fP
kjj
Legendary
*
Offline Offline

Activity: 1302
Merit: 1025



View Profile
March 08, 2012, 06:06:51 PM
 #38

Did you all choose sf.net for mailing list because no one likes it? I'm not clear on the technical details of this but from a higher-level view it seems that the devs should, if not already, think about how to push updates out.  i.e. decentralize the software

I'm not sure how to do what I'm getting at but having everyone update, in the way that's needed to implement this change, isn't going to scale into the future.
Has this been discussed already? Some kind of enforcement that would prevent unsupported software access to the network? Am I making any sense at all?

I realize pushing updates out automatically would be unpopular so another model would be needed. Not sure what model would work.

The people that need to update to make the change stick network-wide are already aware of this, and have committed to updating their nodes.  See the second post.

The way this has been handled is appropriate for the problem at hand and the network of today.  When either of those are different, different methods will be (have been) used to update things.

Getting more than half of the hashing power onboard is all that is needed for this fix, and that is already done.  The fix does open a tiny sliver of a window of vulnerability where people who are doing things that they know they should not be doing might be at a slight risk if an attacker expends great effort for no gain.

The people that want to eliminate that tiny risk and keep being irresponsible have the option to upgrade.

Also, from your other (crazier) post, github has a copy of the bitcoin source, but the developers collectively have the bitcoin source.

17Np17BSrpnHCZ2pgtiMNnhjnsWJ2TMqq8
I routinely ignore posters with paid advertising in their sigs.  You should too.
cypherdoc
Legendary
*
Offline Offline

Activity: 1764
Merit: 1002



View Profile
March 08, 2012, 06:12:09 PM
 #39

+1 LOL!
ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1039


View Profile
March 08, 2012, 06:30:45 PM
 #40

It seems to me that 8 days is not enough time ...
Anyone who can't satisfy themselves about this change in an hour or two isn't trying.
Red Emerald
Hero Member
*****
Offline Offline

Activity: 742
Merit: 500



View Profile WWW
March 09, 2012, 12:00:43 AM
 #41

All of this technical stuff is way over my head, but I'd like to share my ignorant thoughts anyways.

It seems to me that 8 days is not enough time for people to really do their research on the matter. If I were a miner, or the operator of a mining pool, I would want to know EXACTLY what I'm doing before I made any serious changes.

Secondly, even if 8 days is enough time there is something unsettling about this. What you essentially have is a few highly skilled coders in the know. If they can dupe the mining community into something, anything, nefarious it could destroy Bitcoin. Obviously, I'm not saying that they will, but posit this:

Imagine highly skilled feds, or any other criminal organization, infiltrated the upper echelon of Bitcoin coders. They spent long hours making it better and building a reputation for themselves, slowly but surely dominating the entire field of coders. Then, one day, it's time to destroy Bitcoin and they issue some sort of scenario similar to the one we're dealing with now, for the mining community to go to a new blockchain or something, and the mining community takes their word for it, and BAM, Bitcoin is hashed forever.

I understand this is conspiracy theory stuff, but can anyone unsettle my fears about this scenario?

Simple. As long as the process is completely transparent and open there's really nothing anyone could "sneak" past everyone else in order to cause damage. And if you checked the OP, it pretty much doesn't get any more transparent then how they're going about with this fix. I mean you can even read the mailing list how the devs formulated the fix..

Two more questions then:

Is 8 days really enough time to vet a potentially nefarious plan?

What if a few people DO see something nefarious in it, will they be listened to, or ignored?

Also (all?) of the coders are known by real name. If they f*ckup bitcoin, they will be known by everyone as the person who screwed things up.
That would be the same as destroying many peoples money, and time invested and many companies and that is not like something you would like to do, ofcourse there is allways the possibility that they are threatened by jail or whatever unless they do this. But in this case its far from very likely.

No one is going to jail for destroying someone else's bitcoins until a government considers bitcoin a currency.

ssaCEO
Hero Member
*****
Offline Offline

Activity: 568
Merit: 500



View Profile WWW
March 09, 2012, 12:23:43 AM
 #42

I'm running a CentOS 5.6 build that I had to compile myself. Major alterations had to be made to the make file and tons of libraries had to be prepared. And I really, really don't want to do that again.

Any chance of a CentOS binary anytime soon? I know a lot of merchant sites could really use one. And if not - what happens if we don't upgrade?

mc_lovin
Legendary
*
Offline Offline

Activity: 1190
Merit: 1000


www.bitcointrading.com


View Profile WWW
March 09, 2012, 01:28:46 AM
 #43

Wow, I'm glad you guys are so smart.  I posted this on BitcoinTrading.com to spread the word.  Keep up the good work.
BlackPrapor
Hero Member
*****
Offline Offline

Activity: 628
Merit: 504



View Profile WWW
March 10, 2012, 01:37:14 PM
 #44

Very unlikely.  Someone would need to plan the attack very carefully, have huge amounts of hashing power hiding offline, and gain essentially nothing from it.
I don't see very much as unlikely when you're talking about challenging the global banking cartel.
Exactly. When you see these kind of guys rolling out of blue, and security fix is coming up, it makes you think twice before you do anything. Bankers can gain from this one thing, the only thing they want is total control. If BTC is destroyed, then there is no alternative. Or, it would have to be done all over again, from scratch, taking into account all previous mistakes.
Cheers,
May the BitForce be with you  Grin

There is no place like 127.0.0.1
In blockchain we trust
Luke-Jr
Legendary
*
Offline Offline

Activity: 2576
Merit: 1186



View Profile
March 10, 2012, 08:05:43 PM
 #45

Do we have an ETA on the patch getting merged and included in rc3 or whatever?
This is already part of 0.4.4rc3 and 0.5.3rc3.

Diapolo
Hero Member
*****
Offline Offline

Activity: 769
Merit: 500



View Profile WWW
March 11, 2012, 10:37:11 AM
 #46

Do we have an ETA on the patch getting merged and included in rc3 or whatever?
This is already part of 0.4.4rc3 and 0.5.3rc3.

I guess the question was, when will an 0.6 RC3 be released, that includes this fix Smiley. I use RC2 and don't want to switch back to 0.5.x.

Dia

Liked my former work for Bitcoin Core? Drop me a donation via:
1PwnvixzVAKnAqp8LCV8iuv7ohzX2pbn5x
bitcoin:1PwnvixzVAKnAqp8LCV8iuv7ohzX2pbn5x?label=Diapolo
neofutur
Full Member
***
Offline Offline

Activity: 146
Merit: 100



View Profile
March 11, 2012, 10:53:55 AM
 #47

Do we have an ETA on the patch getting merged and included in rc3 or whatever?
This is already part of 0.4.4rc3 and 0.5.3rc3.

 is there an ebuild and / or a USE flag for bip30 in the gentoo ebuild ?
Pieter Wuille (OP)
Legendary
*
Offline Offline

Activity: 1072
Merit: 1174


View Profile WWW
March 15, 2012, 04:38:24 PM
 #48

0.5.3 was just released, with BIP30 support.

I do Bitcoin stuff.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2576
Merit: 1186



View Profile
March 15, 2012, 05:03:30 PM
 #49

is there an ebuild and / or a USE flag for bip30 in the gentoo ebuild ?
There are 0.4.4 and 0.5.3 ebuilds. It's not optional, so no USE flag.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2576
Merit: 1186



View Profile
March 26, 2012, 09:17:29 PM
 #50

FWIW, this issue has been assigned CVE-2012-1909

Pages: 1 2 3 [All]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!