2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community

[PawShaker]

http://spectrum.ieee.org/tech-talk/computing/networks/bitcoin-

Bitcoin went into crisis mode early this morning. This time, the threat wasn't from hackers tampering with poorly secured virtual wallets. It was Bitcoin's own code that was causing the trouble.

A compatibility issue between the two most recent versions of the cryptocurrency's core software has resulted in a split in the Bitcoin blockchain, causing the currency to grow in two different directions at once. What does this mean? The biggest problem that two competing Bitcoin chains could breed is someone trying to spend the same coins on each chain. Bitcoin was explicitly designed to resolve such an occurrence—called "double spending"—and the mere possibility has thrown the validity of some recent Bitcoin transactions into question.
[...]

[marcus_of_augustus]

"I think what will happen now is going to be a good test of the community," says Hearn. "We have to get as many people upgraded to 0.8 as possible, as fast as possible, and then go through a deliberate hard fork much earlier than we had planned."

I wish Mike Hearn wasn't out there saying stuff like this in the press. This is purely his opinion.

Colour me skeptical, but we need to remember that Mike H. put in a major effort to upgrade reference client to the levelDB database, the database widely used by his employer Google. Mike is not an independent (non-conflicted) actor when it comes to wishing everyone would just upgrade to "his" software, 0.8 levelDB.

Some people have massive and expensive infrastructure built around pre-0.8 bitcoind, scripts, supporting code, etc. 0.8 levelDB bitcoin needs to be backwardly compatible for better or worse, for now and the foreseeable future.

[notme]

"I think what will happen now is going to be a good test of the community," says Hearn. "We have to get as many people upgraded to 0.8 as possible, as fast as possible, and then go through a deliberate hard fork much earlier than we had planned."

I wish Mike Hearn wasn't out there saying stuff like this in the press. This is purely his opinion.

Colour me skeptical, but we need to remember that Mike H. put in a major effort to upgrade reference client to the levelDB database, the database widely used by his employer Google. Mike is not an independent (non-conflicted) actor when it comes to wishing everyone would just upgrade to "his" software, 0.8 levelDB.

Some people have massive and expensive infrastructure built around pre-0.8 bitcoind, scripts, supporting code, etc. 0.8 levelDB bitcoin needs to be backwardly compatible for better or worse, for now and the foreseeable future.

I agree that Mike is biased and that his solution is risky.  However, the bdb versions certainly need to be updated to properly configure the database locks.  Once this issue is fixed, we can consider our possibilities.  We either begin allowing larger blocks again and hard fork immediately (there will be some unattended nodes that continue the old chain for some time), or we take the opportunity to address the blocksize debate and maybe some other things on the hardfork wishlist.

[tjohej]

"I think what will happen now is going to be a good test of the community," says Hearn. "We have to get as many people upgraded to 0.8 as possible, as fast as possible, and then go through a deliberate hard fork much earlier than we had planned."

I wish Mike Hearn wasn't out there saying stuff like this in the press. This is purely his opinion.

People have this thing called 'freedom of expression' we all enjoy. Ooops!!  

Also if we don't allow people to defend themselves we are like denying some of their human rights.  
Bitcoin is much about freedom, we should never forget this.  
Disagreeing with another one's opinion is perfectly cool though  
As long as we stay away from Ad Hominem and such  

[marcus_of_augustus]

"I think what will happen now is going to be a good test of the community," says Hearn. "We have to get as many people upgraded to 0.8 as possible, as fast as possible, and then go through a deliberate hard fork much earlier than we had planned."

I wish Mike Hearn wasn't out there saying stuff like this in the press. This is purely his opinion.

People have this thing called 'freedom of expression' we all enjoy. Ooops!!  

Also if we don't allow people to defend themselves we are like denying some of their human rights.  
Bitcoin is much about freedom, we should never forget this.  
Disagreeing with another one's opinion is perfectly cool though  
As long as we stay away from Ad Hominem and such  

Yes, he is free to say whatever he likes ... but since he is one of main devs. people listen to him ... even if he is a loose canon, imho

[db]

Some people have massive and expensive infrastructure built around pre-0.8 bitcoind, scripts, supporting code, etc. 0.8 levelDB bitcoin needs to be backwardly compatible for better or worse, for now and the foreseeable future.

Everyone can't go on being bug compatible with 0.7 forever just because some people may have painted themselves into a corner. And why 0.7? I'm sure the same argument applied to 0.3 too.

[marcus_of_augustus]

Some people have massive and expensive infrastructure built around pre-0.8 bitcoind, scripts, supporting code, etc. 0.8 levelDB bitcoin needs to be backwardly compatible for better or worse, for now and the foreseeable future.

Everyone can't go on being bug compatible with 0.7 forever just because some people may have painted themselves into a corner. And why 0.7? I'm sure the same argument applied to 0.3 too.

Pre-0.8 does NOT have a bug, it was 0.8 that was not backwardly compatible ... do some reading.

[Technomage]

0.8 wasn't backwards compatible, but 0.7 had the bug that caused the split. I'm totally with Mike Hearn, although I don't see how this requires the hard fork any earlier. It will require a soft fork very soon though, as we want to get back to the 1MB limit, and that can only be done with 0.8.

0.8 can be used even now for mining, upgrading to it is not a problem. As long as the block size limit is kept at the default setting. Once almost everybody have upgraded, changing the block size to 1MB is no problem. It doesn't require a hard fork yet.

[marcus_of_augustus]

as we want to get back to the 1MB limit, and that can only be done with 0.8.

Wrong.

[da2ce7]

0.8 wasn't backwards compatible, but 0.7 had the bug that caused the split. I'm totally with Mike Hearn, although I don't see how this requires the hard fork any earlier. It will require a soft fork very soon though, as we want to get back to the 1MB limit, and that can only be done with 0.8.

Completely wrong.  0.8 had the bug, since it didn't imperilment the protocol as defined by the 0.7 clients.  The fact that the limitation in the 0.7 clients was unknown and undocumented is besides the point.

The absolute thing that 0.8 needs to do is not allow blocks that are considered invalid under 0.7 clients.  (no-matter how, or why they are considered invalid).


This is was an "Undocumented regression of the 0.8 client"  nothing more, nothing less.  The fact that it comes about from a "undocumented limitation of the 0.7 (and before) clients" is besides the point.

[justusranvier]

Completely wrong.  0.8 had the bug, since it didn't imperilment the protocol as defined by the 0.7 clients.  The fact that the limitation in the 0.7 clients was unknown and undocumented is besides the point.

If it's true the BDB limit was actually based on the block size of the underlying device, it's not true that previous versions followed an unknown or undocumented protocol - they actually followed no protocol at all in this area. The same version of the client would have different behavior based on the hardware it was installed on. There is no way for 0.8 to maintain "bug for bug" compatibility with non-deterministic behavior.

[Mike Hearn]

Some of you guys are nuts

0.7 has an extremely serious bug and 0.8 does not, yes, because of LevelDB. It's as simple as that. I'm kind of blown away you think I'm "biased" ... of course I'm biased in favor of software that is not seriously bugged. Who wouldn't be?

As to the idea that this fault is a protocol rule - I've never heard anything so stupid. Maybe you think that when Bitcoin 0.1 was released with bugs that allowed anyone to spend anyone elses coins, or create billions of coins out of nothing, that those bugs were "protocol rules" too? No, they weren't, they were bugs and there were hard-forking upgrades done to fix them.

The lock limit in 0.7 is not a protocol rule - it serves no useful purpose, was not previously known about and doesn't even appear to be consistent across different versions of Berkeley DB, so 0.7 nodes are already inconsistent with each other. What's more, the lock limit also applies to re-orgs. What that means is that some 0.7 nodes are in an unstable state in which they may be unable to process a valid re-org and thus permanently hose themselves, even with a 250kb soft block size limit.

In other words, I know some of you have the bizarre idea that Bitcoin should never scale beyond one transaction per second, so having a block size limit of 250kb forever is not a problem. But even with such a serious limit, 0.7 is still fundamentally unstable and may be accidentally kicked off the network by a complex enough re-org at any moment.

I first used Bitcoin in early 2009. Satoshi made hard-forking rule changes multiple times to fix bugs before 99% of you were even around, so the fact that we now need to do another should not shock or anger anyone. If you can't upgrade past 0.7 or reconfigure it with a larger lock size then your involvement with Bitcoin will end there, but I'd hope nobody has been stupid enough to get themselves into such a situation. Especially as 0.8 is API compatible with 0.7!

People who can't upgrade to 0.8 for some reason (they are reliant on custom un-merged patches for example) will have the option of patching 0.7 or adding a file to give bdb more locks, but that's just a way to buy time - it seems to hugely increase resource usage, and all it does is shift the limit a bit higher, not remove it. So there's no real alternative in the long run to upgrading.

[finway]

Some of you guys are nuts

0.7 has an extremely serious bug and 0.8 does not, yes, because of LevelDB. It's as simple as that. I'm kind of blown away you think I'm "biased" ... of course I'm biased in favor of software that is not seriously bugged. Who wouldn't be?

As to the idea that this fault is a protocol rule - I've never heard anything so stupid. Maybe you think that when Bitcoin 0.1 was released with bugs that allowed anyone to spend anyone elses coins, or create billions of coins out of nothing, that those bugs were "protocol rules" too? No, they weren't, they were bugs and there were hard-forking upgrades done to fix them.

The lock limit in 0.7 is not a protocol rule - it serves no useful purpose, was not previously known about and doesn't even appear to be consistent across different versions of Berkeley DB, so 0.7 nodes are already inconsistent with each other. What's more, the lock limit also applies to re-orgs. What that means is that some 0.7 nodes are in an unstable state in which they may be unable to process a valid re-org and thus permanently hose themselves, even with a 250kb soft block size limit.

In other words, I know some of you have the bizarre idea that Bitcoin should never scale beyond one transaction per second, so having a block size limit of 250kb forever is not a problem. But even with such a serious limit, 0.7 is still fundamentally unstable and may be accidentally kicked off the network by a complex enough re-org at any moment.

I first used Bitcoin in early 2009. Satoshi made hard-forking rule changes multiple times to fix bugs before 99% of you were even around, so the fact that we now need to do another should not shock or anger anyone. If you can't upgrade past 0.7 then your involvement with Bitcoin will end there, but I'd hope nobody has been stupid enough to get themselves into such a situation. Especially as 0.8 is API compatible with 0.7!

+1

[Technomage]

+1 from me as well. Thanks for talking some sense Mike. Occasionally I think there is none to be seen.

[Rygon]

Yeah, I don't get it, why didn't everyone just get forced to switch to 0.8 since it didn't have the BDB limit on number of transactions that 0.7 has? Why was it the other way around? Technically, 0.8 was winning in chain length also.

[justusranvier]

Yeah, I don't get it, why didn't everyone just get forced to switch to 0.8 since it didn't have the BDB limit on number of transactions that 0.7 has? Why was it the other way around?

Based on what I've read my guess is that many significant non-mining businesses are still on 0.7 and couldn't upgrade quickly enough.

[Mike Hearn]

Yes, unexpected crash upgrades are bad news, but if this event had happened at a later time when 0.7 was not widely used, it might have made more sense to just abandon it.

[Gabi]

+1 to Mike!

[Akka]

Just one question Mike,

isn't this kind of different as on the previous forks, clients of old versions where still able to accept 100% percent of all blocks the new versions accepted, but the new versions "filtered" some out, that where created with old versions? So old versions where still able to get on the new chain as soon as it was the longer chain.

With the new soon necessary fork it's the other way around, old clients will reject new client blocks, so they will stay on their fork if they don't upgrade.

So it's kind of a new problem.

Correct me if I'm wrong.


But I totally agree with the rest +1 from me too.

We can't expect that bitcoin will never have upgrades that make it downwards incompatible. This will happen again and on the bright side this is an opportunity to test this how it can be done in a smooth way on an issue where (almost) everyone agrees that it needs to be addressed.

[Dusty]

Satoshi made hard-forking rule changes multiple times to fix bugs before 99% of you were even around, so the fact that we now need to do another should not shock or anger anyone. If you can't upgrade past 0.7 or reconfigure it with a larger lock size then your involvement with Bitcoin will end there, but I'd hope nobody has been stupid enough to get themselves into such a situation. Especially as 0.8 is API compatible with 0.7!

+100

Thanks Mike for putting some sense on some of the crazy rants I've read in this thread.