|
Title: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: PawShaker on March 13, 2013, 12:19:05 AM http://spectrum.ieee.org/tech-talk/computing/networks/bitcoin- (http://spectrum.ieee.org/tech-talk/computing/networks/bitcoin-)
Quote Bitcoin went into crisis mode early this morning. This time, the threat wasn't from hackers tampering with poorly secured virtual wallets. It was Bitcoin's own code that was causing the trouble. A compatibility issue between the two most recent versions of the cryptocurrency's core software has resulted in a split in the Bitcoin blockchain, causing the currency to grow in two different directions at once. What does this mean? The biggest problem that two competing Bitcoin chains could breed is someone trying to spend the same coins on each chain. Bitcoin was explicitly designed to resolve such an occurrence—called "double spending"—and the mere possibility has thrown the validity of some recent Bitcoin transactions into question. [...] Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: marcus_of_augustus on March 13, 2013, 03:25:39 AM Quote "I think what will happen now is going to be a good test of the community," says Hearn. "We have to get as many people upgraded to 0.8 as possible, as fast as possible, and then go through a deliberate hard fork much earlier than we had planned." I wish Mike Hearn wasn't out there saying stuff like this in the press. This is purely his opinion. Colour me skeptical, but we need to remember that Mike H. put in a major effort to upgrade reference client to the levelDB database, the database widely used by his employer Google. Mike is not an independent (non-conflicted) actor when it comes to wishing everyone would just upgrade to "his" software, 0.8 levelDB. Some people have massive and expensive infrastructure built around pre-0.8 bitcoind, scripts, supporting code, etc. 0.8 levelDB bitcoin needs to be backwardly compatible for better or worse, for now and the foreseeable future. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: notme on March 13, 2013, 05:59:01 AM Quote "I think what will happen now is going to be a good test of the community," says Hearn. "We have to get as many people upgraded to 0.8 as possible, as fast as possible, and then go through a deliberate hard fork much earlier than we had planned." I wish Mike Hearn wasn't out there saying stuff like this in the press. This is purely his opinion. Colour me skeptical, but we need to remember that Mike H. put in a major effort to upgrade reference client to the levelDB database, the database widely used by his employer Google. Mike is not an independent (non-conflicted) actor when it comes to wishing everyone would just upgrade to "his" software, 0.8 levelDB. Some people have massive and expensive infrastructure built around pre-0.8 bitcoind, scripts, supporting code, etc. 0.8 levelDB bitcoin needs to be backwardly compatible for better or worse, for now and the foreseeable future. I agree that Mike is biased and that his solution is risky. However, the bdb versions certainly need to be updated to properly configure the database locks. Once this issue is fixed, we can consider our possibilities. We either begin allowing larger blocks again and hard fork immediately (there will be some unattended nodes that continue the old chain for some time), or we take the opportunity to address the blocksize debate and maybe some other things on the hardfork wishlist. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: tjohej on March 13, 2013, 06:39:48 AM Quote "I think what will happen now is going to be a good test of the community," says Hearn. "We have to get as many people upgraded to 0.8 as possible, as fast as possible, and then go through a deliberate hard fork much earlier than we had planned." I wish Mike Hearn wasn't out there saying stuff like this in the press. This is purely his opinion. Also if we don't allow people to defend themselves we are like denying some of their human rights. :( Bitcoin is much about freedom, we should never forget this. 8) Disagreeing with another one's opinion is perfectly cool though :D As long as we stay away from Ad Hominem and such :) Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: marcus_of_augustus on March 13, 2013, 07:00:51 AM Quote "I think what will happen now is going to be a good test of the community," says Hearn. "We have to get as many people upgraded to 0.8 as possible, as fast as possible, and then go through a deliberate hard fork much earlier than we had planned." I wish Mike Hearn wasn't out there saying stuff like this in the press. This is purely his opinion. Also if we don't allow people to defend themselves we are like denying some of their human rights. :( Bitcoin is much about freedom, we should never forget this. 8) Disagreeing with another one's opinion is perfectly cool though :D As long as we stay away from Ad Hominem and such :) Yes, he is free to say whatever he likes ... but since he is one of main devs. people listen to him ... even if he is a loose canon, imho ;) Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: db on March 13, 2013, 07:35:14 AM Some people have massive and expensive infrastructure built around pre-0.8 bitcoind, scripts, supporting code, etc. 0.8 levelDB bitcoin needs to be backwardly compatible for better or worse, for now and the foreseeable future. Everyone can't go on being bug compatible with 0.7 forever just because some people may have painted themselves into a corner. And why 0.7? I'm sure the same argument applied to 0.3 too. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: marcus_of_augustus on March 13, 2013, 08:23:30 AM Some people have massive and expensive infrastructure built around pre-0.8 bitcoind, scripts, supporting code, etc. 0.8 levelDB bitcoin needs to be backwardly compatible for better or worse, for now and the foreseeable future. Everyone can't go on being bug compatible with 0.7 forever just because some people may have painted themselves into a corner. And why 0.7? I'm sure the same argument applied to 0.3 too. Pre-0.8 does NOT have a bug, it was 0.8 that was not backwardly compatible ... do some reading. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Technomage on March 13, 2013, 08:52:02 AM 0.8 wasn't backwards compatible, but 0.7 had the bug that caused the split. I'm totally with Mike Hearn, although I don't see how this requires the hard fork any earlier. It will require a soft fork very soon though, as we want to get back to the 1MB limit, and that can only be done with 0.8.
0.8 can be used even now for mining, upgrading to it is not a problem. As long as the block size limit is kept at the default setting. Once almost everybody have upgraded, changing the block size to 1MB is no problem. It doesn't require a hard fork yet. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: marcus_of_augustus on March 13, 2013, 08:56:32 AM Quote as we want to get back to the 1MB limit, and that can only be done with 0.8. Wrong.Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: da2ce7 on March 13, 2013, 11:36:25 AM 0.8 wasn't backwards compatible, but 0.7 had the bug that caused the split. I'm totally with Mike Hearn, although I don't see how this requires the hard fork any earlier. It will require a soft fork very soon though, as we want to get back to the 1MB limit, and that can only be done with 0.8. Completely wrong. 0.8 had the bug, since it didn't imperilment the protocol as defined by the 0.7 clients. The fact that the limitation in the 0.7 clients was unknown and undocumented is besides the point. The absolute thing that 0.8 needs to do is not allow blocks that are considered invalid under 0.7 clients. (no-matter how, or why they are considered invalid). This is was an "Undocumented regression of the 0.8 client" nothing more, nothing less. The fact that it comes about from a "undocumented limitation of the 0.7 (and before) clients" is besides the point. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: justusranvier on March 13, 2013, 12:35:03 PM Completely wrong. 0.8 had the bug, since it didn't imperilment the protocol as defined by the 0.7 clients. The fact that the limitation in the 0.7 clients was unknown and undocumented is besides the point. If it's true the BDB limit was actually based on the block size of the underlying device, it's not true that previous versions followed an unknown or undocumented protocol - they actually followed no protocol at all in this area. The same version of the client would have different behavior based on the hardware it was installed on. There is no way for 0.8 to maintain "bug for bug" compatibility with non-deterministic behavior.Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Mike Hearn on March 13, 2013, 01:01:48 PM Some of you guys are nuts :)
0.7 has an extremely serious bug and 0.8 does not, yes, because of LevelDB. It's as simple as that. I'm kind of blown away you think I'm "biased" ... of course I'm biased in favor of software that is not seriously bugged. Who wouldn't be? As to the idea that this fault is a protocol rule - I've never heard anything so stupid. Maybe you think that when Bitcoin 0.1 was released with bugs that allowed anyone to spend anyone elses coins, or create billions of coins out of nothing, that those bugs were "protocol rules" too? No, they weren't, they were bugs and there were hard-forking upgrades done to fix them. The lock limit in 0.7 is not a protocol rule - it serves no useful purpose, was not previously known about and doesn't even appear to be consistent across different versions of Berkeley DB, so 0.7 nodes are already inconsistent with each other. What's more, the lock limit also applies to re-orgs. What that means is that some 0.7 nodes are in an unstable state in which they may be unable to process a valid re-org and thus permanently hose themselves, even with a 250kb soft block size limit. In other words, I know some of you have the bizarre idea that Bitcoin should never scale beyond one transaction per second, so having a block size limit of 250kb forever is not a problem. But even with such a serious limit, 0.7 is still fundamentally unstable and may be accidentally kicked off the network by a complex enough re-org at any moment. I first used Bitcoin in early 2009. Satoshi made hard-forking rule changes multiple times to fix bugs before 99% of you were even around, so the fact that we now need to do another should not shock or anger anyone. If you can't upgrade past 0.7 or reconfigure it with a larger lock size then your involvement with Bitcoin will end there, but I'd hope nobody has been stupid enough to get themselves into such a situation. Especially as 0.8 is API compatible with 0.7! People who can't upgrade to 0.8 for some reason (they are reliant on custom un-merged patches for example) will have the option of patching 0.7 or adding a file to give bdb more locks, but that's just a way to buy time - it seems to hugely increase resource usage, and all it does is shift the limit a bit higher, not remove it. So there's no real alternative in the long run to upgrading. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: finway on March 13, 2013, 01:35:27 PM Some of you guys are nuts :) +10.7 has an extremely serious bug and 0.8 does not, yes, because of LevelDB. It's as simple as that. I'm kind of blown away you think I'm "biased" ... of course I'm biased in favor of software that is not seriously bugged. Who wouldn't be? As to the idea that this fault is a protocol rule - I've never heard anything so stupid. Maybe you think that when Bitcoin 0.1 was released with bugs that allowed anyone to spend anyone elses coins, or create billions of coins out of nothing, that those bugs were "protocol rules" too? No, they weren't, they were bugs and there were hard-forking upgrades done to fix them. The lock limit in 0.7 is not a protocol rule - it serves no useful purpose, was not previously known about and doesn't even appear to be consistent across different versions of Berkeley DB, so 0.7 nodes are already inconsistent with each other. What's more, the lock limit also applies to re-orgs. What that means is that some 0.7 nodes are in an unstable state in which they may be unable to process a valid re-org and thus permanently hose themselves, even with a 250kb soft block size limit. In other words, I know some of you have the bizarre idea that Bitcoin should never scale beyond one transaction per second, so having a block size limit of 250kb forever is not a problem. But even with such a serious limit, 0.7 is still fundamentally unstable and may be accidentally kicked off the network by a complex enough re-org at any moment. I first used Bitcoin in early 2009. Satoshi made hard-forking rule changes multiple times to fix bugs before 99% of you were even around, so the fact that we now need to do another should not shock or anger anyone. If you can't upgrade past 0.7 then your involvement with Bitcoin will end there, but I'd hope nobody has been stupid enough to get themselves into such a situation. Especially as 0.8 is API compatible with 0.7! Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Technomage on March 13, 2013, 02:08:31 PM +1 from me as well. Thanks for talking some sense Mike. Occasionally I think there is none to be seen.
Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Rygon on March 13, 2013, 02:09:49 PM Yeah, I don't get it, why didn't everyone just get forced to switch to 0.8 since it didn't have the BDB limit on number of transactions that 0.7 has? Why was it the other way around? Technically, 0.8 was winning in chain length also.
Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: justusranvier on March 13, 2013, 02:13:17 PM Yeah, I don't get it, why didn't everyone just get forced to switch to 0.8 since it didn't have the BDB limit on number of transactions that 0.7 has? Why was it the other way around? Based on what I've read my guess is that many significant non-mining businesses are still on 0.7 and couldn't upgrade quickly enough.Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Mike Hearn on March 13, 2013, 02:21:24 PM Yes, unexpected crash upgrades are bad news, but if this event had happened at a later time when 0.7 was not widely used, it might have made more sense to just abandon it.
Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Gabi on March 13, 2013, 02:59:15 PM +1 to Mike!
Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Akka on March 13, 2013, 03:07:08 PM Just one question Mike,
isn't this kind of different as on the previous forks, clients of old versions where still able to accept 100% percent of all blocks the new versions accepted, but the new versions "filtered" some out, that where created with old versions? So old versions where still able to get on the new chain as soon as it was the longer chain. With the new soon necessary fork it's the other way around, old clients will reject new client blocks, so they will stay on their fork if they don't upgrade. So it's kind of a new problem. Correct me if I'm wrong. But I totally agree with the rest +1 from me too. We can't expect that bitcoin will never have upgrades that make it downwards incompatible. This will happen again and on the bright side this is an opportunity to test this how it can be done in a smooth way on an issue where (almost) everyone agrees that it needs to be addressed. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Dusty on March 13, 2013, 03:12:07 PM Satoshi made hard-forking rule changes multiple times to fix bugs before 99% of you were even around, so the fact that we now need to do another should not shock or anger anyone. If you can't upgrade past 0.7 or reconfigure it with a larger lock size then your involvement with Bitcoin will end there, but I'd hope nobody has been stupid enough to get themselves into such a situation. Especially as 0.8 is API compatible with 0.7! +100Thanks Mike for putting some sense on some of the crazy rants I've read in this thread. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: elux on March 13, 2013, 03:16:54 PM Quote from: Mike Hearn "We have to get as many people upgraded to 0.8 as possible, as fast as possible, and then go through a deliberate hard fork much earlier than we had planned." I wish Mike Hearn wasn't out there saying stuff like this in the press. This is purely his opinion. Some people have massive and expensive infrastructure built around pre-0.8 bitcoind, scripts, supporting code, etc. 0.8 levelDB bitcoin needs to be backwardly compatible for better or worse, for now and the foreseeable future. Code: <gavinandresen> We definitely need a hardfork. Version 0.3 and version 0.7 are incompatible, we just didn't know it Yes, he is free to say whatever he likes ... but since he is one of main devs. people listen to him ... even if he is a loose canon, imho ;) Code: <gmaxwell>We _never_ would have release 0.8 with this behavior if we knew about it. Everyone can't go on being bug compatible with 0.7 forever just because some people may have painted themselves into a corner. And why 0.7? I'm sure the same argument applied to 0.3 too. Pre-0.8 does NOT have a bug, it was 0.8 that was not backwardly compatible ... do some reading.Maybe, but it's not that simple: Code: [2013-03-12 17:37:15] <Luke-Jr> zephirum: this was NOT 0.7 vs 0.8 tl;dr: Code: <sipa> num1: i'd say 0.7 has the bug, but 0.8 was incorrect :) Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Mike Hearn on March 13, 2013, 03:36:55 PM isn't this kind of different as on the previous forks, clients of old versions where still able to accept 100% percent of all blocks the new versions accepted, but the new versions "filtered" some out, that where created with old versions? So old versions where still able to get on the new chain as soon as it was the longer chain. No. OK, a bit of history. I was referring two previous hard-forking changes that I'm aware of. There were probably more. When Bitcoin was first released, it contained two completely fatal bugs that made the entire system worthless. Fortunately, they were found and fixed before Bitcoin actually had any serious value. The first bug was that scripts were concatenated before being run instead of just using a shared stack. This meant that anyone could write a scriptSig that always evaluated to true and claim anyone elses coins. Fixed here in v0.3.2: https://github.com/bitcoin/bitcoin/commit/73aa262647ff9948eaf95e83236ec323347e95d0 https://en.bitcoin.it/wiki/Incidents#CVE-2010-5141 Needless to say, if somebody when this version was first released actually wrote such a scriptSig and stole some coins, that would have caused a chain split between old and new versions. Nobody did because, why bother? I'm not even sure Mt Gox existed back then, iirc that came some months later. Many script opcodes were disabled around this time (which is also a hard-forking change). The second bug was found some months later, and this time it was exploited: https://bitcointalk.org/index.php?topic=822.0 A fixed version of the software was quickly released. There was a race between the old and new chains. After some time (was it 12 hours?) the new chain beat out the old one. Even then, miners who didn't upgrade kept trying to extend the broken chain for a long time. Back then there was no talk about clearly broken behaviour being some kind of unbreakable "protocol rule". You may see some people say that bugs in the old software become a part of the protocol and yes, this is true within limits. For instance, OP_CHECKMULTISIG contains a dumb bug, but it's not dangerous and can be easily worked around. So it's better for new versions to just not fix the bug, the cost of a hard fork isn't worth fixing it. Old versions being unable to digest even quite tiny blocks is clearly far more serious and the cost of a hard fork is worth it in that case. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: nevafuse on March 13, 2013, 05:46:11 PM Yeah, I don't get it, why didn't everyone just get forced to switch to 0.8 since it didn't have the BDB limit on number of transactions that 0.7 has? Why was it the other way around? Based on what I've read my guess is that many significant non-mining businesses are still on 0.7 and couldn't upgrade quickly enough.+1 Doesn't matter which chain is longer if a majority of the people aren't on it. Breaking changes need to be given lots of warning to be effective. Trying to force everyone to use 0.8 would have only made the situation worse. From the chat discussion, I don't think mtgox was using 0.8. So trading at the largest exchange would be halted until it could be upgraded. If that doesn't sound disastrous, I'm not sure what does. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Rygon on March 13, 2013, 06:13:11 PM Yeah, I don't get it, why didn't everyone just get forced to switch to 0.8 since it didn't have the BDB limit on number of transactions that 0.7 has? Why was it the other way around? Based on what I've read my guess is that many significant non-mining businesses are still on 0.7 and couldn't upgrade quickly enough.+1 Doesn't matter which chain is longer if a majority of the people aren't on it. Breaking changes need to be given lots of warning to be effective. Trying to force everyone to use 0.8 would have only made the situation worse. From the chat discussion, I don't think mtgox was using 0.8. So trading at the largest exchange would be halted until it could be upgraded. If that doesn't sound disastrous, I'm not sure what does. Thanks. It makes more sense to me now. Basically, 0.8 had created an unplanned hard fork, and took many users (not necessarily miners) hadn't upgraded, so transactions would have been dubious until they did so. Functionally, it sounds to me like 0.80 was a better version, but it wasn't backwards compatible and no one had planned for a hard fork. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: justusranvier on March 13, 2013, 06:20:14 PM I'm pretty sure Mt Gox, blockchain.info, and BitPay were all on 0.7, and probably some other major sites as well. Sites like that are probably less capable of changing versions overnight compared to mining pools.
Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Piper67 on March 13, 2013, 06:22:04 PM I'm pretty sure Mt Gox, blockchain.info, and BitPay were all on 0.7, and probably some other major sites as well. Sites like that are probably less capable of changing versions overnight compared to mining pools. It baffles me why they would not have switched to 0.8. That version was out for weeks before the fork happened. Oh well, live and learn. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: marcus_of_augustus on March 13, 2013, 08:12:43 PM Mike H.
Quote The lock limit in 0.7 is not a protocol rule - it serves no useful purpose, was not previously known about and doesn't even appear to be consistent across different versions of Berkeley DB, so 0.7 nodes are already inconsistent with each other. What's more, the lock limit also applies to re-orgs. What that means is that some 0.7 nodes are in an unstable state in which they may be unable to process a valid re-org and thus permanently hose themselves, even with a 250kb soft block size limit. Incorrect that the lock limit rule was not known about. It is stated explicitly in the pre-0.7 source code, db.cpp lines 82 and 83 Code:
But good to see that you have finally gotten to grips with BDB lock limits ... AFTER implementing the new levelDB that ignored these two lines of code in all pre-0.8 bitcoin reference code ... which recall by your own words ... to paraphrase "if you want a protocol read the source code" 0.8 maybe a superior solution but it has been rushed out. The fact you haven't fully understood the previous code and implemented the upgrade correctly may be more your deficiency than pre-0.7 code. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: nevafuse on March 13, 2013, 08:37:11 PM I'm pretty sure Mt Gox, blockchain.info, and BitPay were all on 0.7, and probably some other major sites as well. Sites like that are probably less capable of changing versions overnight compared to mining pools. It baffles me why they would not have switched to 0.8. That version was out for weeks before the fork happened. Oh well, live and learn. Time, money, etc to test, implement, copy over any custom code, etc. And the consequences if something is missed. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: ShadowOfHarbringer on March 13, 2013, 10:12:03 PM I'm pretty sure Mt Gox, blockchain.info, and BitPay were all on 0.7, and probably some other major sites as well. Sites like that are probably less capable of changing versions overnight compared to mining pools. It baffles me why they would not have switched to 0.8. That version was out for weeks before the fork happened. Oh well, live and learn. Then you don't really know a lot about large institutions, do you ? I would bet that many banks run software written in 1999 or even before, you know why ? Because when huge risks and costs are involved, you don't change something that works properly unless there is a serious reason. Hell, some large companies still run COBOL code written 30-40 years ago ! Changing software versions is a significant risk each time it is done. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: johnyj on March 13, 2013, 10:23:42 PM My rule of thumb after 20+ years in software engineering: Never be the first to try the latest version until it has been used by others for months :) In fact some of my machines still run 0.3
Maybe the newer version is superior, but people could use old version to do many kind of fancy things that developer can never imagine and test. A new version typically affect many infrastructures and relations that were built upon those old version Luckily bitcoin has not reached such maturity level that there are hundreds of other services are dependant on the blockchain Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: Mike Hearn on March 13, 2013, 10:49:01 PM We knew there was a limit. What nobody knew is that the limit was low enough to be hit with blocks smaller than 1 megabyte. Remember that 1mb blocks had been tried before and worked.
The BDB manual doesn't give a formula for selecting the number of locks, presumably because not even the BDB developers know how to do it correctly, or perhaps because it seems to vary across versions/platforms. It just says "pick a number that seems to work and then double it". That kind of nonsense is fortunately not present in LevelDB. Quote But good to see that you have finally gotten to grips with BDB lock limits ... AFTER implementing the new levelDB that ignored these two lines of code in all pre-0.8 bitcoin reference code ... which recall by your own words ... to paraphrase "if you want a protocol read the source code" Seems I was right about that, wasn't I? Hence the warnings against re-implementations. Understanding and documenting all these edge case behaviours is very difficult even for people who work on the existing code. So if you rewrite everything from a wiki page it's going to be even harder to match things precisely. Quote 0.8 maybe a superior solution but it has been rushed out. The fact you haven't fully understood the previous code and implemented the upgrade correctly may be more your deficiency than pre-0.7 code. In what sense was 0.8 rushed out? The lock limits issue has been in the code for a very long time and went away entirely with 0.8, so spending more time testing it would not have helped reveal the problem. You'd have to have been testing old releases to find this issue, and moreover, testing it in non-obvious ways. I'm tired of these sorts of threads and will not be posting on the issue further. The code that Satoshi left behind is certainly not ideal in many ways, and the only way to fix it is incremental upgrades. That means real work. If you are going to make yourself useful instead of just insulting people anonymously, then get to it and start finding bugs. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: marcus_of_augustus on March 14, 2013, 08:47:00 PM Quote I'm tired of these sorts of threads and will not be posting on the issue further. The code that Satoshi left behind is certainly not ideal in many ways, and the only way to fix it is incremental upgrades. That means real work. If you are going to make yourself useful instead of just insulting people anonymously, then get to it and start finding bugs. Probably best if we leave at that then. I'm happy the problem has been laid bare for all to see and the community is made aware of the issues in a transparent way so informed decisions can be made. Fear not, real work is happening. Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: notme on March 15, 2013, 05:01:39 PM We knew there was a limit. What nobody knew is that the limit was low enough to be hit with blocks smaller than 1 megabyte. Remember that 1mb blocks had been tried before and worked. The BDB manual doesn't give a formula for selecting the number of locks, presumably because not even the BDB developers know how to do it correctly, or perhaps because it seems to vary across versions/platforms. It just says "pick a number that seems to work and then double it". That kind of nonsense is fortunately not present in LevelDB. Quote But good to see that you have finally gotten to grips with BDB lock limits ... AFTER implementing the new levelDB that ignored these two lines of code in all pre-0.8 bitcoin reference code ... which recall by your own words ... to paraphrase "if you want a protocol read the source code" Seems I was right about that, wasn't I? Hence the warnings against re-implementations. Understanding and documenting all these edge case behaviours is very difficult even for people who work on the existing code. So if you rewrite everything from a wiki page it's going to be even harder to match things precisely. Quote 0.8 maybe a superior solution but it has been rushed out. The fact you haven't fully understood the previous code and implemented the upgrade correctly may be more your deficiency than pre-0.7 code. In what sense was 0.8 rushed out? The lock limits issue has been in the code for a very long time and went away entirely with 0.8, so spending more time testing it would not have helped reveal the problem. You'd have to have been testing old releases to find this issue, and moreover, testing it in non-obvious ways. I'm tired of these sorts of threads and will not be posting on the issue further. The code that Satoshi left behind is certainly not ideal in many ways, and the only way to fix it is incremental upgrades. That means real work. If you are going to make yourself useful instead of just insulting people anonymously, then get to it and start finding bugs. Can you explain how locking is handled better by leveldb for the particular type of database transaction that can require unbounded locks on bdb? Title: Re: 2013-03-12 IEEE Spectrum: Major Bug In The Bitcoin Software Tests The Community Post by: jgarzik on March 15, 2013, 05:26:36 PM Can you explain how locking is handled better by leveldb for the particular type of database transaction that can require unbounded locks on bdb? It is simply a different design. leveldb does not need per-page locks, and is not an ACID database. leveldb has a concept called a "batch", and batches are committed atomically. |