|
Title: [FIXED] MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: wtogami on November 18, 2013, 03:26:40 AM FIXED.
Can you fix the MacOS X Bitcoin LevelDB data corruption issue? https://bitcointalk.org/index.php?topic=337294.msg3718821#msg3718821 TEST THESE BUILDS NOW! Bounty Funding: 10.00 BTC + 200.2 LTC Gavin Andresen has pledged 5 BTC. BitcoinTalk pledged 4 BTC. Public donations have contributed 1 BTC. Litecoin Dev Team pledges 200 LTC. The public is encouraged to contribute to these addresses to increase the incentive to fix this sooner. BTC: LTC: LS1Rb3bb29TA9PEVGR64bV2cLxC7RdQi8A (http://litecoinscout.com/address/LS1Rb3bb29TA9PEVGR64bV2cLxC7RdQi8A) Conditions The bounty may be awarded under the following conditions.
The Bitcoin developers have ultimate deciding power of how to apportion the bounty award(s) based upon the merit of the contributions This may encourage collaboration that may lead to a fix rather than hoarding of information. Non-developers may be able to figure out #1. These terms may be changed at any time for any or no reason. Background https://github.com/bitcoin/bitcoin/issues/2770 Since Bitcoin 0.8.x and the introduction of LevelDB, MacOS X users have been experiencing periodic LevelDB data corruption. For some Mac users it has never happened, while for others it happens frequently. https://github.com/bitcoin/bitcoin/pull/2916 https://github.com/bitcoin/bitcoin/pull/3000 https://github.com/bitcoin/bitcoin/pull/2933 Bitcoin master now contains two Mac-specific fsync patches and an upgrade to LevelDB 1.13. Bitcoin 0.8.5 OMG3 (https://bitcointalk.org/index.php?topic=320695.0) and Litecoin 0.8.5.2-rc5 contains these same patches. It is possible that a different Mac corruption issue was solved by these earlier patches, but users of these branches have reported continued corruption. Curiously, corruption seems to happen after a clean shutdown and restart of the client. All corruption reports seem to be from MacOS X 10.8.x and 10.9 users. It is unclear if earlier versions of MacOS X are affected. It is unknown if particular hardware or software configurations are involved. https://github.com/bitcoin/bitcoin/issues/2785 Corruption with the same error message apparently is capable of fixing itself. Not clear if this is true of the recent master branch. GPG Signed message Code: -----BEGIN PGP SIGNED MESSAGE----- Title: Re: MacOS X LevelDB Corruption Bounty (5 BTC + 200.2 LTC) Post by: Diapolo on November 18, 2013, 11:17:47 AM I created a pull (not specific to this problem), which uses std::fstream instead of fopen() and such for reading/writing block/undo files.
Perhaps this can help in a way that it works a little different than current code, dunno. I also added somewhat clearer exception error messages. https://github.com/bitcoin/bitcoin/pull/3277 It's not intended for getting merged into the master branch yet, perhaps it never will, but you can give it a try. Dia Title: Re: MacOS X LevelDB Corruption Bounty (5 BTC + 200.2 LTC) Post by: Remember remember the 5th of November on November 18, 2013, 12:07:10 PM Is it not possible that LevelDB or something else related to the data files is failing silently?
Title: Re: MacOS X LevelDB Corruption Bounty (5 BTC + 200.2 LTC) Post by: Diapolo on November 18, 2013, 12:10:36 PM Is it not possible that LevelDB or something else related to the data files is failing silently? I would say that's at least not impossible... Dia Title: Re: MacOS X LevelDB Corruption Bounty (5 BTC + 200.2 LTC) Post by: Valerian77 on November 18, 2013, 01:00:12 PM
Please refer to my posting: https://bitcointalk.org/index.php?topic=337575.msg3622968#msg3622968 (https://bitcointalk.org/index.php?topic=337575.msg3622968#msg3622968) Since I use Windows not IOS the situation may differ slightly. But at least it may be a hint. If you want to donate me: 1METhkrvz2r9d3zkFPQrHnpFC1BjCs64Zf Title: Re: MacOS X LevelDB Corruption Bounty (5 BTC + 200.2 LTC) Post by: donal on November 18, 2013, 10:29:11 PM Litecoin wallet was crashing for me, saying DB corruption, if I open terminal and enter
cd /Applications/Litecoin-Qt.app/Contents/MacOS ./Litecoin-Qt -reindex It works.. These messages are then displayed in terminal, 2013-11-18 19:57:36.821 Litecoin-Qt[991:507] CoreText performance note: Client called CTFontCreateWithName() using name "Arial" and got font with PostScript name "ArialMT". For best performance, only use PostScript names when calling this API. 2013-11-18 19:57:36.821 Litecoin-Qt[991:507] CoreText performance note: Set a breakpoint on CTFontLogSuboptimalRequest to debug. 2013-11-18 19:57:37.657 Litecoin-Qt[991:507] CoreText performance note: Client called CTFontCreateWithName() using name "Courier New" and got font with PostScript name "CourierNewPSMT". For best performance, only use PostScript names when calling this API. Title: Re: MacOS X LevelDB Corruption Bounty (5 BTC + 200.2 LTC) Post by: behindtext on November 19, 2013, 02:26:39 AM it is funny to see this considering that marco just penned a blog entry
https://blog.conformal.com/deslugging-in-go-with-pprof-btcd/ (https://blog.conformal.com/deslugging-in-go-with-pprof-btcd/) about how bitcoind uses leveldb vs what we do in btcd. to quote "Dealing with corrupt journals/flat-file/database is not only complex it has the potential of a very negative user experience. If corruption of any sort is detected then the database components must be validated, this is inherent to the its size a very long operation." apparently when using flat file storage for blocks and referencing by offset versus storing the entire block in leveldb, there are lots of unsavory ways for leveldb to fail. leveldb is a harsh mistress. Title: Re: MacOS X LevelDB Corruption Bounty (5 BTC + 200.2 LTC) Post by: Bismarck on November 19, 2013, 02:27:23 AM I'd like to point everyone's attention to this thread on the LiteCoin forums --
https://forum.litecoin.net/index.php/topic,7147.msg55666.html#msg55666 I have an LTC wallet that doesn't play well with others. I have no problems being someone's guinea pig as I'd really like to get it working again on my laptop. For the new post; I DO have TimeMachine enabled. Just for consistency; Here is the error that Litecoin-Qt keeps throwing; Code: Last login: Mon Nov 18 18:27:48 on ttys000 Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: donal on November 19, 2013, 05:16:36 PM wtogami,
I can confirm that it has nothing to do with time machine, I do not have time machine. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: wtogami on November 19, 2013, 09:40:53 PM wtogami, I can confirm that it has nothing to do with time machine, I do not have time machine. What version exactly are you running? There have been multiple fixes. Please verify specifically with Bitcoin 0.8.5 OMG3 (https://bitcointalk.org/index.php?topic=320695.0). Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: whault on November 20, 2013, 12:40:38 AM Some observations. My setup uses two drives, one with the OS and a lower speed one for general storage. I don't use time machine like the poster above, and there's nothing else non-standard about my software.
That's it really. No other behaviour is specific to corruptions for me. Sometimes they happen twice in a day, sometimes not for weeks. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: moderate on November 20, 2013, 02:42:39 AM If anything, this should serve as a warning for picking up cool new shiny things.
I take there was some discussion about why picking LevelDB was the right choice, surely it wasn't considered only because it performs faster than BDB and is developed at Google ? After that surely there were some good testing in various systems, since this is a very new low level storage, yes ? I'm just mocking here, obviously. Good luck finding and fixing the issues. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: wtogami on November 20, 2013, 03:15:40 AM If anything, this should serve as a warning for picking up cool new shiny things. I take there was some discussion about why picking LevelDB was the right choice, surely it wasn't considered only because it performs faster than BDB and is developed at Google ? After that surely there were some good testing in various systems, since this is a very new low level storage, yes ? I'm just mocking here, obviously. Good luck finding and fixing the issues. It's working quite well on Linux and Windows. Also the old BDB corrupted on all platforms, although less often than Mac users experience this current issue. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: behindtext on November 20, 2013, 11:04:03 AM If anything, this should serve as a warning for picking up cool new shiny things. I take there was some discussion about why picking LevelDB was the right choice, surely it wasn't considered only because it performs faster than BDB and is developed at Google ? After that surely there were some good testing in various systems, since this is a very new low level storage, yes ? the motivation for using leveldb vs other dbs is due to the fact that with large numbers of records, e.g. over roughly 10 mln records, most "normal" dbs start to get really sluggish on inserts and selects. you can see the behavior for yourself by stuffing a ton of records in sqlite, mysql, psql, etc. leveldb is not so much a db as a key-value store, which means that insert speed can be maintained even when there are a massive number of records, e.g. 250 mln. this is where the "level" in leveldb comes from - it load levels on inserts. the only price you pay for the load leveling is episodic compaction by leveldb. however, when doing selects/lookups on data that is already in leveldb, you must do several seeks, similar to more common databases. the likely reason leveldb was chosen is that there aren't a ton of great choices for key-value stores. many of the key-value stores besides leveldb have only a few devs and may not be actively maintained. there are also many key-value stores that have questionable data integrity. using a dependency that goes unmaintained means having to change that dep out later, a giant PITA. the reason the issue that is cited in this thread is so nasty is that not only does bitcoind use leveldb, it uses it in conjunction with flat file storage for the blocks. the act of storing data in flat files and referencing them in the db substantially increases the number and severity of error and failure paths in the combined structure (leveldb + flat file storage). as we can now see, hunting these bugs is very difficult. perhaps something can be inferred from the way in which leveldb + blocks are corrupted. this would require a dev looking at the db and blocks after they have been hosed. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: Mike Hearn on November 20, 2013, 01:15:57 PM You can blame me for LevelDB. We switched to it because it was a large (>2x) speedup over BDB and performance is critical for Bitcoin, for obvious reasons. Also BDB sucks in lots of different ways and LevelDB is very well written.
We already know Apple have made some .... questionable ... decisions in their kernel, with regard to fsync (hint: fsync doesn't). That was at least one source of corruptions, which we already fixed. Given that rather astonishing approach to data integrity there may well be other equally questionable decisions lurking under the covers. The fact that this only happens on MacOS and not any other platform is strongly indicative that Apple have done more than one bad thing. I am wondering if there is something going wrong with mmap. https://code.google.com/p/leveldb/issues/detail?id=196 The behaviour of mmap seems like it can sometimes be broken by kernel developers in subtle ways, I got a bug report for the Android app a few months ago which strongly implies mmap on Motorola devices is broken in ways that can cause data corruption. I wonder if POSIX specifies its behaviour tightly enough. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: gmaxwell on November 20, 2013, 11:40:41 PM Can we get a couple of useful bits of data for someone to work on this:
* Earliest confirmed version of 10.8 with the problem * A sample of a corrupted DB * console logs from *during time of corruption* including dmesg and system.log * Information on how bitcoin built/installed, clang? gcc42? macports/brew for deps? * if the people experiencing the problem have filevault (FDE) turned on or not, whether it was turned on during the install or after, and if it's ever been cycled on/off * also whether people who have hit this are using stock fs settings or if have case-sensitivity/etc turned on Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: italoarmstrong on November 21, 2013, 01:19:00 AM +1 on that... give me some kind of log to start.
I have a possible repro (and potentially solution) on OS X for a db corruption... not sure if its the same issue however. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: Diapolo on November 21, 2013, 04:08:59 PM What filesystems are in use on Mac? And did anyone try my std::stream branch ;)?
Dia Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: moderate on November 21, 2013, 10:21:56 PM This is on Litecoin 0.8.5.2-rc5 (same as Bitcoin 0.8.5 OMG3 (https://bitcointalk.org/index.php?topic=320695.0)) running on MacOS 10.6.8 where it does not corrupt itself. Does not corrupt or you cannot get it corrupted ? If the latter, then you solved the first step in this bounty but didn't announce it ? Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: wtogami on November 21, 2013, 10:23:45 PM This is on Litecoin 0.8.5.2-rc5 (same as Bitcoin 0.8.5 OMG3 (https://bitcointalk.org/index.php?topic=320695.0)) running on MacOS 10.6.8 where it does not corrupt itself. Does not corrupt or you cannot get it corrupted ? If the latter, then you solved the first step in this bounty but didn't announce it ? MacOS X 10.6.8 does not seem to corrupt with native Bitcoin-Qt as far as I can tell, so this test doesn't tell us anything. I am only pointing out that it is possible. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: Rampion on November 21, 2013, 10:26:29 PM Since 0.8x I have the corrupt db (and thus forced rescan) each time that my computer powers off due to battery/electrical failure.
Wtogami: try to power off your computer without shuttind down Bitcoin-QT, the db will probably be damaged. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: wtogami on November 21, 2013, 10:39:34 PM Since 0.8x I have the corrupt db (and thus forced rescan) each time that my computer powers off due to battery/electrical failure. Wtogami: try to power off your computer without shuttind down Bitcoin-QT, the db will probably be damaged. 0.8.5 OMG3 has three patches that prevent other corruption, although not the particular issue of this bounty. Also we have never seen corruption on 10.6.8. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: medicine on November 22, 2013, 07:12:14 AM I run the latest versions of both bitcoin QT and litcoin QT on my Mac running the Maverick OS. I don't use time machine and I've never had a problem with either program.
I won't be using Time machine now either. Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: toffoo on November 23, 2013, 07:43:12 AM Thank you to Gavin and Warren and everyone for your efforts to help resolve this issue. I think this bounty is a great idea and I'm very happy to see the Mac client receive some 1st class attention.
2013 has probably been bitcoin's most exciting year, but it's been a rough one for some Mac bitcoin-qt users. It was way back on December 14, 2012 when the last version of bitcoin-qt (v0.7.2 with the later May 15 workaround) to work reliably on my MacBook Pro was released. Every one of the many v0.8.x releases since has corrupted for me. I've grown weary of reindexing the blockchain and testing out new builds. Some of my experiences and setup are detailed in the original github link: https://github.com/bitcoin/bitcoin/issues/2770 I'm sure that many new Mac users who have gotten into bitcoin this year have not had the same bullet-proof experience that I enjoyed when I first got started with the earlier versions. It can be very frustrating, after all the complexity inherent to bitcoin, to have something not function the way it should when you're first getting started. All financial software needs to be rock solid, especially with the sort of values that are being tossed around these days with our experimental little currency. So with that in mind, I pleased to support this very worthwhile effort with a contribution of 0.48552571 BTC, so you can now update your bounty fund to an even 6.0 BTC + 200.2 LTC: https://blockchain.info/tx/dc17a4e4b9ae40fdec9699936f965b57f0fe3c6fce8722033c56c30a11b01900 May bitcoin-qt have a long and illustrious future running on the world's fruitiest operating system, and may this bug meet a swift and permanent demise!!! Title: Re: MacOS X LevelDB Corruption Bounty (6.00 BTC + 200.2 LTC) Post by: theymos on November 25, 2013, 03:00:23 AM The Bitcoin Forum pledges 4 BTC to this bounty, and we'll probably also put up an ad about this in forum ad round 103. This is more than I really want to spend on non-forum development issues from the forum's treasury, especially when the forum software situation is non-optimal, but this issue seems to be becoming serious despite the efforts of many experts over a long period of time, so I think that increasing the bounty amount is important.
Title: Re: MacOS X LevelDB Corruption Bounty (6.00 BTC + 200.2 LTC) Post by: gmaxwell on November 25, 2013, 03:12:12 AM Wow! Theymos, thats awesome!
Title: Re: MacOS X LevelDB Corruption Bounty (5.51 BTC + 200.2 LTC) Post by: jordan.dev on November 25, 2013, 08:17:00 AM Corrupted wallet.dat is not the issue of this particular bounty. ok nm. than sorry I misread and though my issue was related. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Boussac on November 25, 2013, 11:11:31 AM BitcoinQt 0.8.5 works like a charm on OSX 10.7.5 but crashes (with database corrupted message) on OSX 10.8.5 (brand new machine, no filevault involved nor any funky settings).
Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: cfields on November 26, 2013, 12:36:58 AM I have a theory, and a binary to try out for someone who can reproduce. I don't see the issue on my macbook.
Patch is here: https://github.com/theuni/bitcoin/commit/ac0d918bb49e0f8393b1e27662bc4c25113980b2 Binary is here for anyone who trusts me (you can, but of course you shouldn't) https://www.dropbox.com/s/r959wns6rm4xjsu/Bitcoin-Qt.dmg Theory goes like this: Currently leveldb's atomics use a gcc-style compiler barrier like: asm volatile("" : : : "memory"); OSX has a native memory barrier that can be used instead. It's my understanding that on x86/x86_64, a real memory barrier is not needed, but only a flag to tell the compiler not to optimize+reorder store/loads. Looks to me like it's a mistake, and that the OSX function should be chosen instead. In my tests, the asm hack was often completely optimized away, and loads/stores were reordered anyway. So I believe the barrier in current code is effectively a nop. Since this is a build-time optim and not run-time, it does not explain why 10.8 is affected while 10.6 is. However, if atomics are busted, timings in other underlying functions (write/fsync/mmap/etc) would have a huge effect, so it seems reasonable that this could be the indirect cause. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: moderate on November 26, 2013, 01:30:16 AM I have a theory, and a binary to try out for someone who can reproduce. I don't see the issue on my macbook. Patch is here: https://github.com/theuni/bitcoin/commit/ac0d918bb49e0f8393b1e27662bc4c25113980b2 Care to explain that your patch isn't just about moving a piece of code 20 lines above where it was previously ? I might be too tired, but that patch does nothing that wasn't being done already. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: cfields on November 26, 2013, 01:38:27 AM In the current code, OSX falls into this this case:
Code: #elif defined(ARCH_CPU_X86_FAMILY) && defined(__GNUC__) I believe that's unintentional. Moving the "#elif defined(OS_MACOSX)" up causes the barrier to be Code: OSMemoryBarrier(); Code: asm volatile("" : : : "memory");Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: moderate on November 26, 2013, 01:45:34 AM In the current code, OSX falls into this this case: Code: #elif defined(ARCH_CPU_X86_FAMILY) && defined(__GNUC__) I believe that's unintentional. Moving the "#elif defined(OS_MACOSX)" up causes the barrier to be Code: OSMemoryBarrier(); Code: asm volatile("" : : : "memory");Oh, alright, that makes sense. You should bring it up to the leveldb people. Title: GPG Signed test binaries Post by: wtogami on November 26, 2013, 09:36:55 AM Updated: December 11h, 2013
The following versions contain cfields' memory barrier patch and phantomcircuit's no-mmap patch. Extensive testing indicates no new corruption happens on MacOS X. Official Bitcoin 0.8.6 http://bitcoin.org/en/download Bitcoin 0.8.5-OMG10 http://download1.rpmfusion.org/~warren/bitcoin-0.8.5-OMG10/macosx/ https://github.com/litecoin-project/bitcoinomg/commits/0.8.5-OMG10 This build contains all the same bug fixes as 0.8.6 above. Includes additional features like Coin Control. Litecoin 0.8.6.1 https://download.litecoin.org/litecoin-0.8.6.1/ https://litecoin.info/Upgrading_Litecoin#Upgrading_to_0.8.6.1 Please follow these directions to use Litecoin 0.8.6.1. https://github.com/bitcoin/bitcoin/issues/2770 Read more here. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: cfields on November 26, 2013, 10:08:02 AM Upon further investigation, this does indeed seem to be unintended in leveldb.
Their atomics are borrowed from here (see the note in port/atomic_pointer.h): https://code.google.com/p/gperftools/source/browse/src/base/atomicops.h#102 It's clear there that mach/apple takes precedence over x86. The patch above fixes leveldb to match that behavior. Regardless of whether or not this fixes the db corruption for bitcoin, I'll take it upstream. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Mike Hearn on November 26, 2013, 10:48:28 AM Wow. That's bad and would indeed explain the issues (or at least some of them). Apple really should not be claiming in the headers to be GCC compatible and then ignoring the GCC memory barrier idiom. That's not compatible at all. So technically this is an Apple bug, I'd say, but it's one we can work around pretty easily.
Very exciting that this has been discovered, great work cfields, worthy of a bounty award for sure. It's hard to imagine this not causing corruption. And it would also explain the otherwise unexplainable "self-fixing corruption" problem. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: cfields on November 26, 2013, 11:20:29 AM Thanks Mike. Though I must point out that my summation above is a theory, and certainly shouldn't be taken as fact. But the premise is basically this: We can't really know if the GCC idiom is supported on Apple's compiler-of-the-week, but it seems we can be sure that OSMemoryBarrier will work as intended, so there should be no harm in switching to it.
Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: misterbigg on November 26, 2013, 06:40:53 PM If the Bitcoin code can be refactored, or is already factored, to support easily changing the database "Backend" then you can try HyperLevelDB or RocksDB instead, they are both based off LevelDB and with identical APIs. This can be a good solution to try if problems re-surface with LevelDB.
Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Basjee on November 26, 2013, 06:50:04 PM Noob here, how do I know if my "leveldb" is corrupted?
I have both Bitcoin client and Litecoin client running for approx. 10 days now (with some short breaks / macbook sleep time). Both my wallets are showing the correct value. 2nd question: are mac os x users risking losing their coins by running the current clients? Thanks :-) Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: picobit on November 26, 2013, 06:58:03 PM You know it is corrupted by the client complaining that it is so.
And no, you do not lose coins. The wallet is not affected (although you should always have a backup), the worst that could potentially happen is that your client "fortgets" you have the coins, or think you have too many (forgets you have spent some). But rebuilding the database will fix that, since your bitcoins are not stored on your computer but in the blockchain. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Basjee on November 26, 2013, 06:59:36 PM Thanks, that's clear! ;)
Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: moderate on November 26, 2013, 07:30:48 PM If the Bitcoin code can be refactored, or is already factored, to support easily changing the database "Backend" then you can try HyperLevelDB or RocksDB instead, they are both based off LevelDB and with identical APIs. This can be a good solution to try if problems re-surface with LevelDB. I'm not seeing the reasoning here. Are you saying that those other options fixed this bug and other ones but didn't contribute it back to leveldb ? I doubt that is the case. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: goatpig on November 26, 2013, 07:54:57 PM Are the instances of DB corruptions on Windows caused by the same problems occuring on the OSX builds?
Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: koelen3 on November 26, 2013, 08:42:10 PM Well this is my unofficial explanation.
Due of the mac updates, it changed standard applications. This happened to me when updating to Maverick Due of the changes, when having installed VLC, it uses your VLC to open your wallet.dat On the other side, you would have to unlock your basic documents when installing Bitcoin-QT to have a new transaction being sent. As this changes your file, MAC OS doesn't allow you to change the files without you unlocking them. You also need to change a 'hidden' file on your Mac, to allow you to connect from files to the internet as Mac brought some "security" updates. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: jordan314 on November 27, 2013, 06:33:47 AM I can answer 1) and 2) but don't have the skills to write the code for 3).
1) Go to system preferences and click on Date/Time. Set the date to 2012. Close System preferences and hit save. Open Bitcoin-qt and get the corrupted database message. Click abort. Set system preferences and click set date and time automatically. Open Bitcoin-qt and the database is fine. 2) The database is appearing corrupted to leveldb because the system clock is incorrect. 3) Can you use an atomic clock API and rely on that instead of the system clock to prevent leveldb from becoming corrupted? Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: marcotheminer on November 27, 2013, 06:52:32 AM Passed this on to a friend who might be able to help. we'll see!
Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: cfields on November 27, 2013, 07:13:09 AM As discussed on IRC, I don't think my first change is enough.
I've poked around and now I can reproduce the problem at-will by injecting some sleeps in key places, basically magnifying an unlikely race condition. I'll post my findings and some patches tomorrow, once I can make a bit more sense of it. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: melacs on November 27, 2013, 07:47:01 AM I can only support the people trying to solve this problem, don't have the capacity to program a solution. (It seems that the recent attention for bitcoin is putting more and more pressure on the developers.
It seems as if the sleep mode or shutdown or internet connection break is interrupting the client while it is downloading blocks. Is this true or does the problem also occur when all blocks have been downloaded before shutting down and new blocks need to be added later? In the first case, the problem could lie with an unfininished block that is considered finished or a block that is downloaded multiple times. (probably similar to what cfields stated) If it also occurs after having been shut down with a complete download of blocks, then the problem is in how the complete blockchain or a finished block is ended and restarted. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: sixtyeightmk2 on November 27, 2013, 10:35:47 AM I posted a reply about this in the newbies forums earlier tonight and on github as well.
Basically there are a few things happening with HFS and resource forks, spotlight indexing data, and ssd caching or having a race condition come up when putting the system to sleep or hibernate quickly and taking it out by turning it back on: https://bitcointalk.org/index.php?topic=348426.msg3731858#msg3731858 See portion with links to HFS ssd tuning. If you are running on a FAT partition you don't have these problems. (Can't resource fork FAT) Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Diapolo on November 27, 2013, 03:29:26 PM Are the instances of DB corruptions on Windows caused by the same problems occuring on the OSX builds? About which Windows corruptions are you talking about? Dia Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: goatpig on November 27, 2013, 04:22:55 PM Are the instances of DB corruptions on Windows caused by the same problems occuring on the OSX builds? About which Windows corruptions are you talking about? Dia Not sure under which criterion this corruption falls. About a week ago I try to start Bitcoin-Qt to no avail. Had to delete the DB folder to get Qt to start, which resulted in a full redownload and verification of the blockchain. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Roy Badami on November 28, 2013, 12:20:19 AM Apropos of nothing, I'll contribute my experience of this problem:
To the best of my knowledge, I have had precisely two unclean shutdowns of my Mac whilst Bitcoin-QT 0.8.x was running. In two out of two cases, it triggered a rebuild due to a corrupt database. So I find it interesting that this problem is actually difficult to reproduce - I guess I'm just unlucky. (This is 10.6 on a MacBook Air.) What really surprised me, though, is that on the occasion of the second of those unclean shutdowns (the OS X UI froze and I had to do a force power off) not only was I not actively using Bitcoion-QT at the time, but I am 99% sure that I had no network connection, and hadn't had a connection for several minutes. So it seems pretty unlikely that it was a pathological case involving an interrupted DB transaction. None of the above makes any sense to me, and it is, of course, quite possible just a fluke occurance - but on the off chance it gives anyone any ideas I thought I'd post it anyway. roy Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: misterbigg on November 28, 2013, 12:35:29 AM Are you saying that those other options fixed this bug and other ones but didn't contribute it back to leveldb ? I doubt that is the case. No. I'm suggesting that if there are more serious problems with LevelDB down the line that prevent features from being added (either due to performance or architecture limitations), then both HyperLevelDB and RocksDB (which are based on LevelDB) are proven alternatives with performance improvements. If the bitcoin backend database is customizable, then it is easy to experiment with alternatives. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: rrjr777 on November 28, 2013, 01:13:29 AM Hello All,
I have read through this thread and also researched some of the reports of this issue, I read about bitcoin in the beginning, and had so much going on at the time that I sort of dismissed it and forgot about it. With the recent media coverage, I have been researching bitcoin in general and became very interested, and stumbled across this thread. That being said, I have a comment, a suggestion and my own theory / hypothesis regarding what is going on, however, it is also apparent that there could very well be several unrelated "bugs" throwing the same / similar error and exhibiting similar behavior on the client machine. First, the comment: It is apparent, considering the recent media frenzy surrounding bitcoin and positive news regarding the acceptance of it as a viable currency, that any hint of instability / corruption / bugs within the codebase, could be enough to sabotage the efforts to date, and damage the confidence in the currency. Now the suggestion: I have read the reports, threads, theories, and I am hoping to contribute to the ultimate solution, however, the most important factor in potentially resolving the issue, is to gather accurate facts regarding the issue that is actually being experienced. Not to say that throwing theories out is counterproductive, it could lead someone along the right path, however, I can say for sure, that the probability of actually fixing the issue, is directly correlated to the quality of data collected. Without the exact problem / circumstances, etc. we would be shooting in the dark. I have read reports of Mac's, Windows, and other OS's at some point getting a corrupt database, and I suspect based on the sparse descriptions of the event, that the innate behavior of LevelDB is the culprit, specifically the way a write is actually committed, (or not), to media, but under what conditions this occurs is paramount to finding a solution. I wish I had gone with my intuition at the beginning of this Bitcoin project, and became involved, I'm too late to the game of mining, and can't afford to even buy a bitcoin at this point. Its taking so long for me to even download the blockchain, that I decided to start this post in parallel, and hope to make a contribution in some way. If anyone reading this thread, has a machine that is experiencing the corrupted database, it would be priceless to the community at large to post some basic information, such as: Computer OS - (version / service pack) Hard Drive - (Solid State / SATA / IDE?) Date of first experiencing the issue. The exact Error Message. What you or the computer were doing when / immediately before the error occurred. Whether you have hibernate / sleep / power off options on the Network Adapter configured. Whether your computer was hard wired or wireless. If the computer crashed prior to the database corruption. What version of BitcoinQT you were using at the time the error occurred. Whether you had a previous version of BitcoinQT installed and later upgraded. If you have been able to resolve the error, (perhaps re-installing an earlier version), and if so, how. I believe, if we have enough of the individuals who are actually experiencing the issue, report, to the best of their ability, the answers to the questions above, we as a group, may be able to put together the exact scenario / circumstance to reproduce the issue, and most importantly, begin to understand it, and progress to the path of resolution. Sorry for the lengthy post, I hope it helps. - Rich Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: cfields on November 28, 2013, 01:32:16 AM Though I hate to admit defeat (I had nailed down the issue, but my approach to fix was incorrect), Robert Escriva has submitted the correct fix: https://groups.google.com/forum/#!topic/leveldb/GXhx8YvFiig (https://groups.google.com/forum/#!topic/leveldb/GXhx8YvFiig)
It probably didn't hurt that he's a leveldb consultant ;) His victory speech: http://hackingdistributed.com/2013/11/27/bitcoin-leveldb/ I'd like to thank Robert for diving into the issue. He's got my +1 for the bounty. Title: New builds with possible MacOS fix Post by: wtogami on November 28, 2013, 10:23:02 AM https://bitcointalk.org/index.php?topic=337294.msg3718821#msg3718821
Builds were updated with Robert Escriva's patch. Please test! Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: toffoo on November 28, 2013, 06:19:53 PM Robert Escriva has submitted the correct fix: https://groups.google.com/forum/#!topic/leveldb/GXhx8YvFiig (https://groups.google.com/forum/#!topic/leveldb/GXhx8YvFiig) "It works for me" (so far ...) https://github.com/bitcoin/bitcoin/issues/2770#issuecomment-29479456 Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: doof on November 28, 2013, 09:22:00 PM I have had 2 power trips in 2 days, thanks to my Butterflylabs miner. Both times have forced me to download the chain. Relieved to see this thread and its not my mac.
Might take a look at the code tonight. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: doof on November 28, 2013, 09:29:02 PM Quote Computer OS - (version / service pack) Hard Drive - (Solid State / SATA / IDE?) Date of first experiencing the issue. The exact Error Message. What you or the computer were doing when / immediately before the error occurred. Whether you have hibernate / sleep / power off options on the Network Adapter configured. Whether your computer was hard wired or wireless. If the computer crashed prior to the database corruption. What version of BitcoinQT you were using at the time the error occurred. Whether you had a previous version of BitcoinQT installed and later upgraded. If you have been able to resolve the error, (perhaps re-installing an earlier version), and if so, how. 2011 27" iMac with HDD 28-11-2013 & 29-11-2012. The exact same times. Each morning I have woken up around 5 and plugged in my BTF Jala miner. This is cause the power to trip, and turn off my mac and other devices. I don't have a screen shot of the exact message, but off memory "Blockchain db is corrupt and needs to be re indexed" This as actually occurred on litecoin-qt. I haven't opend bitcoin-qt to see if the same has occurred. Seeing how it has happened twice, I'm certain I can reproduce. Both times, the block chain had been entirely synced, and the mac sitting there powered on. Title: Re: New builds with possible MacOS fix Post by: doof on November 28, 2013, 09:30:15 PM https://bitcointalk.org/index.php?topic=337294.msg3718821#msg3718821 Builds were updated with Robert Escriva's patch. Please test! I will apply this patch and crash my mac in the same was as the above. See if it doesn't result in a corrupt db. Title: Test new MacOS Corruption Fix Builds Post by: wtogami on November 29, 2013, 09:03:30 AM https://bitcointalk.org/index.php?topic=337294.msg3718821#msg3718821
New builds again. We need feedback from MacOS X users who were previously experiencing corruption. This is a blocker for the release of Bitcoin 0.8.6. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: edubai on November 29, 2013, 11:12:27 AM Hi Everyone,
I've tried bitcoinqt with my Mac (10.9) without any corruption problems, but it seems that for everyone who is facing this problem should consider testing his RAM for hardware problems.. a defective RAM will cause a database corruption almost instantly ! Seriously consider testing your RAM ! **RAM TEST TOOLS** for a Mac : http://osxdaily.com/2011/05/03/memtest-mac-ram-test/ (http://osxdaily.com/2011/05/03/memtest-mac-ram-test/) for Windows : http://www.memtest86.com/ (http://www.memtest86.com/) Good Luck :) Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: antanst on November 29, 2013, 06:11:48 PM No corruption (yet) in 10.9 with the latest version for me.
Title: Re: Test new MacOS Corruption Fix Builds Post by: KaosMcRage on November 30, 2013, 12:07:26 AM https://bitcointalk.org/index.php?topic=337294.msg3718821#msg3718821 New builds again. We need feedback from MacOS X users who were previously experiencing corruption. This is a blocker for the release of Bitcoin 0.8.6. As I mentioned on GitHub, I had what OMG4 considered to be a working/clean database. I didn't sync for a couple days, then downloaded and ran OMG5. Database corruption detected after attempting to sync for a minute or two. Since then, I tried opening OMG5 again at a later time and it synced just fine (no corruption detected). Downloading OMG6 now. P.S. Is coming from a database deemed clean by an older client (OMG4, 5, etc.) an adequate test or do we need to rebuild/resync from scratch (or the latest torrent copy) with each new version? Thanks. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: cfields on November 30, 2013, 04:00:48 AM After thinking this over a bit more, and seeing new crash data from toffoo, I believe there's still a bit more to fix.
From IRC: <cfields> the issue is that while it's looping through the write, it unmaps/remaps. and if there's a write that straddles a mapping, it can be split and not flushed to disk properly by osx <cfields> robert's patch fixes that part <cfields> however, once the write is complete, it's not immediately unmapped <cfields> so at that point, the very tail may not be flushed So the scenario would look like this: - Append a new record that straddles the write boundary. Let it return from a successful Append() and don't write any new data (this is the distinction) - Kill -9 or pull the plug before Close() - Robert's change has caused the interim data in Append() to be flushed to disk, however the tail will not be unmapped until Close() I've pushed a proof-of-concept "fix" here: https://github.com/theuni/bitcoin/commit/1d0e54bfb76b3891468582df97c4429174063c3c This is just a copy/paste of what happens in Sync(), except that it's forced at the end of each logical write. Note however that Sync() first flushes to disk, so it would be sending possibly unsync'd data. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: moderate on November 30, 2013, 04:03:32 AM After thinking this over a bit more, and seeing new crash data from toffoo, I believe there's still a bit more to fix. From IRC: <cfields> the issue is that while it's looping through the write, it unmaps/remaps. and if there's a write that straddles a mapping, it can be split and not flushed to disk properly by osx <cfields> robert's patch fixes that part <cfields> however, once the write is complete, it's not immediately unmapped <cfields> so at that point, the very tail may not be flushed I'm actually wondering about a very trivial thing here. Why is no one setting up small reproduceable test cases ? There is no other way to claim bugs are fixed other than doing that. Make a test where it always fail before the patch, and that it always passes after the patch. Repeat, announce fixes. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Mike Hearn on November 30, 2013, 10:09:31 AM Because it's not easy to reproduce. mmap on OS X is clearly broken in some very subtle ways, whether or not data is flushed to disk correctly almost certainly depends on the vagaries of kernel mm algorithms that can make decisions based on a lot of uncontrollable factors. That's why it's so hard to track down.
I think Sanjay may be right that on MacOS we may just have to give up on write-through mmap. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: moderate on November 30, 2013, 02:54:21 PM Maybe try contacting one of the OSX developers responsible for the memory management ? Check the darwin-kernel maillist.
People that claim something is clearly broken, yet there are many other databases running in this same OS, might just not know how to do it properly (hard to accept it, I know). I'm not saying it's an easy problem (but maybe it is, for someone that actually knows what he is doing), neither that it is easy to reproduce. Title: New Bitcoin build Post by: wtogami on December 01, 2013, 08:22:56 AM https://bitcointalk.org/index.php?topic=337294.msg3718821#msg3718821
Updated Bitcoin build to match the 0.8.6 work in progress branch at 6003954be08586092d652ca2828e86e92d96c660 This includes the patch by Robert Escriva. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Roy Badami on December 02, 2013, 12:35:04 AM So this boils down to mmap() is just broken on OS X? Wow. Just wow.
Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: wtogami on December 02, 2013, 02:36:25 AM So this boils down to mmap() is just broken on OS X? Wow. Just wow. http://s3.amazonaws.com/rapgenius/filepicker%2FMnipLDyQkWrNeFP86Ivk_Think_Different.jpg Title: Re: GPG Signed test binaries with possible fix Post by: motomike on December 02, 2013, 11:20:43 PM Updated: December 2nd, 2013 The following binaries contain the patch from Robert Escriva to workaround leveldb issue 197. Testing is needed by ALL MacOS X users, Your detailed reports of success or failure including exactly what you tested is needed. Bitcoin 0.8.6-mactest1 http://download1.rpmfusion.org/~warren/bitcoin-0.8.6-mactest1/ https://github.com/bitcoin/bitcoin/commits/0.8.6 Includes only rescriv's patch. toffoo had corruption with this build. Bitcoin 0.8.5-OMG7 (two variants) http://download1.rpmfusion.org/~warren/bitcoin-0.8.5-OMG7/ https://github.com/litecoin-project/bitcoinomg/commits/0.8.5-OMG7-no-mmap OMG7-no-mmap2 variant includes phantomcircuit's no-mmap patch and cfields mem barrier patch https://github.com/litecoin-project/bitcoinomg/commits/0.8.5-OMG7 OMG7 - like OMG5 includes rescriv's patch and cfields' mem barrier patch Litecoin 0.8.5.3-rc4 http://download1.rpmfusion.org/~warren/litecoin-0.8.5.3-rc4/macosx/ https://github.com/litecoin-project/litecoin/commits/exp-0.8.5.3-rc4 https://github.com/bitcoin/bitcoin/issues/2770 Read the latest status here. There are possibly more corruption bugs. =( Tried all of the above betas. Bitcoin-QT will start up and I can use it UNLESS it is connected to the internet. Soon as it finds Peers, Boom I get a force close, and the error. Where do I go from here? I have a few thousand worth of BTC and I need to move it. http://pastebin.com/3bFSwP5Z <- thats the whole thing. Code: Process: Bitcoin-Qt [2476] Title: Re: GPG Signed test binaries with possible fix Post by: wtogami on December 03, 2013, 12:08:14 AM Tried all of the above betas. Bitcoin-QT will start up and I can use it UNLESS it is connected to the internet. Soon as it finds Peers, Boom I get a force close, and the error. Where do I go from here? I have a few thousand worth of BTC and I need to move it. These builds with hypothetical Mac fixes will not fix an already corrupted database. Please consider using -reindex. Use 0.8.5-OMG7-no-mmap2 for now. New builds coming soon. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Coinkle on December 03, 2013, 07:22:50 AM Having always used Electrum I have never used Bitcoin-QT
I'm running 10.8.5 still. I could check for errors on a clean download of the block. I did download 0.8.5.1-beta Litecoin-QT 2days ago and it seems to be fine. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: wtogami on December 03, 2013, 07:54:26 AM Having always used Electrum I have never used Bitcoin-QT I'm running 10.8.5 still. I could check for errors on a clean download of the block. I did download 0.8.5.1-beta Litecoin-QT 2days ago and it seems to be fine. Corruption of Bitcoin-Qt or Litecoin-Qt on MacOS X 10.8+ seems to be rare, affecting only particular machines for reasons we do not understand. Title: New builds again! Post by: wtogami on December 03, 2013, 08:16:32 AM https://bitcointalk.org/index.php?topic=337294.msg3718821#msg3718821
Test now! Entirely different from yesterday. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Diapolo on December 03, 2013, 09:10:58 AM I'm not sure spaming users with Bitcoin-OMG builds (what does that name mean anyway -_-) is the right thing to do. It's nice you are doing this, but IMHO you try to be too quickly perhaps and I'm also not sure all testers are understanding, what they are testing really...
Dia Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: wtogami on December 03, 2013, 11:02:38 AM I'm not sure spaming users with Bitcoin-OMG builds (what does that name mean anyway -_-) is the right thing to do. It's nice you are doing this, but IMHO you try to be too quickly perhaps and I'm also not sure all testers are understanding, what they are testing really... Dia Prior to this bounty, the Bitcoin OMG branch has brought useful, highly tested features to a relatively stable 0.8 Bitcoin branch. This bounty was a bit tumultuous in testing of various Mac patches and the discovery of another Mac-specific GUI crash in code that was completely stable on Linux and Windows. I agree the process has been confusing but I believe it has been fruitful in exploring multiple solutions to the important Mac corruption issue and unexpectedly finding and fixing other bugs that may be an issue in HEAD. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: motomike on December 03, 2013, 09:04:32 PM I just fired up my Virtual machine and poped my wallet.dat into a win build. Waiting for this DB sucks.
Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: wtogami on December 04, 2013, 10:27:16 AM I just fired up my Virtual machine and poped my wallet.dat into a win build. Waiting for this DB sucks. Why virtual machine? The latest builds here have zero reports of new database corruption on MacOS X. Title: New builds! Post by: wtogami on December 04, 2013, 10:30:02 AM https://bitcointalk.org/index.php?topic=337294.msg3718821#msg3718821
Builds updated again with the send crash fix from laanwj (https://github.com/bitcoin/bitcoin/pull/3355). Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: doof on December 04, 2013, 11:46:06 PM Had another power cut last night before applying the patch. I have to get a ups.
Litecoin-qt was not running and it was fine. Bitcoin-qt was running at the time and it had to re-index. I've applied both patches, so Ill see how it goes. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: wtogami on December 05, 2013, 12:59:52 AM Had another power cut last night before applying the patch. I have to get a ups. Litecoin-qt was not running and it was fine. Bitcoin-qt was running at the time and it had to re-index. I've applied both patches, so Ill see how it goes. Mac? You do your own builds? Please test my latest builds. It is hard to know if your test results are accurate if you use anything else. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: gamefixer on December 05, 2013, 01:35:10 AM I'm joining the conversation as one that has tried both Bitcoin-OMG9 and MacTest1. Both crash right away with a database corruption error.
MacBook Pro 10,1 running OSX 10.8.5 if that matters. After I re-index the data base BitCoin-Qt will allow me to close and reopen the app over and over again UNTIL I power the computer down. After that I have ot re-index. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: toffoo on December 05, 2013, 03:39:53 AM I'm joining the conversation as one that has tried both Bitcoin-OMG9 and MacTest1. Both crash right away with a database corruption error. MacBook Pro 10,1 running OSX 10.8.5 if that matters. After I re-index the data base BitCoin-Qt will allow me to close and reopen the app over and over again UNTIL I power the computer down. After that I have ot re-index. None of the new builds can fix a database corrupted from an earlier version. Try running: Code: /Applications/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt -dbcache=1000 -reindex one time to reindex with the Bitcoin-OMG9 binary and let us know how it works out for you. I'm running on a similar setup to you and mactest1 did cause corruption for me but the recent OMG7-8-9 builds are running good for me now...so far. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: doof on December 05, 2013, 04:55:03 AM Had another power cut last night before applying the patch. I have to get a ups. Litecoin-qt was not running and it was fine. Bitcoin-qt was running at the time and it had to re-index. I've applied both patches, so Ill see how it goes. Mac? You do your own builds? Please test my latest builds. It is hard to know if your test results are accurate if you use anything else. Yes, 2011 iMac 27" with HDD (not SSD) running OSX Mavericks. As said before, if i kill the power on the mac, i can reproduce. Im not at home atm, but off memory, bitcoin-qt 0.8.5 and latest build of litecoin-qt off litecoin.org No I dont build my own, download them from litecoin-qt etc. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: doof on December 05, 2013, 04:57:12 AM Now I have applied both patches, Ill kill the power on litecoin-qt and see if it happens again.
Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: wtogami on December 05, 2013, 06:09:45 AM Now I have applied both patches, Ill kill the power on litecoin-qt and see if it happens again. I don't know what you mean by "both patches". Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: doof on December 05, 2013, 09:12:04 AM Now I have applied both patches, Ill kill the power on litecoin-qt and see if it happens again. I don't know what you mean by "both patches". I meant i patched litecoin-qt and bitcoin-qt Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: gamefixer on December 05, 2013, 11:55:52 AM I'm joining the conversation as one that has tried both Bitcoin-OMG9 and MacTest1. Both crash right away with a database corruption error. MacBook Pro 10,1 running OSX 10.8.5 if that matters. After I re-index the data base BitCoin-Qt will allow me to close and reopen the app over and over again UNTIL I power the computer down. After that I have ot re-index. None of the new builds can fix a database corrupted from an earlier version. Try running: Code: /Applications/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt -dbcache=1000 -reindex one time to reindex with the Bitcoin-OMG9 binary and let us know how it works out for you. I'm running on a similar setup to you and mactest1 did cause corruption for me but the recent OMG7-8-9 builds are running good for me now...so far. Thanks, I will try that when I get this computer to its power adapter. Odd thing is that I installed OMG9 on my iMac (which has never had Bitcoin-QT installed on it) and it cried about the database corruption too. ??? Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: gamefixer on December 05, 2013, 09:01:20 PM None of the new builds can fix a database corrupted from an earlier version. Try running: Code: /Applications/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt -dbcache=1000 -reindex one time to reindex with the Bitcoin-OMG9 binary and let us know how it works out for you. I'm running on a similar setup to you and mactest1 did cause corruption for me but the recent OMG7-8-9 builds are running good for me now...so far. Tried it. Worked to open bitcoin-qt OMG9 one time. After I shut it down and opened it back up again I get the corruption error again. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: wtogami on December 06, 2013, 01:52:38 AM None of the new builds can fix a database corrupted from an earlier version. Try running: Code: /Applications/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt -dbcache=1000 -reindex one time to reindex with the Bitcoin-OMG9 binary and let us know how it works out for you. I'm running on a similar setup to you and mactest1 did cause corruption for me but the recent OMG7-8-9 builds are running good for me now...so far. Tried it. Worked to open bitcoin-qt OMG9 one time. After I shut it down and opened it back up again I get the corruption error again. Can you join us in Freenode channel #bitcoin-dev? We need details from you to debug this issue. Please preserve a copy of the chainstate/ directory, compress it into a .tar.bz2 or .tar.xz archive and upload it to somewhere like Dropbox for the devs to analyze. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: gamefixer on December 06, 2013, 01:58:19 AM Can you join us in Freenode channel #bitcoin-dev? We need details from you to debug this issue. Please preserve a copy of the chainstate/ directory, compress it into a .tar.bz2 or .tar.xz archive and upload it to somewhere like Dropbox for the devs to analyze. I've got to go and get my kids so I'll do this when I get back. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: gamefixer on December 06, 2013, 02:26:17 AM Can you join us in Freenode channel #bitcoin-dev? We need details from you to debug this issue. Please preserve a copy of the chainstate/ directory, compress it into a .tar.bz2 or .tar.xz archive and upload it to somewhere like Dropbox for the devs to analyze. Hmm... I thought I could figure out how to join you guys on freenode but I'm clearly not that smart... Still trying, user gamefixer if I make it in there. Title: Re: GPG Signed test binaries Post by: kwaaak on December 06, 2013, 04:03:18 AM Bitcoin 0.8.5-OMG10 December 5th http://download1.rpmfusion.org/~warren/bitcoin-0.8.5-OMG10/macosx/ https://github.com/litecoin-project/bitcoinomg/commits/0.8.5-OMG10 This build uses PR #3340 (https://github.com/bitcoin/bitcoin/pull/3340) which switches away from mmap for writes on MacOS X. Previous OMG builds had an intermittent crash during GUI Send on MacOS X only. We think this separate MacOS-only issue may be fixed, but tests of tx sending would be very helpful. Please use Testnet. Sent a tx on mainnet, no crash, broadcasts well. Title: Re: GPG Signed test binaries Post by: wtogami on December 06, 2013, 04:29:37 AM Bitcoin 0.8.5-OMG10 December 5th http://download1.rpmfusion.org/~warren/bitcoin-0.8.5-OMG10/macosx/ https://github.com/litecoin-project/bitcoinomg/commits/0.8.5-OMG10 This build uses PR #3340 (https://github.com/bitcoin/bitcoin/pull/3340) which switches away from mmap for writes on MacOS X. Previous OMG builds had an intermittent crash during GUI Send on MacOS X only. We think this separate MacOS-only issue may be fixed, but tests of tx sending would be very helpful. Please use Testnet. Sent a tx on mainnet, no crash, broadcasts well. The old crash was like a 1 out of 50 random event. Please use testnet and test many sends. Title: FIXED MacOS builds Post by: wtogami on December 11, 2013, 08:14:28 PM https://bitcointalk.org/index.php?topic=337294.msg3718821#msg3718821
Build list updated. Official 0.8.6 releases of Bitcoin and Litecoin and OMG with additional features all contain the same patches. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: wtogami on December 12, 2013, 09:59:04 AM Quote Conditions The bounty may be awarded under the following conditions. Document how anyone can consistently reproduce the data corruption. Explain why it happens. Write a code fix that is acceptable to the Bitcoin core developers and merged into Bitcoin git master. The Bitcoin developers have ultimate deciding power of how to apportion the bounty award(s) based upon the merit of the contributions This may encourage collaboration that may lead to a fix rather than hoarding of information. Non-developers may be able to figure out #1. These terms may be changed at any time for any or no reason. These were the original bounty conditions. After consulting with core Bitcoin devs, it turns out "into Bitcoin git master" was a bad idea. What was shipped in 0.8.6 was correct and needed for Bitcoin users as quick as possible. There is a desire to be far more deliberate in testing the code that goes into git master for what will eventually become Bitcoin 0.9. For this reason we are exercising the "These terms may be changed at any time for any or no reason." clause. The final term will be changed, s/merged into Bitcoin git master/merged into Bitcoin/. The apportionment of the bounty to split to different contributors involved in fixing the issue will be the decision of Bitcoin devs including myself, Gavin, laanwj and others. Depending on the country where the various contributors are located tax forms may be required to make payments. If they refuse to do required tax forms then they may opt for their portion to be instead directed at a Bitcoin-accepting charitable organization like the FSF or EFF. Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: nazgul104 on December 27, 2013, 01:45:21 AM Is the mac LevelDB corruption issues fixed in 0.8.6?
I was trying to look for the version history but couldn't find much information in there. Could you pls confirm or point me to the right place where I can get more info on this version. By the way, I'm running OMG10 right now (thanks for that) and its been good so far without any crashes... Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: dewdeded on December 28, 2013, 08:28:53 AM Fixed.
Title: Re: [FIXED] MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: Bismarck on January 15, 2014, 11:38:14 PM Well, I have applied the 8.6.2-beta patch and it still immediately crashes upon finding peers.
Help!? Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: toffoo on June 19, 2014, 04:02:03 PM These were the original bounty conditions. After consulting with core Bitcoin devs, it turns out "into Bitcoin git master" was a bad idea. What was shipped in 0.8.6 was correct and needed for Bitcoin users as quick as possible. There is a desire to be far more deliberate in testing the code that goes into git master for what will eventually become Bitcoin 0.9. For this reason we are exercising the "These terms may be changed at any time for any or no reason." clause. The final term will be changed, s/merged into Bitcoin git master/merged into Bitcoin/. The apportionment of the bounty to split to different contributors involved in fixing the issue will be the decision of Bitcoin devs including myself, Gavin, laanwj and others. Depending on the country where the various contributors are located tax forms may be required to make payments. If they refuse to do required tax forms then they may opt for their portion to be instead directed at a Bitcoin-accepting charitable organization like the FSF or EFF. So did this bounty ever get paid? Who won it and when? Title: Re: MacOS X LevelDB Corruption Bounty (10.00 BTC + 200.2 LTC) Post by: theymos on June 19, 2014, 09:52:17 PM So did this bounty ever get paid? Who won it and when? The bounty was split equally between Cory Fields, Patrick Strateman, and Robert Escriva. |
