BDM error?

[opentoe]

I upgraded the bitcoin engine to the latest and armory also. I selected to re-scan everything when it asked me when I started the Armory. The next day I ran Armory and kept getting this BDM error indicating it needs to scan the DB again if it keeps happening. Sure enough I had to re-scan again. Has anyone come into this problem?

Windows 8 64bit. 8GB RAM.

[japerry]

I upgraded the bitcoin engine to the latest and armory also. I selected to re-scan everything when it asked me when I started the Armory. The next day I ran Armory and kept getting this BDM error indicating it needs to scan the DB again if it keeps happening. Sure enough I had to re-scan again. Has anyone come into this problem?

Windows 8 64bit. 8GB RAM.

It did it to me once but after re-scanning it's been behaving since then. Same OS as you.

[shoesman]

well ive updated from 0.92.3 with bitcoin core 0.93 wich was working great to 0.93.70 and bitcoin core 0.10 which is broken, mind you i do not have armory manage bitcoin core,  but i have bitcoin core auto startup on windows login  , on my W7 x64 install and i get a BDM error window after the blockchain has been built from scratch and once it gets to 48% transaction history scan it just gives me the BDM error window every freakin time,  wtf  , ive now rebuilt and rescanned 5 times since yesterday , plus one manual delete of all folders in the armory folder under appdata/roaming except my wallet files of course and the armory settings everything else deleted , it boggles the mind really , heres a log from armorylog

update> removed code wall due to apparently solving my issue , we shall see



btw this is not cool guys even for beta this shouldnt be happening , you should test your stuff better

[goatpig]

Code:

-WARN  - 1426014846: (..\BlockUtils.cpp:1074) Scanning from 287300 to 347035
-ERROR - 1426014865: (..\BlockWriteBatcher.cpp:963) No block in DB at height 287301

Your copy of the blockchain is missing block data for height 287301. No amount of rescanning will fix that, you need to boostrap anew.

btw this is not cool guys even for beta this shouldnt be happening , you should test your stuff better

Did you participate in the month long public testing phase?

[shoesman]

i see , well that explains alot then , and about the month long public testing well i would say yes as of yesterday apparently 

ill get back to you once i redownload the entire database over bitcoin core anew , goddamnit 

[shoesman]

ok so it finally downloaded the entire blockchain anew over bitcoin core , and i had to factory reset on the settings and data which armory suggested on restart after database rescan and rebuild , everything is working fine now , wohoo!

btw its odd , cause bitcoin core 0.10 verifys the blockchain on startup from what ive seen , and gave me no issues aka it didnt detect the No block in DB at height 287301


why is that?


and to this day bitcoin core aka bitcoinQT hasnt implemented a system to prevent database corruption ? , like when my pc loses power ,  wouldnt you think this should be the first thing to add in ? i know that utorrent implements a good way of preventing corrupt downloads by checking the files with its original hashes on download verification aka once its finished downloading , it mustve been atleast 20 times that ive redownloaded the entire blockchain database over the few years that ive been using bitcoin,  id like some thoughts on this, thanks

[Carlton Banks]

why is that?


and to this day bitcoin core aka bitcoinQT hasnt implemented a system to prevent database corruption ? , like when my pc loses power ,  wouldnt you think this should be the first thing to add in ? i know that utorrent implements a good way of preventing corrupt downloads by checking the files with its original hashes on download verification aka once its finished downloading , it mustve been atleast 20 times that ive redownloaded the entire blockchain database over the few years that ive been using bitcoin,  id like some thoughts on this, thanks

Stuff like that is the job of the OS developers, not application developers. The difference between what uTorrent and Bitcoin Core are doing as far as data integrity goes is negligible. The main difference with Bitcoin is that it has alot more hashes to verify than the typical torrent download, and so it only ever verifies the most recent 288 (IIRC) blocks when you launch Core. Verifying 347, 000+ blocks every launch would be overkill (assuming you can rely on your physical storage device, as every hash in the blockchain must be verified by Core at some point for just one Bitcoin node to function, not to mention for the overall bitcoin network to function). The design intention with uTorrent and Bitcoin is to detect corrupt data that's been received in transit, not to maintain the integrity of data from corruption where it is stored. Once it's corrupted, the hashes aren't going to bring it back.

A corrupt blockchain/Db is typically a symptom of a hard disk that's beginning to fail. No amount of software engineering can get around that problem.

[btchris]

why is that?

A corrupt blockchain/Db is typically a symptom of a hard disk that's beginning to fail. No amount of software engineering can get around that problem.

Just wanted to add that failing memory can cause file corruption as well; you might want to consider running memtest overnight just to double-check.

I had a bad RAM stick with just a single bad bit. Just one bad bit out of 64 billion was enough for me to get DB corruption issues every couple of days (and there were no other obvious symptoms anywhere else on the PC).

[shoesman]

why is that?


and to this day bitcoin core aka bitcoinQT hasnt implemented a system to prevent database corruption ? , like when my pc loses power ,  wouldnt you think this should be the first thing to add in ? i know that utorrent implements a good way of preventing corrupt downloads by checking the files with its original hashes on download verification aka once its finished downloading , it mustve been atleast 20 times that ive redownloaded the entire blockchain database over the few years that ive been using bitcoin,  id like some thoughts on this, thanks

Stuff like that is the job of the OS developers, not application developers. The difference between what uTorrent and Bitcoin Core are doing as far as data integrity goes is negligible. The main difference with Bitcoin is that it has alot more hashes to verify than the typical torrent download, and so it only ever verifies the most recent 288 (IIRC) blocks when you launch Core. Verifying 347, 000+ blocks every launch would be overkill (assuming you can rely on your physical storage device, as every hash in the blockchain must be verified by Core at some point for just one Bitcoin node to function, not to mention for the overall bitcoin network to function). The design intention with uTorrent and Bitcoin is to detect corrupt data that's been received in transit, not to maintain the integrity of data from corruption where it is stored. Once it's corrupted, the hashes aren't going to bring it back.

A corrupt blockchain/Db is typically a symptom of a hard disk that's beginning to fail. No amount of software engineering can get around that problem.

OS devs? not really bitcoins database should be setup to be able to withstand corruption from a simple immediate powerloss or continuous immediate shutdowns  , mind you bitcoins database is the only thing that corrupts on a immediate powerloss or continous immediate shutdowns while bitcoin is updating the blockchain database on my setup , all other files dont have any corruptions whatsoever , this should be fixed if possible

and no its not my storage device either since i just bought it a couple months ago , as said where talking immediate powerloss here , thats while bitcoin core and armory are running ,

 

Just wanted to add that failing memory can cause file corruption as well; you might want to consider running memtest (http://www.memtest.org/) overnight just to double-check.

I had a bad RAM stick with just a single bad bit. Just one bad bit out of 64 billion was enough for me to get DB corruption issues every couple of days (and there were no other obvious symptoms anywhere else on the PC).

failing memory? sure thing , thing is thou , youd notice this in random memory related OS Bsods, thou ive had none as well have i too ran memtest for a week on my setup after installing my ram , nothing had been found , btw memtest isnt as reliable for doing a full blown memory ics chip module test , youd actually to be 100% sure would require a hardware based ics chip tester such as what memory module manufacturers have in theyre techlabs to really be sure if your ram is 100% bit perfect or not , btw ive been through alot of ram manufacturers and the most reliable solid fault free modules ive come across are the ones from either Samsung , if you can get your hands on some , or from 2nd best , micron, when you got those theres no need for memtest , memtest is only good for a rough estimate of whats going on with your modules but wont give you an actual detailed insight no matter how long you run it

[opentoe]

I installed 93.0.70 which was supposed to take care of those BDM errors but they are back again. Every time I open the Armory I get that error and it wants to re-scan when I open it back up. Is there any light in the future when this bug will be resolved?

[opentoe]

I installed 93.0.70 which was supposed to take care of those BDM errors but they are back again. Every time I open the Armory I get that error and it wants to re-scan when I open it back up. Is there any light in the future when this bug will be resolved?

I did install 93.0.80 and was able to get to my transaction screen this time. I'll run it a couple more times.

[Carlton Banks]

I installed 93.0.70 which was supposed to take care of those BDM errors but they are back again. Every time I open the Armory I get that error and it wants to re-scan when I open it back up. Is there any light in the future when this bug will be resolved?

latest test build (0.93.0.82) here: https://bitcointalk.org/index.php?topic=919202.msg10765420#msg10765420

It has improved handling of BDM errors, try it out (cautiously).

[opentoe]

I installed 93.0.70 which was supposed to take care of those BDM errors but they are back again. Every time I open the Armory I get that error and it wants to re-scan when I open it back up. Is there any light in the future when this bug will be resolved?

latest test build (0.93.0.82) here: https://bitcointalk.org/index.php?topic=919202.msg10765420#msg10765420

It has improved handling of BDM errors, try it out (cautiously).

93.0.70 supposed to take care of BDM errors.
93.0.80 supposed to take care of BDM errors.
Now 93.0.82 supposed to take care of BDM, but with a caution.
This all makes me weary a little.

[btchris]

93.0.70 supposed to take care of BDM errors.
93.0.80 supposed to take care of BDM errors.
Now 93.0.82 supposed to take care of BDM, but with a caution.
This all makes me weary a little.

Weary, how?

Just to be clear, the BDM issues have nothing to do with your wallet or your funds. If you run into them (I have only once), they are inconvenient, and they can cause a delay if you need to send funds right away, however they shouldn't ever lead to a loss of funds.

If the potential delays of having to do a rebuild/rescan before being able to send out funds make you weary, I can understand that. In this case it might be best if you stick with 0.92.x until you're comfortable with 0.93.x.

Edited to add: I just noticed your comment in the "0.93 testing release" thread, and that you [opentoe] know all this already, so sorry for implying otherwise....

BTW, Carlton Banks seems to have a slightly unusual setup which could be why he runs into this issue more often then others. goatpig has said that CB's many issues have been helpful to him [goatpig], but I don't think this many issues is common for others (it certainly hasn't been for me).

[doug_armory]

BTW, Carlton Banks seems to have a slightly unusual setup which could be why he runs into this issue more often then others. goatpig has said that CB's many issues have been helpful to him [goatpig], but I don't think this many issues is common for others (it certainly hasn't been for me).

This. Like it or not, we just don't have massive testing rigs at Armory. In general, what we release is code that works for our setups. We know others will have different setups, which is one reason why we have test releases. The more people who bang on them and report issues to us, the more likely that the final release will be solid.

Of course, there are people like CB who just have wacky setups that take more time to fix. We're not knocking people like him. Far from it, these kinds of setups further solidify our code. It's just a frustrating process, especially if you're looking from the outside.

"Okay, I think this will work. Thoughts?"
"Looks good so far. ... Uh oh, I'm seeing it again."
"Arrrrgh! Logs."
"Sent. By the way, here's my setup."
"Okay. ... Tried this. It's a slight band-aid but it should be fine 'til we overhaul X portion of the code."
"Looks better."
"Great! Let's release...oh crap, that last-minute fix botched this other thing. Gotta fix that. ... Okay, here we go."
"I'm happy."
"Okay. Here we go."

One day, when coins are worth $100K each and we've got a warehouse full of computers, we'll come up with tests across many different setup types. Until then, we're trying hard to make everybody happy. Please bear with us.