0.94 preliminary testing phase

[goatpig]

The new DB is about ready. It passes all C++ tests and runs *almost* fine. Turns out my own copy of the mainnet blockchain has a corrupt block (or at least some corner case that displays the same behavior), so I'll be taking extra time putting together a full diagnostic and repair code to get through that bad block. In the mean time I'd like some testing on what I have so far.

Haven't hooked the progress bars yet (so refer to your terminal/console output) but the rest should work (spends and ZC included). The code is in dev (https://github.com/goatpig/BitcoinArmory/tree/dev), pull away. I'm still pushing commits daily to that branch for now. Once it stabilizes and I put out the first testing builds, I'll move that code to a branch named testing.

If your scan fails please repeat and let me know if the error is random or deterministic.

EDIT: haven't tested rebuilds/rescans, they are probably broken. Delete the content of your DB folder manually if you want to rebuild. Obviously use a new folder (this db format is not compatible with ATI's 0.93/0.94 and coexist with it in the same folder). Expected DB size for mainnet is ~150MB.

[Carlton Banks]

  sync'ing & building   

[Carlton Banks]

Code:

(DatabaseBuilder.cpp:47) updated HEADERS db in 4322.26s

Code:

(DatabaseBuilder.cpp:108) scanned new blocks in 2111.9s

All appears well with basic Db build and tx scan (other than the disconnected progress bars). Will test some more tomorrow.

Edit: testing with Haswell i5 laptop, Qubes 3.0/Xen 4.4, Debian 8.3, Bitcoin 0.11.2

[goatpig]

No segfaults? How often have you ran it? Usually I consider 3 successful runs to be enough.

[Carlton Banks]

None. I've run it at least half a dozen times now, no segfaults so far. It does scan the wallet integrity twice, the second time after finishing the tx scan. Other than that it's in good shape on this platform.

[bitpop]

Can't wait thank you

Will you change the donation in it to you?

[achow101]

Great work! I will be testing this and will report back if anything bad happens.

Will 0.94 be just these database changes and RBF signaling?

Edit:
While armory was building the databases it crashed, twice now. I had renamed the old databases folder so armory create a new one for the new databases. This is the error that I got after both crashes:

Code:

terminate called after throwing an instance of 'LMDBException'
  what():  Failed to close env tx (Input/output error)
Aborted (core dumped)

I'm thinking this may also have to do with low disk space on my computer, so once I clean up some junk I will try again.

[goatpig]

Can't wait thank you

Will you change the donation in it to you?

Once I put out a release.

Will 0.94 be just these database changes and RBF signaling?

My plan is to put out a testing release after this phase. I'll go over all the PRs I have received and merge them in while I wait on feedback. Once I'm done with all DB bugs and PR merging, I'll release the rc1 and take it from there.

[goatpig]

I'm thinking this may also have to do with low disk space on my computer, so once I clean up some junk I will try again.

My DB is ~150MB on the mainnet. If you can't get it to scan mainnet, try the testnet, also build and run the C++ unit tests in the /cppForSwig/gtest folder and run those. Let me know if you can pass any of these and if you get errors, whether they are deterministic or random.

[bitpop]

You need a bitcoin address in your profile and sig here

[goatpig]

Not accepting any donations until I have an actual release out there. I'll deal with all that stuff once the release is ready.

[neuronics]

Not accepting any donations until I have an actual release out there. I'll deal with all that stuff once the release is ready.

Respect !

Thank you for your efforts, very much appreciated.   

Waiting for the release (and the donation address)

[RoadStress]

Not accepting any donations until I have an actual release out there. I'll deal with all that stuff once the release is ready.

Respect !

Thank you for your efforts, very much appreciated.   

Waiting for the release (and the donation address)

Thank you too! Words can't describe the appreciation for doing this.

[achow101]

I'm thinking this may also have to do with low disk space on my computer, so once I clean up some junk I will try again.

My DB is ~150MB on the mainnet. If you can't get it to scan mainnet, try the testnet, also build and run the C++ unit tests in the /cppForSwig/gtest folder and run those. Let me know if you can pass any of these and if you get errors, whether they are deterministic or random.

It looks like cleaning up my hard drive did the trick. I think that for those kinds of errors it would be useful to have something say that there isn't enough disk space available, which is what Bitcoin Core does.

[Carlton Banks]

Today's commit produces this compile error:

Code:

BlockchainScanner.cpp: In lambda function:
BlockchainScanner.cpp:235:30: error: invalid initialization of non-const reference of type ‘BlockData&’ from an rvalue of type ‘BlockData’
             return BlockData();
                              ^
Makefile:97: recipe for target 'BlockchainScanner.o' failed
make[1]: *** [BlockchainScanner.o] Error 1
make[1]: Leaving directory '/home/user/BitcoinArmory/cppForSwig'
Makefile:9: recipe for target 'all' failed
make: *** [all] Error 2

Sounds trivial to debug, but I'll leave that to those who code to live. Unless I'm doing something obviously wrong?

[goatpig]

Pushed db auto repair core and progress bar support. Scan and Rebuild should work too now. Fixed the build issue as well. Progress bars are a little wacky for now, I'll look into making them smoother tomorrow.

[droark]

Just FYI, the Mac build process is borked. I plan to do a PR with an update pretty soon.

EDIT: Should have the PR up later tonight.

[achow101]

I am having some build problems that I can't figure out. My build output is on pastebin here: http://pastebin.com/QAVGGuui

[goatpig]

I am having some build problems that I can't figure out. My build output is on pastebin here: http://pastebin.com/QAVGGuui

Looks like you are missing entire parts of the  C++ STL. Which compiler are you using? Also, is it C++11 compatible?

[droark]

I am having some build problems that I can't figure out. My build output is on pastebin here: http://pastebin.com/QAVGGuui

Same problem here. OS X. Latest Xcode (clang). Definitely C++11 compatible. (Hell, you could go up to C++14 and it wouldn't care.)

EDIT: goatpig, the following code should fix the problem.

Code:

   if (fileMap_ == MAP_FAILED) {
      fileMap_ = NULL;
      stringstream errStr;
      errStr << "Failed to create map of file. Error Code: " << errno;
      cout << errStr.str() << endl;
      throw runtime_error(errStr.str());
   }

EDIT 2: Confirmed. Armory now builds properly. Running the OS X build right now. We'll see what happens next....