|
Title: 0.94 preliminary testing phase Post by: goatpig on February 21, 2016, 06:15:59 PM The new DB is about ready. It passes all C++ tests and runs *almost* fine. Turns out my own copy of the mainnet blockchain has a corrupt block (or at least some corner case that displays the same behavior), so I'll be taking extra time putting together a full diagnostic and repair code to get through that bad block. In the mean time I'd like some testing on what I have so far.
Haven't hooked the progress bars yet (so refer to your terminal/console output) but the rest should work (spends and ZC included). The code is in dev (https://github.com/goatpig/BitcoinArmory/tree/dev), pull away. I'm still pushing commits daily to that branch for now. Once it stabilizes and I put out the first testing builds, I'll move that code to a branch named testing. If your scan fails please repeat and let me know if the error is random or deterministic. EDIT: haven't tested rebuilds/rescans, they are probably broken. Delete the content of your DB folder manually if you want to rebuild. Obviously use a new folder (this db format is not compatible with ATI's 0.93/0.94 and coexist with it in the same folder). Expected DB size for mainnet is ~150MB. Title: Re: 0.94 preliminary testing phase Post by: Carlton Banks on February 22, 2016, 12:13:31 AM :) sync'ing & building 8)
Title: Re: 0.94 preliminary testing phase Post by: Carlton Banks on February 22, 2016, 04:20:27 AM Code: (DatabaseBuilder.cpp:47) updated HEADERS db in 4322.26s Code: (DatabaseBuilder.cpp:108) scanned new blocks in 2111.9s All appears well with basic Db build and tx scan (other than the disconnected progress bars). Will test some more tomorrow. Edit: testing with Haswell i5 laptop, Qubes 3.0/Xen 4.4, Debian 8.3, Bitcoin 0.11.2 Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 22, 2016, 05:53:16 PM No segfaults? How often have you ran it? Usually I consider 3 successful runs to be enough.
Title: Re: 0.94 preliminary testing phase Post by: Carlton Banks on February 23, 2016, 12:34:54 AM None. I've run it at least half a dozen times now, no segfaults so far. It does scan the wallet integrity twice, the second time after finishing the tx scan. Other than that it's in good shape on this platform.
Title: Re: 0.94 preliminary testing phase Post by: bitpop on February 23, 2016, 01:55:06 AM Can't wait thank you
Will you change the donation in it to you? Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 23, 2016, 02:33:28 AM Great work! I will be testing this and will report back if anything bad happens.
Will 0.94 be just these database changes and RBF signaling? Edit: While armory was building the databases it crashed, twice now. I had renamed the old databases folder so armory create a new one for the new databases. This is the error that I got after both crashes: Code: terminate called after throwing an instance of 'LMDBException' Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 23, 2016, 04:49:03 AM Can't wait thank you Will you change the donation in it to you? Once I put out a release. Quote Will 0.94 be just these database changes and RBF signaling? My plan is to put out a testing release after this phase. I'll go over all the PRs I have received and merge them in while I wait on feedback. Once I'm done with all DB bugs and PR merging, I'll release the rc1 and take it from there. Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 23, 2016, 04:50:41 AM I'm thinking this may also have to do with low disk space on my computer, so once I clean up some junk I will try again. My DB is ~150MB on the mainnet. If you can't get it to scan mainnet, try the testnet, also build and run the C++ unit tests in the /cppForSwig/gtest folder and run those. Let me know if you can pass any of these and if you get errors, whether they are deterministic or random. Title: Re: 0.94 preliminary testing phase Post by: bitpop on February 23, 2016, 04:51:11 AM You need a bitcoin address in your profile and sig here
Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 23, 2016, 04:54:06 AM Not accepting any donations until I have an actual release out there. I'll deal with all that stuff once the release is ready.
Title: Re: 0.94 preliminary testing phase Post by: neuronics on February 23, 2016, 08:44:32 AM Not accepting any donations until I have an actual release out there. I'll deal with all that stuff once the release is ready. Respect ! Thank you for your efforts, very much appreciated. Waiting for the release (and the donation address) Title: Re: 0.94 preliminary testing phase Post by: RoadStress on February 23, 2016, 10:00:17 PM Not accepting any donations until I have an actual release out there. I'll deal with all that stuff once the release is ready. Respect ! Thank you for your efforts, very much appreciated. Waiting for the release (and the donation address) Thank you too! Words can't describe the appreciation for doing this. Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 23, 2016, 11:14:48 PM I'm thinking this may also have to do with low disk space on my computer, so once I clean up some junk I will try again. My DB is ~150MB on the mainnet. If you can't get it to scan mainnet, try the testnet, also build and run the C++ unit tests in the /cppForSwig/gtest folder and run those. Let me know if you can pass any of these and if you get errors, whether they are deterministic or random. Title: Re: 0.94 preliminary testing phase Post by: Carlton Banks on February 23, 2016, 11:24:21 PM Today's commit produces this compile error:
Code: BlockchainScanner.cpp: In lambda function: Sounds trivial to debug, but I'll leave that to those who code to live. Unless I'm doing something obviously wrong? Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 24, 2016, 01:29:31 AM Pushed db auto repair core and progress bar support. Scan and Rebuild should work too now. Fixed the build issue as well. Progress bars are a little wacky for now, I'll look into making them smoother tomorrow.
Title: Re: 0.94 preliminary testing phase Post by: droark on February 24, 2016, 01:45:17 AM Just FYI, the Mac build process is borked. I plan to do a PR with an update pretty soon.
EDIT: Should have the PR up later tonight. Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 24, 2016, 04:18:15 AM I am having some build problems that I can't figure out. My build output is on pastebin here: http://pastebin.com/QAVGGuui
Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 24, 2016, 05:09:01 AM I am having some build problems that I can't figure out. My build output is on pastebin here: http://pastebin.com/QAVGGuui Looks like you are missing entire parts of the C++ STL. Which compiler are you using? Also, is it C++11 compatible? Title: Re: 0.94 preliminary testing phase Post by: droark on February 24, 2016, 06:18:06 AM I am having some build problems that I can't figure out. My build output is on pastebin here: http://pastebin.com/QAVGGuui Same problem here. OS X. Latest Xcode (clang). Definitely C++11 compatible. (Hell, you could go up to C++14 and it wouldn't care.) EDIT: goatpig, the following code should fix the problem. Code: if (fileMap_ == MAP_FAILED) {EDIT 2: Confirmed. Armory now builds properly. Running the OS X build right now. We'll see what happens next.... Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 24, 2016, 07:49:03 AM That is some weird implementation of operator<< with stringstream on these compilers! I'll push that change and merge the OSX build PR in a minute.
Title: Re: 0.94 preliminary testing phase Post by: droark on February 24, 2016, 08:15:51 AM Okay. Having two problems with the OS X build.
1)The balance isn't quite right. It's missing a few transactions here and there. Nothing in the logs that I can see. 2)I can't send coins. Am getting the following error. Code: (ERROR) Traceback (most recent call last): Title: Re: 0.94 preliminary testing phase Post by: Carlton Banks on February 24, 2016, 01:42:21 PM Both 16a1569 and 3f02b93 are compiling and Db building OK for me. Now testing with Core 0.12.0 final also (still have a 0.11.2 copy of the blockchain that I will continue to test)
Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 24, 2016, 02:15:34 PM I am having some build problems that I can't figure out. My build output is on pastebin here: http://pastebin.com/QAVGGuui Looks like you are missing entire parts of the C++ STL. Which compiler are you using? Also, is it C++11 compatible? It looks like the problem is fixed and now I am running it for testing. Edit: I'm getting an error about invalid varint Code: -DEBUG - 1456328054: (Blockchain.cpp:213) Organizing chain Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 24, 2016, 07:17:27 PM Lots of changes, everybody will have to rebuild the DB from scratch.
Quote 2)I can't send coins. Am getting the following error. should be fixed now Quote 1)The balance isn't quite right. It's missing a few transactions here and there. Nothing in the logs that I can see. Are you getting a different balance/ledger scanning the same datadir with another version of Armory? Is there a pattern in the missing transactions (I don't do multisig refs in this db, if you have a lot of lockbox activity you will be missing ledger entries but not balance)? I updated the top block reporting mechanic, is the new db reporting the proper top block now? Quote Edit: I'm getting an error about invalid varint Does the error persist with this version? Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 24, 2016, 08:42:26 PM Pushed some more changes, make sure you pull from b24dcb2 onwards.
Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 24, 2016, 10:23:51 PM Does the error persist with this version? Pushed some more changes, make sure you pull from b24dcb2 onwards. Working well. No problems yet.Edit: I also get the same coin control problem droark mentioned in the other thread. Here is the traceback: Code: (ERROR) Traceback (most recent call last): Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 25, 2016, 08:58:42 PM Quote I also get the same coin control problem droark mentioned in the other thread Should be fixed now. Introduced a new feature to rescan balances. Let's you fix balance errors without doing a full rescan, takes less than a minute on average (it's actually <1 sec on my machine). No need to wipe the DB with this commit, just try it as is. Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 25, 2016, 09:22:37 PM Quote I also get the same coin control problem droark mentioned in the other thread Should be fixed now. Introduced a new feature to rescan balances. Let's you fix balance errors without doing a full rescan, takes less than a minute on average (it's actually <1 sec on my machine). No need to wipe the DB with this commit, just try it as is. Title: Re: 0.94 preliminary testing phase Post by: Carlton Banks on February 26, 2016, 02:08:33 PM An issue switching Core client between 0.12 and 0.11.2: I'm finding that building the Armory db with 0.11.2 initially, then restarting Armory using 0.12 (with all 0.12 validated blocks) renders Armory stuck at the last block the initial build using 0.11.2 had previously reached. Doing the same test, but with 0.12 providing Armory with the initial block data to build the db recognises the latest block when restarted using 0.11.2. Had this problem with builds previous to b24dcb2 also. Not sure what happens when doing 0.12.0 > 0.11.2 > 0.12.0 etc, haven't tried that
Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 26, 2016, 02:14:54 PM An issue switching Core client between 0.12 and 0.11.2: I'm finding that building the Armory db with 0.11.2 initially, then restarting Armory using 0.12 (with all 0.12 validated blocks) renders Armory stuck at the last block the initial build using 0.11.2 had previously reached. Doing the same test, but with 0.12 providing Armory with the initial block data to build the db recognises the latest block when restarted using 0.11.2. Had this problem with builds previous to b24dcb2 also. Not sure what happens when doing 0.12.0 > 0.11.2 > 0.12.0 etc, haven't tried that Perhaps that has something to do with bitcoin core 0.12 obfuscating the chainstate after syncs and reindexs. See the downgrade warning in the release notes.Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 26, 2016, 05:19:08 PM I'll test that directly sometimes later.
Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 26, 2016, 07:27:24 PM Added a couple changes. Maybe that will fix the 0.11.x -> 0.12 swapping.
Going over the RBF PR, it's missing some stuff so I'll have to implement that before merging in into dev. Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 26, 2016, 07:58:54 PM Added a couple changes. Maybe that will fix the 0.11.x -> 0.12 swapping. IIRC RBF is only missing the actual mempool replacement stuff and the ability to create RBF transactions.Going over the RBF PR, it's missing some stuff so I'll have to implement that before merging in into dev. For the clean up ATI stuff, do you want to remove all of the phone home stuff? In my PR, I only have it commented out because you said earlier that it may be brought back in the future. Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 26, 2016, 09:37:25 PM Added a couple changes. Maybe that will fix the 0.11.x -> 0.12 swapping. IIRC RBF is only missing the actual mempool replacement stuff and the ability to create RBF transactions.Going over the RBF PR, it's missing some stuff so I'll have to implement that before merging in into dev. RBF transactions aren't flagged in the main ledger nor the tx info dialog. No detection of RBF inheritence either. Quote For the clean up ATI stuff, do you want to remove all of the phone home stuff? In my PR, I only have it commented out because you said earlier that it may be brought back in the future. Deactivating it is fine for now. I will have to evaluate the code and its usefulness eventually. Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 26, 2016, 10:31:29 PM RBF handling is actually a lot more complicated than I expected. This is gonna take me a day.
Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 26, 2016, 11:01:28 PM Added a couple changes. Maybe that will fix the 0.11.x -> 0.12 swapping. IIRC RBF is only missing the actual mempool replacement stuff and the ability to create RBF transactions.Going over the RBF PR, it's missing some stuff so I'll have to implement that before merging in into dev. RBF transactions aren't flagged in the main ledger nor the tx info dialog. No detection of RBF inheritence either. Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 26, 2016, 11:11:56 PM Hm. I thought my PR flagged RBF transactions in the tx info dialog. It detects RBF in transaction.py, which is a file limited to creating transactions, not processing transaction history. Detection needs to take place at ledger creation (ideally on the C++ side), and it has to deal with inheritance properly, which I am working on. Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 28, 2016, 02:45:40 AM Pushed opt-in RBF detection. Regarding inheritence, I went took the simple route: any ZC of ZC is flagged as mempool replaceable, whether the chain of ZC has an RBF input somewhere or not.
If this version passes the test, I'll release the first testing build based on this state. Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 28, 2016, 02:57:11 AM Pushed opt-in RBF detection. Regarding inheritence, I went took the simple route: any ZC of ZC is flagged as mempool replaceable, whether the chain of ZC has an RBF input somewhere or not. Awesome. Will be building and testing.If this version passes the test, I'll release the first testing build based on this state. Edit: Build error: Code: g++ -Icryptopp -Imdb -DUSE_CRYPTOPP -D__STDC_LIMIT_MACROS -I/usr/include/python2.7 -I/usr/include/x86_64-linux-gnu/python2.7 -std=c++11 -O2 -pipe -fPIC -c BDM_supportClasses.cpp Edit 2: Does armoryd indicate if the transaction is RBF? Or is it just GUI right now? Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 28, 2016, 05:40:26 AM Quote Build error: fixed Quote Does armoryd indicate if the transaction is RBF? Or is it just GUI right now? only the GUI Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 28, 2016, 03:31:02 PM Quote Build error: fixed Quote Does armoryd indicate if the transaction is RBF? Or is it just GUI right now? only the GUI Also, I used to be able to double click an address in my wallet for more details but it seems that I can't do that now. Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 28, 2016, 07:13:39 PM Also, I used to be able to double click an address in my wallet for more details but it seems that I can't do that now. Fixed. Ima delay the testing build release for as long it will take me to go over the changes in the cleanupATI branch and merge it in. I'd rather figure out the due diligence before putting out binaries. Title: Re: 0.94 preliminary testing phase Post by: droark on February 28, 2016, 11:25:29 PM Edit: Build error: Interesting. I just built the latest version without any problems. What's your platform? That said, I wasn't able to send coins on the latest version. Code: Traceback (most recent call last): Had to comment out that line and the previous one. At that point, everything went through. Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 28, 2016, 11:29:51 PM Edit: Build error: Interesting. I just built the latest version without any problems. What's your platform? Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 29, 2016, 07:34:53 PM Merged in knightdk's changes around the ATI references and turning off the phone home code. Give it a spin, try to find broken dialogs (although I think I got them all). I'll be traveling tomorrow so the testing builds will have to wait for Thursday/Friday.
Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 29, 2016, 10:06:44 PM Merged in knightdk's changes around the ATI references and turning off the phone home code. Give it a spin, try to find broken dialogs (although I think I got them all). I'll be traveling tomorrow so the testing builds will have to wait for Thursday/Friday. What about the links to bitcoinarmory.com like those on the dashboard when armory is offline? What should be done about those?Edit: Also, I am unable to open the Lockboxes and the Address Book dialogs. Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 29, 2016, 11:04:07 PM Also, I am unable to open the Lockboxes and the Address Book dialogs. Fixed the lockbox issue, can't reproduce the address book one. Do you have some Python error in your terminal when that happens? Quote What about the links to bitcoinarmory.com like those on the dashboard when armory is offline? What should be done about those? Comment them out. Just PR to dev if you have minor changes. Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 29, 2016, 11:19:27 PM Also, I am unable to open the Lockboxes and the Address Book dialogs. Fixed the lockbox issue, can't reproduce the address book one. Do you have some Python error in your terminal when that happens? Code: (ERROR) Traceback (most recent call last): Quote What about the links to bitcoinarmory.com like those on the dashboard when armory is offline? What should be done about those? Comment them out. Just PR to dev if you have minor changes. Title: Re: 0.94 preliminary testing phase Post by: goatpig on February 29, 2016, 11:27:42 PM Yeah, I get this error: That's pretty severe, I should get this error too but I can't reproduce it. Anyone else running into this? Can you git stash git pull and try again? Title: Re: 0.94 preliminary testing phase Post by: achow101 on February 29, 2016, 11:32:03 PM Yeah, I get this error: That's pretty severe, I should get this error too but I can't reproduce it. Anyone else running into this? Can you git stash git pull and try again? Title: Re: 0.94 preliminary testing phase Post by: goatpig on March 01, 2016, 04:32:09 AM Same error. Nothing was stashed and nothing was pulled. The part of the code that is failing has not been changed in over 2 years. If you are getting this error now you should be getting it with other versions. Start 0.93 offline and try. |