Latest commit has a fix for error code 22.
I'm getting a consistent segfault on BlockDir.HeadersFirst when running on Linux. OS X passes. However, OS X chokes on BlockUtilsBare.FCGIStack and just seems to be stuck, doing nothing for hours. Slowing zero-ing in on the underlying bug. As soon as I have it isolated it should be easy to correct. I hope to have something tomorrow. |
|
|
|
Latest commit has a fix for error code 22.
I'm getting a consistent segfault on BlockDir.HeadersFirst when running on Linux. OS X passes. However, OS X chokes on BlockUtilsBare.FCGIStack and just seems to be stuck, doing nothing for hours. Should be a little better with the current push. Still getting the occasional test hanging on valgrind, will figure that out soon. Ah, that would be a legitimate convenience. Seems no-one's tried judging by the response, my guess would be it's likely Armory wouldn't be able to read the Bitcore database.
it could come down to which is easier for goatpig to code; compatibility with Trezor or compatibility with Bitcore. Once I implement blocks over p2p, the DB will be agnostic to the underlying node. It will be able to talk to any implementation as long as it gets the p2p layer right. |
|
|
|
Why Tails or Whonix? Running Bitcoin Core (which is needed for Armory) in either Tails or Whonix is a pain in the ass - the blockchain is simply too big.
I'd say the best option is to run a standalone Bitcoin Core (not managed by Armory) over Tor - you just need the Tor Browser Bundle to do that.
In Bitcoin Core: go to Preferences/Network and check "connect through SOCKS5 Proxy". Proxy IP: 127.0.0.1, Port: 9150 - and then have Tor Browser Bundle open together with Bitcoin Core.
After the above, in Armory just check check "enable settings for proxies/Tor" in Privacy Settings.
The issue with connecting to a anon/private network through a proxy is that any traffic that doesn't go through the proxy can reveal you. As long you use something like the Tor bundle on top of a regular system, that remains a possibility. The main offenders in these setups are DNS lookups: regular traffic will go through the proxy, but your machine will be sending domains in plain text to your ISP for resolution. It is fairly hard to sanitize an application/system to avoid this kind of (otherwise) routine and harmless side channel traffic. With an OS like Tails or Whonix, that problem goes out the way. The other point is your OS spying on you. That's a given for Windows so let's not touch on this topic here. For Unix flavors, while the purpose isn't actually spying, there is a wealth of phone home code that will ignore the Tor proxy. From my perspective it's easier to keep Bitcoin and its blockchain data in an encrypted removable disk for using on Tails than to try and sanitize my regular Debian install. Also, AFAIK Tails is the live Tor OS whereas Whonix is the perma install one. It would be an even easier setup with Whonix. |
|
|
|
|
Let's see some log files first.
|
|
|
|
droak: I've managed to reproduce the error code 22 issue with valgrind and the new test. I've split a stress test that doesn't fail on valgrind. Please build the unit tests then run DB1kIterTest with valgrind massif. This shouldn't fail. I'll be looking at error 22 in the mean time.
Thanks. Had to write a patch to get everything to compile.  Anyway, when running valgrind w/ massif, it's slow as hell. I suspect this one will be running for awhile. I'll report back when it's done. Also, just to confirm, I'll still getting the ArmoryDB crash. Haven't run the other tests yet. Latest commit has a fix for error code 22. |
|
|
|
Looking good so far. Previous issues seem to have been dealt with, other than the incomplete list of addresses in Wallet Properties (coin control consistently gets the list of in-use addresses correct, however). Input-fine coin control seems good (maybe using tree layout to expand and collapse the inputs pertaining to a given address would make long lists of inputs more manageable). Satoshi/byte seems to work Ok. Progress bar overhaul works well (functionally and visually). Provokes the same Gtk "object drawn out of bounds" errors we've seen in the past, so if it could be fixed then, presumably a new fix exists in this instance (it's never been more than a concerning looking warning in the logs anyhow).
I was going for the cheapest GUI implementation possible. If someone feels like beefing up the GUI in a PR, by all means be my guest. Otherwise I'll start taking suggestions once the testing builds are out. |
|
|
|
|
knightdk submitted a PR that makes use of new cookie auth feature from Core 0.12 to setup an RPC socket with the local node in case there is no password setup in bitcoin.conf,
The means Armory can now setup an RPC connection to your node regardless of auto bitcoind management. The bottom line, most evident effect is that it should display your node's calculated minimum acceptable fee per byte when selecting that fee option in the send dialog (as opposed to a plain 0).
This feature will only work if your client has access to a local Core instance. The feature is disabled for remote clients. This is because this feature was done through the path of least resistance, i.e. leveraging the existing RPC code in the Python client.
I will leverage the full benefits of this change as well as make it available to remote clients once I migrate this code to the C++ DB. For now, this is an added bonus to "default" users. On a side note, this should also work on OSX (once I get the db rolling on Macs that is)
Please test this feature thoroughly with both self managed and auto bitcoind. I didn't have time to test it directly yet so I'm hoping you guys can list the issues before hand.
-----
droak: I've managed to reproduce the error code 22 issue with valgrind and the new test. I've split a stress test that doesn't fail on valgrind. Please build the unit tests then run DB1kIterTest with valgrind massif. This shouldn't fail. I'll be looking at error 22 in the mean time.
|
|
|
|
|
How much free space is on your disk?
|
|
|
|
2016-09-01 00:10:32 ERROR: ReadBlockFromDisk: Errors in block header at CBlockDiskPos(nFile=594, nPos=128428074)
2016-09-01 00:10:32 *** Failed to read block
In your bitcoin /databases folder, you will have to delete all blkXXXXX.dat and revXXXXX.dat files from 594 onward. Then start BitcoinQt and let it redownload what's missing (it will recheck the whole chain). |
|
|
|
A solution is to modify self.dbExecutable in SDM.py to be prepended by the following path. path = os.path.dirname(os.path.abspath(__file__))
self.dbExecutable = os.path.join(path, 'ArmoryDB')
if OS_WINDOWS:
# add .exe to self.dbExecutable
Done. You should just PR these kind of quick fixes in the future. Provokes the same Gtk "object drawn out of bounds" errors we've seen in the past, so if it could be fixed then, presumably a new fix exists in this instance (it's never been more than a concerning looking warning in the logs anyhow).
I have no idea what the underlying issue actually is, besides that it triggers with older version of PyQt4. I can see it on my Debian7 VM, I don't think I can on Debian 8. What version are you using Qt are you using? |
|
|
|
|
Added ZC notifications. It will give you an specific error message on transactions that failed to broadcast.
Fixed some progress bars, added one for the tx resolution. It's still a bit whacky, mainly the ETA, but the progress reporting is decent.
Added fee per byte option to the send dialog.
Also added a bunch of tentative fixes for the other issues listed here (that I sadly can't reproduce T_T). Please test away.
|
|
|
|
Hmmm. I'm seeing something different. After one successful run on Linux (Ubuntu 16.04), I kept getting the "Error Code: 22" error on Load3Blocks_ZC_Plus3_TestLedgers and had to disable it. After that, I did see this once.
Clearly there's something off with the comparison function, or it is fed an empty pointer eventually. Is it happening at random in any test or always the same spot? Also, I forgot to run ContainerTests. No crashes, at least not yet. First is the normal version, second is the Valgrind version, which is still running after almost an hour. (I'll update it if it finishes anytime soon.) This is on Linux. I'll update if OSX has significant differences.
All those are benign or false positives, nothing alarming here at least EDIT: Interestingly, on OSX, ContainerTests passes every time, yet when running under valgrind, BlockingStackTest_Concurrent seems to get stuck, just like on Linux.
Yep, expected, forgot to rework the BlockingStack clean up method, which will end up hanging on heavy loads. It's only used when shutting down the DB, so I'm not in a hurry to fix it yet. |
|
|
|
[ RUN ] BlockDir.HeadersFirst
--6464-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option
--6464-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 2 times)
--6464-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 4 times)
--6464-- UNKNOWN mach_msg unhandled MACH_SEND_TRAILER option (repeated 8 times)
[ OK ] BlockDir.HeadersFirst (21467 ms)
Never seen these. Sounds like OSX specific, not particularly concerned if they don't result in a segfault. Failed to create map of file. Error Code: 22
Can happen sometimes when the OS doesn't create file handles fast enough. Each test completely wipes the db. On some rare occasions (like a system on full load), lmdb will create the db file and try to access it after the call returns but before the OS actually flushes the file to the system (which is a very rare and unexpected window to hit). Not a concern per se, file creation only takes place with empty db folders. Anyway, for some reason, valgrind tripped up here and caused some Armory error code to be triggered (or maybe valgrind just caught the error?). This is actually code that was put in place 6 months ago in order for Armory to compile properly after some other changes. Might be worth looking here too, even though I never see the message when running ArmoryDB. [ RUN ] BlockUtilsBare.Load3Blocks_ZC_Plus3_TestLedgers
==6785==
==6785== Process terminating with default action of signal 10 (SIGBUS)
==6785== Non-existent physical address at address 0x1027D6000
==6785== at 0x101935DA0: _platform_memcmp (in /usr/lib/system/libsystem_platform.dylib)
==6785== by 0x100107213: BinaryDataRef::operator==(BinaryData const&) const (BinaryData.h:891)
==6785== by 0x1000F97DC: BinaryDataRef::operator!=(BinaryData const&) const (BinaryData.h:901)
==6785== by 0x1002E7C46: DatabaseBuilder::parseBlockFile(unsigned char const*, unsigned long, unsigned long, std::__1::function<bool (unsigned char const*, unsigned long, unsigned long)>) (DatabaseBuilder.cpp:381)
==6785== by 0x1002E67BD: DatabaseBuilder::addBlocksToDB(BlockDataLoader&, unsigned short, unsigned long, std::__1::shared_ptr<BlockOffset>) (DatabaseBuilder.cpp:327)
==6785== by 0x1002E62EA: DatabaseBuilder::updateBlocksInDB(std::__1::function<void (BDMPhase, double, unsigned int, unsigned int)> const&, bool, bool)::$_2::operator()(unsigned short, unsigned long, std::__1::shared_ptr<BlockOffset>, bool) const (DatabaseBuilder.cpp:217)
==6785== by 0x1002E4C06: DatabaseBuilder::updateBlocksInDB(std::__1::function<void (BDMPhase, double, unsigned int, unsigned int)> const&, bool, bool) (DatabaseBuilder.cpp:257)
==6785== by 0x1002E8586: DatabaseBuilder::update() (DatabaseBuilder.cpp:473)
==6785== by 0x10013C241: BlockDataManager::readBlkFileUpdate(BlockDataManager::BlkFileUpdateCallbacks const&) (BlockUtils.cpp:1559)
==6785== by 0x1001F160D: BlockDataManagerThread::run()::$_4::operator()() const (BDM_mainthread.cpp:178)
==6785== by 0x1001F0EB2: BlockDataManagerThread::run() (BDM_mainthread.cpp:236)
==6785== by 0x1001F04AC: BlockDataManagerThread::thrun(void*) (BDM_mainthread.cpp:261)
--6785:0:schedule VG_(sema_down): read returned -4
==6785==
Killed: 9
Seeing this consistently after one successful run. The invalid address varies slightly but it's always around this particular value. Definitely not good. Now that's something I can't reproduce on my end. My valgrind tests all pass 100%. Can you try reproducing this on a linux setup too? Definitely worth looking into. |
|
|
|
|
Something is off with your setup. You should post your debug.log on the technical support forum.
|
|
|
|
Update:
Day 2
Woke up again to see the message "Disk space low", however, I am seriously confused. The blockchain is stored on my D:/ drive which has a couple hundred Gigabytes free. The only "disk" that is low in space right now in disk usage analyzer is my Ubuntu partition which has three gigabytes left. However, for some reason I got the notification that my disk has only 500 mb left. What's the low disk space?
Does Bitcoin-qt use any other drives for settings or other storage unrelated to the blockchain?
Core only sticks to the -datadir you give it. You should make sure this is the path it is actually writing to. |
|
|
|
You're welcome. Still crashes, unfortunately. Just to consolidate all the info, 0.94.1 on OSX doesn't crash, and these files don't crash on Linux at all. (I copied the testnet blockchain to Linux, and it was fine.)
Try the unit tests in the gtest folder. Both test binaries should pass valgrind --tool=massif |
|
|
|
I think most of the bugs and instability issues reported here should be fixed now. Testers, please pull dev and try again. DB format changed slightly, please start with a fresh folder.
Still seeing testnet crashes w/ ArmoryDB on OS X. Latest code. Same problem as before. Can you: 1) upload the incriminated block file 2) try to sync against another copy of the testnet Thanks for the file. Can you build a DB from scratch using --db-type=DB_BARE? |
|
|
|
Sounds like you are syncing a fresh blockchain. Did you delete the .bitcoin folder in the process of updating Core, or were you using a custom folder previously?
No, I did the full factory reset Yeah that will nuke your blockchain data, you didn't have to go that far. Thanks, I got it to work at last.
Does anybody know the current size of the data to be downloaded? I have downloaded ~30 gigs already in a matter of a few hours.
In the 80GB vicinity |
|
|
|
I think most of the bugs and instability issues reported here should be fixed now. Testers, please pull dev and try again. DB format changed slightly, please start with a fresh folder.
Still getting tons of these errors: Traceback (most recent call last):
File "ArmoryQt.py", line 5771, in handleCppNotification
self.updateWalletData()
File "ArmoryQt.py", line 5735, in updateWalletData
self.walletMap[wltid].getAddrDataFromDB()
File "/home/andy/bitcoin/BitcoinArmory/armoryengine/PyBtcWallet.py", line 52, in inner
return func(*args, **kwargs)
File "/home/andy/bitcoin/BitcoinArmory/armoryengine/PyBtcWallet.py", line 3119, in getAddrDataFromDB
countList = self.cppWallet.getAddrTxnCountsFromDB()
File "/home/andy/bitcoin/BitcoinArmory/CppBlockUtils.py", line 2354, in getAddrTxnCountsFromDB
return _CppBlockUtils.BtcWallet_getAddrTxnCountsFromDB(self)
<class 'CppBlockUtils.DbErrorMsg'>: <CppBlockUtils.DbErrorMsg; proxy of <Swig Object of type 'DbErrorMsg *' at 0x7f485abb8cf0> >
and it seems that it is stuck at "Scanning Transaction History 100% 1 second". ArmoryDb process still detaches itself from the parent process after Quit (i.e. survives when it should die). Both issues should be fixed now. Upgraded coin control to display utxos, give that a spin as well. Will work on ZC and progress bars next. |
|
|
|
I think most of the bugs and instability issues reported here should be fixed now. Testers, please pull dev and try again. DB format changed slightly, please start with a fresh folder.
Still seeing testnet crashes w/ ArmoryDB on OS X. Latest code. Same problem as before. Can you: 1) upload the incriminated block file 2) try to sync against another copy of the testnet |
|
|
|
|
Show Posts
