Armory 0.95 testing phase

[droark]

Hmmm. I'm seeing something different. After one successful run on Linux (Ubuntu 16.04), I kept getting the "Error Code: 22" error on Load3Blocks_ZC_Plus3_TestLedgers and had to disable it. After that, I did see this once.

Clearly there's something off with the comparison function, or it is fed an empty pointer eventually. Is it happening at random in any test or always the same spot?

The error code? It happens consistently on Linux when valgrind is used.

Code:

[----------] 18 tests from BlockUtilsBare
[ RUN      ] BlockUtilsBare.Load5Blocks
[       OK ] BlockUtilsBare.Load5Blocks (1398 ms)
[ RUN      ] BlockUtilsBare.Load5Blocks_DamagedBlkFile
[       OK ] BlockUtilsBare.Load5Blocks_DamagedBlkFile (7102 ms)
[ RUN      ] BlockUtilsBare.Load4Blocks_Plus2
[       OK ] BlockUtilsBare.Load4Blocks_Plus2 (1251 ms)
[ RUN      ] BlockUtilsBare.Load4Blocks_ReloadBDM_ZC_Plus2
[       OK ] BlockUtilsBare.Load4Blocks_ReloadBDM_ZC_Plus2 (10502 ms)
[ RUN      ] BlockUtilsBare.Load3Blocks_ZC_Plus3_TestLedgers
Failed to create map of file. Error Code: 22 (Invalid argument)

On its own, the tests pass. Sorry. Didn't make that clear the first time.

[1714027504]

1714027504

[1714027504]

1714027504

[goatpig]

Added ZC notifications. It will give you an specific error message on transactions that failed to broadcast.
Fixed some progress bars, added one for the tx resolution. It's still a bit whacky, mainly the ETA, but the progress reporting is decent.
Added fee per byte option to the send dialog.

Also added a bunch of tentative fixes for the other issues listed here (that I sadly can't reproduce T_T). Please test away.

[Carlton Banks]

Looking good so far. Previous issues seem to have been dealt with, other than the incomplete list of addresses in Wallet Properties (coin control consistently gets the list of in-use addresses correct, however). Input-fine coin control seems good (maybe using tree layout to expand and collapse the inputs pertaining to a given address would make long lists of inputs more manageable). Satoshi/byte seems to work Ok. Progress bar overhaul works well (functionally and visually). Provokes the same Gtk "object drawn out of bounds" errors we've seen in the past, so if it could be fixed then, presumably a new fix exists in this instance (it's never been more than a concerning looking warning in the logs anyhow).

[achow101]

This is strange. If I use the shortcut I made to launch Armory, it doesn't work.

This is the error from the log:

Code:

2016-09-03 01:02 (INFO) -- ArmoryUtils.py:639 - Executing popen: ['./ArmoryDB', '--db-type="DB_FULL"', '--spawnId="5uWPvi3P2wvYyo2xNdpeWKU4spxWvHuDRuCJ9gfxevYd"', '--satoshi-datadir="/media/andy/Data/Programs/Bitcoin/data/blocks"', '--dbdir="/media/andy/Data/Data/bitcoin/ArmoryData/databases"']
2016-09-03 01:02 (ERROR) -- Traceback (most recent call last):
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 6631, in method_signal
    method()
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 6668, in completeBlockchainProcessingInitialization
    gotDB = self.startArmoryDBIfNecessary()
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 2198, in startArmoryDBIfNecessary
    spawnId = TheSDM.spawnDB(TheBDM.armoryDBDir)
  File "/home/andy/bitcoin/BitcoinArmory/SDM.py", line 490, in spawnDB
    launchProcess(pargs, **kargs)
  File "/home/andy/bitcoin/BitcoinArmory/armoryengine/ArmoryUtils.py", line 642, in launchProcess
    return Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE, *args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

However if I run the same command from the terminal, I have no problems whatsoever.

I am running Ubuntu 16.04. This is probably an issue with my system, but I think it is something to note.

[droark]

ArmoryDB crash still happening on OSX, even with the new code. Will double check to see if it's in the same location but I'd imagine this is the case.

[josephbisch]

This is strange. If I use the shortcut I made to launch Armory, it doesn't work.

This is the error from the log:

Code:

2016-09-03 01:02 (INFO) -- ArmoryUtils.py:639 - Executing popen: ['./ArmoryDB', '--db-type="DB_FULL"', '--spawnId="5uWPvi3P2wvYyo2xNdpeWKU4spxWvHuDRuCJ9gfxevYd"', '--satoshi-datadir="/media/andy/Data/Programs/Bitcoin/data/blocks"', '--dbdir="/media/andy/Data/Data/bitcoin/ArmoryData/databases"']
2016-09-03 01:02 (ERROR) -- Traceback (most recent call last):
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 6631, in method_signal
    method()
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 6668, in completeBlockchainProcessingInitialization
    gotDB = self.startArmoryDBIfNecessary()
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 2198, in startArmoryDBIfNecessary
    spawnId = TheSDM.spawnDB(TheBDM.armoryDBDir)
  File "/home/andy/bitcoin/BitcoinArmory/SDM.py", line 490, in spawnDB
    launchProcess(pargs, **kargs)
  File "/home/andy/bitcoin/BitcoinArmory/armoryengine/ArmoryUtils.py", line 642, in launchProcess
    return Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE, *args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

However if I run the same command from the terminal, I have no problems whatsoever.

I am running Ubuntu 16.04. This is probably an issue with my system, but I think it is something to note.

Armory is trying to open ArmoryDB as a relative path. That only works if you are have used cd to enter the Armory source tree first. The solution is for Armory to use absolute paths, so people don't need to be in the same directory as Armory to run it.

[achow101]

This is strange. If I use the shortcut I made to launch Armory, it doesn't work.

This is the error from the log:

Code:

2016-09-03 01:02 (INFO) -- ArmoryUtils.py:639 - Executing popen: ['./ArmoryDB', '--db-type="DB_FULL"', '--spawnId="5uWPvi3P2wvYyo2xNdpeWKU4spxWvHuDRuCJ9gfxevYd"', '--satoshi-datadir="/media/andy/Data/Programs/Bitcoin/data/blocks"', '--dbdir="/media/andy/Data/Data/bitcoin/ArmoryData/databases"']
2016-09-03 01:02 (ERROR) -- Traceback (most recent call last):
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 6631, in method_signal
    method()
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 6668, in completeBlockchainProcessingInitialization
    gotDB = self.startArmoryDBIfNecessary()
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 2198, in startArmoryDBIfNecessary
    spawnId = TheSDM.spawnDB(TheBDM.armoryDBDir)
  File "/home/andy/bitcoin/BitcoinArmory/SDM.py", line 490, in spawnDB
    launchProcess(pargs, **kargs)
  File "/home/andy/bitcoin/BitcoinArmory/armoryengine/ArmoryUtils.py", line 642, in launchProcess
    return Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE, *args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

However if I run the same command from the terminal, I have no problems whatsoever.

I am running Ubuntu 16.04. This is probably an issue with my system, but I think it is something to note.

Armory is trying to open ArmoryDB as a relative path. That only works if you are have used cd to enter the Armory source tree first. The solution is for Armory to use absolute paths, so people don't need to be in the same directory as Armory to run it.

So this update breaks shortcuts right now. That is a bit of an issue.

[josephbisch]

This is strange. If I use the shortcut I made to launch Armory, it doesn't work.

This is the error from the log:

Code:

2016-09-03 01:02 (INFO) -- ArmoryUtils.py:639 - Executing popen: ['./ArmoryDB', '--db-type="DB_FULL"', '--spawnId="5uWPvi3P2wvYyo2xNdpeWKU4spxWvHuDRuCJ9gfxevYd"', '--satoshi-datadir="/media/andy/Data/Programs/Bitcoin/data/blocks"', '--dbdir="/media/andy/Data/Data/bitcoin/ArmoryData/databases"']
2016-09-03 01:02 (ERROR) -- Traceback (most recent call last):
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 6631, in method_signal
    method()
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 6668, in completeBlockchainProcessingInitialization
    gotDB = self.startArmoryDBIfNecessary()
  File "/home/andy/bitcoin/BitcoinArmory/ArmoryQt.py", line 2198, in startArmoryDBIfNecessary
    spawnId = TheSDM.spawnDB(TheBDM.armoryDBDir)
  File "/home/andy/bitcoin/BitcoinArmory/SDM.py", line 490, in spawnDB
    launchProcess(pargs, **kargs)
  File "/home/andy/bitcoin/BitcoinArmory/armoryengine/ArmoryUtils.py", line 642, in launchProcess
    return Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE, *args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

However if I run the same command from the terminal, I have no problems whatsoever.

I am running Ubuntu 16.04. This is probably an issue with my system, but I think it is something to note.

Armory is trying to open ArmoryDB as a relative path. That only works if you are have used cd to enter the Armory source tree first. The solution is for Armory to use absolute paths, so people don't need to be in the same directory as Armory to run it.

So this update breaks shortcuts right now. That is a bit of an issue.

It breaks shortcuts or any use case where someone is trying to run Armory from somewhere other than the Armory source tree. So if they are above a level and try to run "python BitcoinArmory/ArmoryQt.py" that should fail too.

A solution is to modify self.dbExecutable in SDM.py to be prepended by the following path.

Code:

path = os.path.dirname(os.path.abspath(__file__))
self.dbExecutable = os.path.join(path, 'ArmoryDB')
if OS_WINDOWS:
    # add .exe to self.dbExecutable

[goatpig]

A solution is to modify self.dbExecutable in SDM.py to be prepended by the following path.

Code:

path = os.path.dirname(os.path.abspath(__file__))
self.dbExecutable = os.path.join(path, 'ArmoryDB')
if OS_WINDOWS:
    # add .exe to self.dbExecutable

Done. You should just PR these kind of quick fixes in the future.

Provokes the same Gtk "object drawn out of bounds" errors we've seen in the past, so if it could be fixed then, presumably a new fix exists in this instance (it's never been more than a concerning looking warning in the logs anyhow).

I have no idea what the underlying issue actually is, besides that it triggers with older version of PyQt4. I can see it on my Debian7 VM, I don't think I can on Debian 8. What version are you using Qt are you using?

[Carlton Banks]

Provokes the same Gtk "object drawn out of bounds" errors we've seen in the past, so if it could be fixed then, presumably a new fix exists in this instance (it's never been more than a concerning looking warning in the logs anyhow).

I have no idea what the underlying issue actually is, besides that it triggers with older version of PyQt4. I can see it on my Debian7 VM, I don't think I can on Debian 8. What version are you using Qt are you using?

Version 4.8.6, Debian 8.5. Pretty sure that's the most up to date version in the Debian 8 standard repo.

[goatpig]

knightdk submitted a PR that makes use of new cookie auth feature from Core 0.12 to setup an RPC socket with the local node in case there is no password setup in bitcoin.conf,

The means Armory can now setup an RPC connection to your node regardless of auto bitcoind management. The bottom line, most evident effect is that it should display your node's calculated minimum acceptable fee per byte when selecting that fee option in the send dialog (as opposed to a plain 0).

This feature will only work if your client has access to a local Core instance. The feature is disabled for remote clients. This is because this feature was done through the path of least resistance, i.e. leveraging the existing RPC code in the Python client.

I will leverage the full benefits of this change as well as make it available to remote clients once I migrate this code to the C++ DB. For now, this is an added bonus to "default" users. On a side note, this should also work on OSX (once I get the db rolling on Macs that is)

Please test this feature thoroughly with both self managed and auto bitcoind. I didn't have time to test it directly yet so I'm hoping you guys can list the issues before hand.

-----

droak: I've managed to reproduce the error code 22 issue with valgrind and the new test. I've split a stress test that doesn't fail on valgrind. Please build the unit tests then run DB1kIterTest with valgrind massif. This shouldn't fail. I'll be looking at error 22 in the mean time.

[goatpig]

Looking good so far. Previous issues seem to have been dealt with, other than the incomplete list of addresses in Wallet Properties (coin control consistently gets the list of in-use addresses correct, however). Input-fine coin control seems good (maybe using tree layout to expand and collapse the inputs pertaining to a given address would make long lists of inputs more manageable). Satoshi/byte seems to work Ok. Progress bar overhaul works well (functionally and visually). Provokes the same Gtk "object drawn out of bounds" errors we've seen in the past, so if it could be fixed then, presumably a new fix exists in this instance (it's never been more than a concerning looking warning in the logs anyhow).

I was going for the cheapest GUI implementation possible. If someone feels like beefing up the GUI in a PR, by all means be my guest. Otherwise I'll start taking suggestions once the testing builds are out.

[droark]

droak: I've managed to reproduce the error code 22 issue with valgrind and the new test. I've split a stress test that doesn't fail on valgrind. Please build the unit tests then run DB1kIterTest with valgrind massif. This shouldn't fail. I'll be looking at error 22 in the mean time.

Thanks. Had to write a patch to get everything to compile. Anyway, when running valgrind w/ massif, it's slow as hell. I suspect this one will be running for awhile. I'll report back when it's done.

Also, just to confirm, I'll still getting the ArmoryDB crash. Haven't run the other tests yet.

[goatpig]

droak: I've managed to reproduce the error code 22 issue with valgrind and the new test. I've split a stress test that doesn't fail on valgrind. Please build the unit tests then run DB1kIterTest with valgrind massif. This shouldn't fail. I'll be looking at error 22 in the mean time.

Thanks. Had to write a patch to get everything to compile. Anyway, when running valgrind w/ massif, it's slow as hell. I suspect this one will be running for awhile. I'll report back when it's done.

Also, just to confirm, I'll still getting the ArmoryDB crash. Haven't run the other tests yet.

Latest commit has a fix for error code 22.

[alomar]

is Armory compatible with Bitcore from Bitpay?

This should be a thread in it's own right, first off.

I'm not sure that Bitcore/Armory has ever been tried. Bitcore is in many ways just a re-implementation of Bitcoin Core as far as I'm aware, but there aren't particularly compelling reasons to use it. Although I've not heard of any recent changes to the development direction that might somehow make it attractive for use with Armory? (and not your own personal convenience).

the only reason it would be nice is b/c Trezor works with Bitcore.  if Armory did as well, we could have 2 forms of cold storage options with one Bitcore database.

[Carlton Banks]

Ah, that would be a legitimate convenience. Seems no-one's tried judging by the response, my guess would be it's likely Armory wouldn't be able to read the Bitcore database.

[alomar]

Ah, that would be a legitimate convenience. Seems no-one's tried judging by the response, my guess would be it's likely Armory wouldn't be able to read the Bitcore database.

it could come down to which is easier for goatpig to code; compatibility with Trezor or compatibility with Bitcore.

[droark]

Latest commit has a fix for error code 22.

I'm getting a consistent segfault on BlockDir.HeadersFirst when running on Linux. OS X passes. However, OS X chokes on BlockUtilsBare.FCGIStack and just seems to be stuck, doing nothing for hours.

[goatpig]

Latest commit has a fix for error code 22.

I'm getting a consistent segfault on BlockDir.HeadersFirst when running on Linux. OS X passes. However, OS X chokes on BlockUtilsBare.FCGIStack and just seems to be stuck, doing nothing for hours.

Should be a little better with the current push. Still getting the occasional test hanging on valgrind, will figure that out soon.

Ah, that would be a legitimate convenience. Seems no-one's tried judging by the response, my guess would be it's likely Armory wouldn't be able to read the Bitcore database.

it could come down to which is easier for goatpig to code; compatibility with Trezor or compatibility with Bitcore.

Once I implement blocks over p2p, the DB will be agnostic to the underlying node. It will be able to talk to any implementation as long as it gets the p2p layer right.

[Carlton Banks]

Ah, that would be a legitimate convenience. Seems no-one's tried judging by the response, my guess would be it's likely Armory wouldn't be able to read the Bitcore database.

it could come down to which is easier for goatpig to code; compatibility with Trezor or compatibility with Bitcore.

Not really. I've seen the way the Bitpay organisation behaves, and don't like them. Not using their software, don't trust them.