ArmoryQt on Windows still broken with remote ArmoryDB on Ubuntu?

[skyhawk]

Sorry to dredge this up from the past, but I've found questions relating to this with no real answers....

I'm trying to run ArmoryQt on win10-64, with ArmoryDB and bitcoind running on an Ubuntu VM -- Using armoryDB remotely is a new configuration for me, so I don't know if this issue is related to using 0.96.0.3-testing

ArmoryQt is running from the following command-line

Code:

@runas /savecred /user:<redacted> "D:\Storage\<redacted>\armory\program\ArmoryQt.exe --datadir=D:\Storage\<redacted>\armory\data --armorydb-ip=<redacted> --armorydb-port=9002"

ArmoryQt shows the splash screen, gets to 100%, then pops up "Errors occurred" with a reference to ArmoryQt.exe.log
I think the relevant bits from the logfile are:

Code:

2017-06-24 18:36:10 (INFO) -- ArmoryQt.py:1820 - startBitcoindIfNecessary
2017-06-24 18:36:10 (WARNING) -- ArmoryQt.py:1810 - DB is already running
2017-06-24 18:36:10 (INFO) -- ArmoryQt.py:1908 - Setting netmode: 1
2017-06-24 18:36:10 (ERROR) -- Traceback (most recent call last):
  File "ArmoryQt.py", line 5958, in <module>
  File "ArmoryQt.py", line 679, in __init__
  File "ArmoryQt.py", line 5726, in startBlockchainProcessingInitialization
  File "ArmoryQt.py", line 5748, in completeBlockchainProcessingInitialization
  File "armoryengine\BDM.pyc", line 189, in instantiateBDV
TypeError: in method 'BlockDataViewer_getNewBDV', argument 1 of type 'string const &'

ArmoryDB is running with just the binary (I fished it out of the Ubuntu package because I don't want to install 500MB of dependencies for it), with the following command:

Code:

./ArmoryDB --satoshi-datadir="/data/bitcoind" --datadir="/data-ssd/armorydb"

ArmoryDB does not react at all -- no log entries when I launch ArmoryQt. It does appear to be fully synced up and ready to go.

Code:

logging in /data-ssd/armorydb/dbLog.txt
-INFO  - 19:04:17: (main.cpp:29) Running on 4 threads
-INFO  - 19:04:17: (main.cpp:30) Ram usage level: 4
-INFO  - 19:04:17: (BlockUtils.cpp:907) blkfile dir: /data/bitcoind/blocks
-INFO  - 19:04:17: (BlockUtils.cpp:908) lmdb dir: /data-ssd/armorydb/databases
-INFO  - 19:04:17: (lmdb_wrapper.cpp:388) Opening databases...
-INFO  - 19:04:17: (BDM_Server.h:248) Listening on port 9001
-ERROR - 19:04:17: (nodeRPC.cpp:150) unexpected cookie file content
-ERROR - 19:04:17: (nodeRPC.cpp:150) unexpected cookie file content
-INFO  - 19:04:17: (BlockUtils.cpp:1091) Executing: doInitialSyncOnLoad
-INFO  - 19:04:17: (BitcoinP2P.cpp:947) Connected to Bitcoin node
-INFO  - 19:04:17: (DatabaseBuilder.cpp:169) Reading headers from db
-INFO  - 19:04:19: (DatabaseBuilder.cpp:208) Found 472811 headers in db
-INFO  - 19:04:20: (DatabaseBuilder.cpp:51) updating HEADERS db
-INFO  - 19:04:21: (DatabaseBuilder.cpp:268) parsed block file #914
-INFO  - 19:04:21: (DatabaseBuilder.cpp:268) parsed block file #915
-DEBUG - 19:04:21: (Blockchain.cpp:242) Organizing chain
-INFO  - 19:04:21: (DatabaseBuilder.cpp:56) updated HEADERS db in 1.01823s
-INFO  - 19:04:21: (BDM_supportClasses.cpp:1841) Enabling zero-conf tracking
-WARN  - 19:04:25: (BDM_supportClasses.cpp:1898) running 10 zc parser threads
-WARN  - 19:04:25: (BDM_supportClasses.cpp:1898) running 15 zc parser threads
-WARN  - 19:04:25: (BDM_supportClasses.cpp:1898) running 20 zc parser threads
-WARN  - 19:05:17: (BDM_supportClasses.cpp:1898) running 25 zc parser threads
-WARN  - 19:05:17: (BDM_supportClasses.cpp:1898) running 30 zc parser threads

nginx is running on ubuntu, with the following config:

Code:

user www-data;
worker_processes auto;
pid /run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    keepalive_timeout  0;

    server {
        listen       9002;
        server_name  localhost;

        location / {
            root           /;
            fastcgi_pass   127.0.0.1:9001;
            fastcgi_index  /;
            fastcgi_buffering on;
            fastcgi_buffer_size 4k;
            fastcgi_buffers 16 4k;
            fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
            include        fastcgi_params;
        }
    }
}

I can connect to http://<redacted>:9002/ and I get a hex value returned

Sorry for the long post, is there any more debug information I can get you, and do you have any idea what's going wrong with this setup?

[goatpig]

Code:

fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
include        fastcgi_params;

Does the file with the fcgi_params exist?

[skyhawk]

Part of the default nginx installation, /etc/nginx/fastcgi_params

[skyhawk]

Most interesting -- running Wireshark I'm not seeing any connection from ArmoryQt at all. When I access the ArmoryDB URL in a browser Wireshark sees it

[skyhawk]

I've tried going back to 0.96 -- same result
I've also tried disabling Windows Firewall -- same result

ArmoryQt doesn't even appear to be trying to connect to the remote ArmoryDB -- At this point I think I'm doing a stupid, but I can't figure out what that might be...

[goatpig]

Code:

--armorydb-port=9002

Try and put the port in quotes, it should be a string, not an integer. It's possible the arg parser sees it as an integer instead (Python is garbage after all).

[skyhawk]

No change. Armory does appear to be getting the options correctly

Running the following command:

Code:

runas /savecred /user:<redacted> "D:\Storage\<redacted>\armory-core\program\ArmoryQt.exe --datadir=D:\Storage\<redacted>\armory-core\data --armorydb-ip=\"192.168.192.217\" --armorydb-port=\"9002\""

Code:

2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1280 - Named options/arguments to armoryengine.py:
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     thread_count    : -1
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     rescan          : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     ignoreAllZC     : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     rescanBalance   : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     disableModules  : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     port            : None
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     interport       : 8223
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     coverageOutputDir: None
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     forceWalletCheck: False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     regtest         : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     rebuild         : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     nettimeout      : 2
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     datadir         : D:\Storage\<redacted>\armory-core\data
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     clearMempool    : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     offline         : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     armoryDBDir     : DEFAULT
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     armorydb_port   : 9002
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     satoshiPort     : DEFAULT
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     useTorSettings  : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     netlog          : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     keypool         : 100
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     coverageInclude : None
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     forceOnline     : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     redownload      : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     rpcBindAddr     : 127.0.0.1
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     armorydb_ip     : 192.168.192.217
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     multisigFile    : DEFAULT
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     ram_usage       : -1
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     mtdebug         : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     logDisable      : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     settingsPath    : D:\Storage\<redacted>\armory-core\data\ArmorySettings.txt
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     language        : en
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     db_type         : DB_FULL
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     doDebug         : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     enableDetSign   : True
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     disableConfPermis: False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     testnet         : False
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     rpcport         : DEFAULT
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     satoshiHome     : DEFAULT
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     satoshiRpcport  : DEFAULT
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     logFile         : D:\Storage\<redacted>\armory-core\data\ArmoryQt.exe.log.txt
2017-06-24 21:19:07 (INFO) -- ArmoryUtils.pyc:1282 -     verbosity       : None

Is it possible to specify db_ip and db_port in a config file as opposed to the command line?

I'm going to experiment with running Armory outside of my little sandbox for it... maybe it's something to do with how runas is parsing things?

EDIT: No change when running without runas

[goatpig]

Is it possible to specify db_ip and db_port in a config file as opposed to the command line?

You can put any cli arg in armoryqt.conf (for the client) and armorydb.conf (for the db). These have to be located in the respective binary's datadir to be parsed.

Your latest log doesn't even show a traceback, that's something at least. I know this stuff works because I have users running Windows clients vs Linux DBs.

Can you build the code on Windows? Placing some key print statements could go a long way figuring out what's going on.

[skyhawk]

I apologize for any confusion regarding logging -- ArmoryQt has been consistently producing the following:

Code:

(ERROR) ArmoryUtils.pyc:3744 - Unsupported language  specified. Defaulting to English (en)
(WARNING) ArmoryQt.py:1810 - DB is already running
(ERROR) Traceback (most recent call last):
  File "ArmoryQt.py", line 5958, in <module>
  File "ArmoryQt.py", line 679, in __init__
  File "ArmoryQt.py", line 5726, in startBlockchainProcessingInitialization
  File "ArmoryQt.py", line 5748, in completeBlockchainProcessingInitialization
  File "armoryengine\BDM.pyc", line 189, in instantiateBDV
TypeError: in method 'BlockDataViewer_getNewBDV', argument 1 of type 'string const &'

Traceback (most recent call last):
  File "ArmoryQt.py", line 5958, in <module>
  File "ArmoryQt.py", line 679, in __init__
  File "ArmoryQt.py", line 5726, in startBlockchainProcessingInitialization
  File "ArmoryQt.py", line 5748, in completeBlockchainProcessingInitialization
  File "armoryengine\BDM.pyc", line 189, in instantiateBDV
TypeError: in method 'BlockDataViewer_getNewBDV', argument 1 of type 'string const &'

I moved armorydb_ip and armorydb_port from the command-line into armoryqt.conf. Both with and without quotes on armorydb_port. No change in behavior.

I'll look into setting up a Windows build environment, but I don't have high hopes of success....

[goatpig]

I moved armorydb_ip and armorydb_port from the command-line into armoryqt.conf. Both with and without quotes on armorydb_port. No change in behavior.

I'll look into setting up a Windows build environment, but I don't have high hopes of success....

The .conf processor always strips quote so it won't help there.

You can try this fix:

https://github.com/goatpig/BitcoinArmory/blob/testing/armoryengine/BDM.py#L189

concatenate port in str()

[skyhawk]

Not having a whole lot of luck here.  The build instructions cover dependencies and what to do with them, but nowhere can I find documentation on how to _actually build it_ and package it for distribution.

Additionally, Visual Studio Express 2013 has vanished from the face of the planet. Best I can find is Visual Studio Express 2015.

I've got it compiling, produces several thousand warnings and a handful of errors. I note several of them regard a missing \cppForSwig\fcgi\include\fastcgi.h

Still banging my head on this

EDIT - 29 errors, 28 of them for missing fastcgi.h, one for missing Python.h
Vast majority of the 1015 warnings are for type conversions

EDIT the EDIT - Whoops, some of these are for fcgiapp.h too...

[goatpig]

Additionally, Visual Studio Express 2013 has vanished from the face of the planet. Best I can find is Visual Studio Express 2015.

2015 is fine

I've got it compiling

You don't need to build the C++ code to just modify a line in Python. You can simply swap in the binaries from the packges.

produces several thousand warnings and a handful of errors. I note several of them regard a missing \cppForSwig\fcgi\include\fastcgi.h

You have to init the submodules:

Code:

git submodule init
git submodule update

Code:

EDIT - 29 errors, 28 of them for missing fastcgi.h, one for missing Python.h

You need to install Python and all of its dependencies. You have also need to make sure you picked the Release target and x64 platform.

[skyhawk]

Unfortunately, I'm about ready to throw in the towel on this.

I've got the build progressing to the linking stage, where the linker is choking on a 170MB cryptopp.lib

The latest version of Crypto++ compiles to a 33MB library, but it doesn't drop into armory -- looks like armory implemented a custom DetSign file/header into the previous version.

You don't need to build the C++ code to just modify a line in Python. You can simply swap in the binaries from the packges.

I don't follow? I need to build the c++ executables in order to bundle them together into the final Armory .exes, yes?  (I am in no way claiming to be a developer or in any way competent enough to be successful at tinkering at this level)

[goatpig]

Unfortunately, I'm about ready to throw in the towel on this.

I've got the build progressing to the linking stage, where the linker is choking on a 170MB cryptopp.lib

The latest version of Crypto++ compiles to a 33MB library, but it doesn't drop into armory -- looks like armory implemented a custom DetSign file/header into the previous version.

Right click the cryptopp project in the solution explorer and pick "Build".

I don't follow? I need to build the c++ executables in order to bundle them together into the final Armory .exes, yes?  (I am in no way claiming to be a developer or in any way competent enough to be successful at tinkering at this level)

No you don't. That executable is a "lie". It just "freezes" Python and the Armory scripts into an exec, but what it essentially does can be resumed to this:

Code:

python ArmoryQt.py

If you setup Python on your system, you don't need to bother with the freezing process, you can just invoke Python from the command line/IDE instead. You don't need to build the C++ code as long as you swap in the binaries from the installer. You need Python installed whether you want to invoke the script manually or "freeze" into an executable.

[skyhawk]

Having trouble with CppBlockUtils -- Python doesn't see it because it's got the _ in the filename

Code:

C:\Users\armory-build\Desktop\BitcoinArmory>python ArmoryQt.py
Traceback (most recent call last):
  File "ArmoryQt.py", line 40, in <module>
    import CppBlockUtils as Cpp
ImportError: No module named CppBlockUtils

If I remove it Python wobbles for not having an init function?

Code:

C:\Users\armory-build\Desktop\BitcoinArmory>python ArmoryQt.py
Traceback (most recent call last):
  File "ArmoryQt.py", line 40, in <module>
    import CppBlockUtils as Cpp
ImportError: dynamic module does not define init function (initCppBlockUtils)

I tried adding the _ to the import line in ArmoryQt.py, which then failed on another import further down. I add the _ to ArmoryUtils.py and then it failed with:

Code:

Traceback (most recent call last):
  File "ArmoryQt.py", line 42, in <module>
    from armoryengine.ALL import *
  File "C:\Users\armory-build\Desktop\BitcoinArmory\armoryengine\ALL.py", line 8, in <module>
    from armoryengine.ArmoryUtils import *
  File "C:\Users\armory-build\Desktop\BitcoinArmory\armoryengine\ArmoryUtils.py", line 49, in <module>
    from _CppBlockUtils import KdfRomix, CryptoAES, ConfigFile_fleshOutArgs
ImportError: cannot import name KdfRomix

I also tried running from my fresh Win10 build VM with no previous config or bitcoind or anything else, just configured armoryDB IP and Port in armoryqt.conf and it's behaving exactly the same as on my workstation.  I also tried it from a fresh Windows 7 VM, again same behavior.

[goatpig]

You need to copy _CppBlockUtils.pyd and CppBlockUtils.py

[skyhawk]

CppBlockUtils.py does not appear to exist in the source distribution....

[achow101]

CppBlockUtils.py does not appear to exist in the source distribution....

That's a file that is generated by SWIG when you compile the C++ code.

[goatpig]

ugh you need to run swig to create that actually. You are stuck with getting the build process to work or waiting for a new one.

[skyhawk]

I'm pleased to report ArmoryQt on Windows is now able to connect to remote ArmoryDB as of 0.96.0.4-testing!

Thanks goatpig!