Bitcoin Armory installation on Mint 20.04

[tpofo]

I am having problems running after upgrade to Mint 20.04.  Mint 20.04 is based on Ubuntu 20.04 LTS (Focal Fossa).
I tried to reinstall and some dependencies fail.
On execution I get:

Traceback (most recent call last):
  File "/usr/bin/../lib/armory/ArmoryQt.py", line 36, in <module>
    from PyQt4.QtCore import *
ImportError: No module named PyQt4.QtCore


On trying to reinstall from archive, I get:

dpkg: dependency problems prevent configuration of armory:
 armory depends on python-qt4; however:
  Package python-qt4 is not installed.
 armory depends on python-psutil; however:
  Package python-psutil is not installed.

I tried to install python-qt4 and other dependencies, but had some issues as well. 

Anyone get armory working on the new Mint or Ubuntu?

[bob123]

I tried to install python-qt4 and other dependencies, but had some issues as well. 

You need to install python-qt4.
Which errors did you encounter when trying to install it?

[goatpig]

Ubuntu 20 comes with py3 by default, not py2.

[tpofo]

Ubuntu 20 comes with py3 by default, not py2.

I tried installing qt4 but doesn't seem available.  I have python2 installed from the past and it is still installed after upgrade.  I tried many ways to install python-qt4 and it doesn't show up.
"armory : Depends: python-qt4 but it is not installable"

Does Armory absolutely need qt4 to be installed?  It cannot use Qt5 with python3?
What would you suggest in my situation to get armory working?  Force install qt4 somehow? But, I am worried it might make a mess to have qt4 installed.  It isn't any longer in the repository and seems like mint 20 doesnt want it installed when qt5 is installed already. 

I was thinking maybe a search and replace for qt4 to qt5 in the python files or is armory going to break if I do that.

Thanks in advance for the help.

[goatpig]

Does Armory absolutely need qt4 to be installed?  It cannot use Qt5 with python3?

I wish it was that easy. Working on the migration to py3/Qt5 for the next release.

What would you suggest in my situation to get armory working?

If you want to go the install route, you need to install Qt4 on your system from an archive repo then install pyqt4 for py2 from pip. If I was you, I'd setup Core and ArmoryDB on the main system then have an encrypted Ubuntu19 VM for ArmoryQt (which will have your wallets) and setup a tunnel between the VM and the host so that the client can open a socket to the DB.

[tpofo]

I was actually thinking of the VM way too.  I use offline cold wallet and already have signing armory on a USB bootable linux(I unplug internet on machine while I boot from USB). 
I don't mind too much, for the moment, doing it this way as I don't need super frequent access to armory. 

I am just curious, but will armory ever see a future of a fully compiled Qt app just like bitcoin core wallet?  I understand you want the GUI in python and all the fast stuff in C++.  But, Qt in C++ is very mature and easily to maintain.  Yes, I know it is a lot of work.  I was just asking about the far future and evolution of armory.  I have to say I love this wallet and have been using it for probably more than 7 years.  I've learned to trust it.

How about making a docker container instead of a VM?

Thanks again for the help here.

[goatpig]

I understand you want the GUI in python and all the fast stuff in C++.

I dont want that, it's grand fathered in from etotheipi's era. A lot of code was implement in Python, not just the GUI. Wallets, transaction signing, part of the interfacing with Core. In the current development state (as seen in the dev branch), everything but the GUI was redone in C++. It's an incremental development state. In 2017, I had to add SegWit support so I rewrote the signing code in C++. For this release cycle I needed BIP32, PSBT and public data encryption, so the whole wallet structure was redone in C++.

If I could just take the PyQt5 GUI and convert it to native Qt5 cheaply, I would. Issue is the GUI is hardcoded Qt4 into a ~10k LoC file, and then another ~10k across a few other files. Therefor, I have to take one development cycle to first migrate to Py3/Qt5 before I can look at this stuff. The migration though gets rid of SWIG and interfaces the client with the DB in an agnostic fashion.

Once I have Qt5 in there, I need to deliver some GUI elements and rework some others. I'll take that opportunity to implement that stuff in QML instead. Over time, once the enough of the new GUI elements have replaced the old hardcoded stuff, I'll bite the bullet and convert the rest of the GUI to QML. Then I'll consider getting rid of Python entirely or not. I don't like Python in the first place, but some useful libs like HWI are in Python, and I was considering doing hardware wallet support through that on the cheap, so I'm not 100% sure I want to prune Python out entirely in the long run.

How about making a docker container instead of a VM?

Client talks to DB over a socket (DB listens), as long as you allow that through the docker container, you could run the client within a docker instance. I would say it's more demanding to run the whole stack in docker  since the DB requires local disk access to the blockchain data, meaning you need to bundle Core with its data in there as well.

[tpofo]

Thanks for taking the time for the explanation of future.  I appreciate it.  I am only an amateur coder, but I understand exactly what you are talking about.
Are you a 1-man team on this?  Has nobody else joined the team to accelerate stuff?  It is way too much for 1 person to do.  But, the good news is that even if armory was never updated, I love it the way it is.  But, my fear is of it becoming un-runnable on systems or because of bitcoin core wallet updates(happened in past) or the new BIPs and wallet address conventions.
There is a new bitcoin softfork in the works.  schnorr and taproot implementation.  I hope all goes smooth.


As far as the docker idea, I may try playing around with it.  Kind of a fun project.  I think it is a great idea having the db communication of course outside the container to the local OS. 
I've seen some examples of having a remote display setup through docker.  So, you can actually have the gui app display on local OS, but it is running in docker(which is communicating to local HD db for both bitcoin and armorydb).  Sounds like an awesome cool Sunday project.

[goatpig]

Thanks for taking the time for the explanation of future.  I appreciate it.  I am only an amateur coder, but I understand exactly what you are talking about.
Are you a 1-man team on this?  Has nobody else joined the team to accelerate stuff?  It is way too much for 1 person to do.  But, the good news is that even if armory was never updated, I love it the way it is.  But, my fear is of it becoming un-runnable on systems or because of bitcoin core wallet updates(happened in past) or the new BIPs and wallet address conventions.
There is a new bitcoin softfork in the works.  schnorr and taproot implementation.  I hope all goes smooth.

There are a couple people contributing to some specific aspects. I did pay a guy for a few months when I could afford it. The core of the work I do on my own. Development has been lagging behind cause of my day job these past couple years, but I hope getting more time starting this year to go back to a more frequent release schedule. I don't intent on dropping maintenance ever, even if there are ups and lows in the development speed.

[tpofo]

I got armory working with Mint 19 inside docker.  I made a custom image from the mint 19.3 docker container and made up a Dockerfile to handle the armory installation as well as copying over the needed bitcoind file.  Note the bitcoind file must be where you run the docker build.  Installing bitcoin on the docker can be done, but I had issues with the PPA, so I just copied over the latest 0.20 daemon.  Then, I mounted the shared drives needed for bitcoin blockchain, bitcoin conf (telling it where blockchain is located), and lastly the armorydb directory.  So, a total of 3 mappings outside of docker to local HDs.  I needed 3 mappings because that is the way I had it before and my armory configured that way.  I have armory launching automatically upon docker container run and then I specify the display environment so I see the gui.  The blocks synced and armory is online with daemon running background on docker referring to external files.
One small issue is that the newly written blocks are root perms because docker accesses the local filesystem as root.  If I want to run local bitcoin-qt, I must first change the perms back to user.  No big deal for now.  I probably can tweak this more and fix things up more elegantly.  For now, this is great and is how I will be using armory.

Surprisingly it runs extremely fast.  I can't tell the difference from before.   

Dockerfile

Code:

FROM linuxmintd/mint19.3-amd64
RUN apt update
RUN apt -y install python-qt4 python-psutil
RUN wget https://github.com/goatpig/BitcoinArmory/releases/download/v0.96.5/armory_0.96.5_amd64_gcc7.2.deb
RUN dpkg -i armory_0.96.5_amd64_gcc7.2.deb
COPY bitcoind /usr/bin/
CMD ["/usr/bin/armory"]

Then build the new image(same dir as Dockerfile):

Code:

docker build -t armory:latest .

Command to start docker container:

Code:

docker container run -it -v $HOME/.armory:/root/.armory -v $HOME/.bitcoin:/root/.bitcoin -v /media/ssd/.bitcoin:/media/ssd/.bitcoin -e DISPLAY=unix$DISPLAY --net=host armory

[goatpig]

That's an odd way of doing it (running all process within docker and share the files across the host). My suggestion was to run Core and ArmoryDB on the host system, ArmoryQt within docker/VM, and point it to the port ArmoryDB is listening to. Would fix your permission issue too.

[tpofo]

That's an odd way of doing it (running all process within docker and share the files across the host). My suggestion was to run Core and ArmoryDB on the host system, ArmoryQt within docker/VM, and point it to the port ArmoryDB is listening to. Would fix your permission issue too.

I think I misunderstood you and did it the way I thought was more intuitive to me.  haha.  I am still up to the challenge and doing another take on this with your way.  I didn't understand the part about getting armoryDB and core running separately while armoryQt knowing they are running on the port.  I assumed launching armory executes the bitcoind and armoryDB no matter what.
So, I would launch the bitcoind and armoryDB first.  Then, docker running ArmoryQt would see them by port?  That is the part I don't understand.

[goatpig]

ArmoryQt talks to ArmoryDB over a socket. ArmoryDB talks to Core over a socket and reads Core's blockchain files. ArmoryQt does not talk to Core directly, it gets its status reports from ArmoryDB.

When you start ArmoryQt it will:

1) Check there's a Bitcoin Core process running, if not, try to spawn it. You can disable this option in the GUI (File -> Settings).
2) Try to connect to ArmoryDB, If that failed, try to spawn ArmoryDB, then try to connect again.

In the setup I propose, you would:

1) Start Core and ArmoryDB manually on the host system.
2) Start ArmoryQt within a docker container.
3) Setup the docker container so that ArmoryQt can connect to port 9001 on the host machine.

A few relevant Armory settings:

a) You can set a custom ArmoryDB port (--armorydb-port=xxxx on the client side, --fcgi-port on the DB side)
b) If the ArmoryDB IP is not 127.0.0.1 (you can set custom IPs with --armorydb-ip), the client will try to speak HTTP instead of FCGI to the DB. This is because FCGI is a protocol wrapped in HTTP, and the client expects to speak a remote DB, and FCGI expects to sit behind an HTTP deamon. This step is foregone for local client to DB connections, for ease of use. In this case, you should spawn the client --force-fcgi as well, to avoid issues. Otherwise, you'd have to put the DB behind nginx and connect the client to nginx.

As an example assuming your host system is accessible as 10.0.0.40 within the docker container, and you want to connect over port 5000, you would spawn ArmoryDB host side with this command:

Code:

./armorydb --fcgi-port=5000

and the client within the docker container with this command:

Code:

python ArmoryQt.py --armorydb-ip="10.0.0.40" --armorydb-ip="5000" --force-fcgi

[tpofo]

I got it to work your way!

There was a typo in your armorydb-ip port, you meant "armorydb-port="5000".  Since I am running docker with "--net=host" parameter all network communication in docker is on host network.  That solves the IP resolution issues.  I still have to mount the local .armory directory in the docker commandline so ArmoryQt sees my config file when it is launched in there.

I built the docker image with the following Dockerfile. 

Dockerfile

Code:

FROM linuxmintd/mint19.3-amd64
RUN apt update
RUN apt -y install python-qt4 python-psutil
RUN wget https://github.com/goatpig/BitcoinArmory/releases/download/v0.96.5/armory_0.96.5_amd64_gcc7.2.deb
RUN dpkg -i armory_0.96.5_amd64_gcc7.2.deb
CMD /usr/bin/python /usr/lib/armory/ArmoryQt.py --armorydb-port="5000" --force-fcgi

I build docker image calling it armoryqt:

Code:

docker build -t armoryqt .

Now to run ArmoryQT, I first start bitcoind and ArmoryDB (passing my nonstandard datadir to it) on local host:

Code:

bitcoind
ArmoryDB --fcgi-port=5000 --satoshi-datadir="/media/ssd/.bitcoin"

Then, I start ArmoryQt on docker:

Code:

docker container run -it -v $HOME/.armory:/root/.armory -e DISPLAY=unix$DISPLAY --net=host armoryqt

I can see the connection on local host to ArmoryDB by "registered bdb".  Inside ArmoryQt, I can see the green block number showing it is connected and with latest bitcoin block updated.
Great!  Thanks for the help on this one.  I'll stick with your way since it is more elegant and efficient.  So far works well.

[goatpig]

You can set the command line args in their respective config files btw:

- armoryqt.conf for ArmoryQt
- armorydb.conf for ArmoryDB

One command per line, no -- prefix

[tpofo]

Thanks,
I just added those 2 conf files and now even simpler.  I just run:

Code:

bitcoind
ArmoryDB
docker container run -it -v $HOME/.armory:/root/.armory -e DISPLAY=unix$DISPLAY --net=host armoryqt

This can be put into a batch file and added to the menu with a armory icon I guess.  So, pretty seamless.  I have to say it is extremely fast and I don't notice any difference.  This little project made me learn docker more and I have to say I like it.

[tpofo]

What is .cookie_ written in .armory directory?  That is causing a small issue as it is written by ArmoryQt inside docker as root.

I decided to make the batch file:

Code:

#!/bin/bash
x-terminal-emulator -e bitcoind &
x-terminal-emulator -e ArmoryDB &
docker container run -it -v $HOME/.armory:/root/.armory -e DISPLAY=unix$DISPLAY --net=host armoryqt

This way it launches the bitcoind and ArmoryDB in a separate spawned terminal.  I can view the output as well as closing them off when I am finished.
But, how does ArmoryQT normally shutown the bitcoind and ArmoryDB when you exit?  Is it done by port communication or some terminal kill process?

[goatpig]

The cookie file is created by ArmoryDB on startup if spawned with --cookie. A local client that spawns the DB will use --cookie. The file contains a one time token (random number generated at DB spawn) that the client reads. That token has to be passed to use certain elevated commands, notably the shutdown sequence. Without that token, a client cannot shutdown the DB, and the DB will persist the client. Restarting the client will have it connect to the existing DB process.

The DB controls Core. Core generates a cookie file as well, that holds a token to authenticate the user for Core's JSON-RPC API. If ArmoryDB cannot access that cookie file, it can't connect to Core's RPC. This means it will fail to get some status reports from Core, and it won't be able to shut it down.

Shutdowns are all graceful, requested through commands. There may be some verbose left in the Python client suggesting otherwise, but these are left overs from the previous implementation (when the Python client handled Core). They don't do anything anymore.