Standalone Armory -- Struggling with python/OS issues

[etotheipi]

I've decided it's finally time for Armory to manage its own bitcoind instance.  Instead of having the user start bitcoind/Bitcoin-Qt and wait for it to sync, then start Armory.   Armory will actually require bitcoind/-qt to not be running, so that it can start its own bitcoind instance (and setup the RPC interface if it's not setup yet).  It would then use the RPC interface to monitor bitcoind's progress sync'ing.

This will be a major usability upgrade -- the user will spend the same amount of time getting from week-old cold boot to having a synchronized bitcoind+Armory but they didn't have to do any of the work.  And it will also likely be a better experience than Bitcoin-Qt, since Armory likes to explain things to the user, so that they have some idea of what's going on and what to expect.  Armory will be able to estimate how long until bitcoind is synchronized, and tell the user how it's only a one-time synchronization and future loads will be much faster, etc.  And it will only connect "the old way" when it's fully sync'd (which resolves more usability problems, because users get impatient and open Armory prematurely and it chokes).

However, I'm having some serious problems with the implementation.  I can't reliably guarantee that bitcoind is shutdown, especially if Armory were to crash.  So here's what I've got right now:

• I have added a SatoshiDaemonManager to Armory.  It will autodetect .bitcoin dir and find the bitcoind executable.
• If bitcoin.conf does not exist, it creates it and sets the permissions to 600 (Linux only, still need to figure out windows)
• Use the subprocess module to launch:  "sdm.bitcoind = subprocess.Popen([pathToBitcoind, arg1, arg2, ...])"
• Include a call to sdm.bitcoind.terminate() in the shutdown procedure to do a clean shutdown of bitcoind before exiting cleanly.

Now's the hard part: dealing with unclean shutdowns.   subprocess.Popen() creates a fork that doesn't necessary go away when the parent process does.  I've been doing testing in Linux, and it seems to be inconsistent.  But it definitely can leave a ghost bitcoind running which then blocks the user's attempt to restart Bitcoin-Qt or Armory.    

The really challenging part of this is that I think I can make a solution work in Linux, but it is most definitely not cross-platform.  And Windows is even more complicated, well, because it's windows.

One thing I had considered was wrapping Armory itself with an outer script that will run Armory itself in a subprocess, wait for it to quit/die, then check whether the process it spawned is still open and kill it if necessary (will write out the PID when Popen is called, to a file in ARMORY_HOME_DIR, which the outer script can check for after the Armory subprocess exits).  The nice thing about this solution is that even if Armory segfaults, the outer script will finish running.

However, it's not so clear how easy this will be in windows (which has a totally different process management system), and it certainly will require an OS-dependent code branch -- and both may be kinda complicated.  I presume it will work, it just may be a lot of effort...

Another solution could be to just check for open Bitcoin-Qt/bitcoind when Armory is started, and ask the user if they are willing to restart it.  This is probably not going to work, because it would allow them to reopen Armory, but prevent them from opening Bitcoin-Qt after Armory dies (on the upside, it's good for creating competitive advantage for Armory )

An ideal solution would be to find a way to create a subprocess that is linked to the current process -- if the current process goes away for any reason, the subprocess should too.  So far, I don't see a clean way to do that.  It looked like process id groups were the solution in Linux, but may not have an equivalent in Windows.

By the way:  I will have an option to just use an existing bitcoind/Bitcoin-Qt instance, but most users, especially those who have never used Bitcoin, will benefit dramatically from having this as the default, all-in-one, standalone solution.  It will probably show up in the preferences to "Allow using existing already-open Bitcoin-Qt/bitcoind instances".  

Recommendations, welcome!

[picobit]

One problem you are going to have to face is that there is no way to guarantee that a codes cleanup part runs.  You can e.g. try to catch any reasonable Unix signal and then clean up, but you cannot catch SIGKILL and SIGSTOP, the user can always do a "kill -9" on the Python process.

One way of doing it that I would consider is a separate "guardian process".  Armory starts bitcoind, gets its PID.  Armory then starts the guardian process, passing its own and bitcoind's PID.  Every second, the guardian process checks that Armory is still running, and kills bitcoind if not.  Every second Armory checks that the guardian is running, kills bitcoind and shuts down cleanly if not (*).  Every second Armory checks that bitcoind is running, and restarts it (and the guardian) if not.

(*) Restarting the guardian would be tempting, but it will only crash if something is seriously wrong.

[etotheipi]

One problem you are going to have to face is that there is no way to guarantee that a codes cleanup part runs.  You can e.g. try to catch any reasonable Unix signal and then clean up, but you cannot catch SIGKILL and SIGSTOP, the user can always do a "kill -9" on the Python process.

One way of doing it that I would consider is a separate "guardian process".  Armory starts bitcoind, gets its PID.  Armory then starts the guardian process, passing its own and bitcoind's PID.  Every second, the guardian process checks that Armory is still running, and kills bitcoind if not.  Every second Armory checks that the guardian is running, kills bitcoind and shuts down cleanly if not (*).  Every second Armory checks that bitcoind is running, and restarts it (and the guardian) if not.

(*) Restarting the guardian would be tempting, but it will only crash if something is seriously wrong.

Excellent idea!  I like it.  Either Armory will shutdown both, or neither of them.  If it doesn't shut down either of them, the guardian will take care of it. 

This should be fairly straightforward in Linux.   But, I still have to figure out how to apply this in Windows.  I hate to create a new python project and convert it to an executable with py2exe, but anything is better than batch/DOS scripting...

[Red Emerald]

I'm not sure how vital this is.  Sure, it's kind of a pain to manually setup both bitcoind and armory, but isn't bitcoind going to go away eventually anyway?  Why bother with this when you could be working on removing bitcoind all together?

[K1773R]

One problem you are going to have to face is that there is no way to guarantee that a codes cleanup part runs.  You can e.g. try to catch any reasonable Unix signal and then clean up, but you cannot catch SIGKILL and SIGSTOP, the user can always do a "kill -9" on the Python process.

One way of doing it that I would consider is a separate "guardian process".  Armory starts bitcoind, gets its PID.  Armory then starts the guardian process, passing its own and bitcoind's PID.  Every second, the guardian process checks that Armory is still running, and kills bitcoind if not.  Every second Armory checks that the guardian is running, kills bitcoind and shuts down cleanly if not (*).  Every second Armory checks that bitcoind is running, and restarts it (and the guardian) if not.

(*) Restarting the guardian would be tempting, but it will only crash if something is seriously wrong.

why should this prevent me to kill both at the same time?

[goatpig]

For windows,

CreateProcessWithLogonW is your go to WinAPI function to start a process from another one. I haven't read into it thouroughly, but I think you can set it so that the created thread will be terminated if the original thread dies. Granted, I'd have to dig into it.

Would have to then make a small dll with the windows specific code, and integrate all that for the windows environment part.

[picobit]

why should this prevent me to kill both at the same time?

It won't.  It is not a 100% solution, you can certainly shoot yourself in the foot.  But it will solve the problem of a process dying due to a bug or being killed by the user.


EDIT: In case you don't think the solution is elegant: it is because it ain't!

[etotheipi]

I have one lingering, serious issue with this solution as I work to implement it...  creating bitcoin.conf in Windows with a username and password, and then setting read permisison on that file only for that user.  Permissions on Windows is pretty darned ugly.  On the other hand, I think Windows blocks off users' directories by default, so maybe this isn't so important -- but I need to be diligent in making sure this is done right.  I don't want to induce security holes in Bitcoin-Qt because of this!

So in both Windows and Linux:

(1) I check to see if the /home/username/.bitcoin directory is there (or equiv on Windows).  
(2) If not (because they've never run Bitcoin-Qt), I create it.
(3) Check whether .bitcoin/bitcoin.conf file exists
(4) If not (because they've never used RPC interface), then I create it with simple username and a random 128-bit password
(4a) If we just created the file, set the permissions on it so no other users can read it!  In Linux, it's trivial to use os.chmod(...).  Windows, not so simple!
(5) Start bitcoind as a separate process
(6) Start up "guardian" process (as recommended by picobit), pass it the PID of both bitcoind and the PID of Armory.
(7) Read bitcoin.conf file and extract username and password
(8) Connect to bitcoind RPC, and monitor synchronization status
(9) On clean shutdown, kill bitcoind, then kill guardian.
(9a) If Armory is killed, then the guardian will see the PID disappear (or change names), and it will kill the bitcoind instance and exit.

There's a lot of complexity with #8, because there is not a hard definition of "synchronized", but I think I have something sufficient figured out for that (estimating when bitcoind is sync'd, and only then starting the Armory BlockDataManager).  My main concern is making sure that 4a is done correctly.

In Linux, it seems like default behavior is for users to be able to read each other directories (including malicious system processes can, too), and thus it is very important to set the permissions on that file.  In Windows, it looks like default is for users to not even be able to open the home directories of other users.  I tried digging into the Windows permissions on the file, and it's not even clear to me how those permissions are supposed to be set.

Anyone here even use bitcoind.exe in Windows?  Do you do anything to secure your bitcoin.conf file?

[ErebusBat]

Been awhile since I have lived in windows, but I used to use cacls from batch scripts to change the rights.

IIRC something like this would probably do what you need:

Code:

CACLS bitcoin.conf /G "Owner":F

That should removed any existing ACLs and then grant the Owner of the file full control.

See:
  http://ss64.com/nt/cacls.html
  http://stackoverflow.com/questions/2928738/how-to-grant-permission-to-users-for-a-directory-using-command-line-in-windows

[etotheipi]

Been awhile since I have lived in windows, but I used to use cacls from batch scripts to change the rights.

IIRC something like this would probably do what you need:

Code:

CACLS bitcoin.conf /G "Owner":F

That should removed any existing ACLs and then grant the Owner of the file full control.

See:
  http://ss64.com/nt/cacls.html
  http://stackoverflow.com/questions/2928738/how-to-grant-permission-to-users-for-a-directory-using-command-line-in-windows

Thanks Erebus, I will play with that. I had seen something about CACLS, but didn't realize that's a ubiquitous, reliable cmd tool in Windows.  (it looks like icacls is the one to use, actually; cacls has been deprecated, apparently).


As I implement this solution further, it's feeling very hack-y.  I know it's not elegant.  And I'm likely to have problems with non-standard installations, etc.  For standard setups it looks like it will work fine, and in some cases I will be installing/copying with a standard configuration.  But I am wondering if there's a better way.  One thing that crossed my mind was that I could try to jam all the Satoshi code into a library, and change the main() function to a regular function that could be called in a separate thread from inside Armory.  Then I don't have to deal with any of this multi-process stuff...

But that comes with it's own inconveniences -- mainly that the user no longer gets the choice about which bitcoind/-qt version to use, and I have to a lot of work to integrate and compile new version of bitcoind directly into my codebase (this is probably pretty ugly in Windows).  I have to manually integrate and modify each new release of the satoshi client, and if, say, there was a critical vulnerability/update to the satoshi client, I would have to critically update and release a new version of Armory immediately.

I just don't feel like there's any clean solutions here.  But I feel like something must be done to improve accessibility -- having specific requirements/instructions for downloading and installing a separate program, and then not starting/initializing Armory until certain conditions are met... it's a lot to ask of marginally-interested users who don't really understand Bitcoin...

[rini17]

By the way:  I will have an option to just use an existing bitcoind/Bitcoin-Qt instance, but most users, especially those who have never used Bitcoin, will benefit dramatically from having this as the default, all-in-one, standalone solution.  It will probably show up in the preferences to "Allow using existing already-open Bitcoin-Qt/bitcoind instances".  

Recommendations, welcome!

How will users benefit running bitcoind only when armory is started? I call BS, bitcoind can (and is meant to) run all the time in background. With 0.8 disk trashing is no longer an issue. Running it as a service benefits from more open connections, always most recent data readily at hand and also user's transactions propagate quicker. Maybe if user has low-spec computer... but then he shouldn't use bitcoin-qt nor armory at all...

[etotheipi]

How will users benefit running bitcoind only when armory is started? I call BS, bitcoind can (and is meant to) run all the time in background. With 0.8 disk trashing is no longer an issue. Running it as a service benefits from more open connections, always most recent data readily at hand and also user's transactions propagate quicker. Maybe if user has low-spec computer... but then he shouldn't use bitcoin-qt nor armory at all...

Once I get Armory's resource usage down, then it too will run in the background, along with Bitcoin-Qt, bitcoind.  The user will just keep both programs up.  Running and keeping Armory open will be done in the same way that users would run and keep Bitcoin-Qt/bitcoind open all the time.

The problem with your logic is assuming that more than 10% of target users understand or care about these details.  They heard about Bitcoin from a friend and want to get some coins and buy stuff from Bitcoinstore.com because it has good prices.  I have instantly lost that user if they have to install multiple programs and follow directions to start it.  And it's unnecessary -- I can manage it myself, and they'd prefer me to manage it for them.  This can be win-win for the vast majority of users.   And its default behavior can be the same as bitcoind/-qt:  minimize to tray and run all the time.  Nothing's really different than the way they run bitcoind/-qt, except they have a much more powerful interface on top of bitcoin-qt.

For the 10% of users that do care/understand:  that's what the "Allow using existing already-open Bitcoin-Qt/bitcoind instances" option is for.

[rini17]

How will users benefit running bitcoind only when armory is started? I call BS, bitcoind can (and is meant to) run all the time in background. With 0.8 disk trashing is no longer an issue. Running it as a service benefits from more open connections, always most recent data readily at hand and also user's transactions propagate quicker. Maybe if user has low-spec computer... but then he shouldn't use bitcoin-qt nor armory at all...

Once I get Armory's resource usage down, then it too will run in the background, along with Bitcoin-Qt, bitcoind.  The user will just keep both programs up.  Running and keeping Armory open will be done in the same way that users would run and keep Bitcoin-Qt/bitcoind open all the time.

The problem with your logic is assuming that more than 10% of target users understand or care about these details.  They heard about Bitcoin from a friend and want to get some coins and buy stuff from Bitcoinstore.com because it has good prices.  I have instantly lost that user if they have to install multiple programs and follow directions to start it.  And it's unnecessary -- I can manage it myself, and they'd prefer me to manage it for them.  This can be win-win for the vast majority of users.   And its default behavior can be the same as bitcoind/-qt:  minimize to tray and run all the time.  Nothing's really different than the way they run bitcoind/-qt, except they have a much more powerful interface on top of bitcoin-qt.

For the 10% of users that do care/understand:  that's what the "Allow using existing already-open Bitcoin-Qt/bitcoind instances" option is for.

We mix two issues here. One is transparent bundling/installation of bitcoind together with armory. Other,and completely indep. dent, is running bitcoind as service on startup by default. The first thing is clearly preferable, the second thing can be configurable and I recommend the default to be "yes". As opposed to all the other windows crapware that has no reservations against draining computer resources, running bitcoind server is useful by strengthening whole network. If users don't care anyway, where's the problem? Who doesn't care, he will lose bitcoins to malware soon anyway.

[picobit]

We mix two issues here. One is transparent bundling/installation of bitcoind together with armory. Other,and completely indep. dent, is running bitcoind as service on startup by default. The first thing is clearly preferable, the second thing can be configurable and I recommend the default to be "yes". As opposed to all the other windows crapware that has no reservations against draining computer resources, running bitcoind server is useful by strengthening whole network. If users don't care anyway, where's the problem? Who doesn't care, he will lose bitcoins to malware soon anyway.

In principle you are probably right.  It is just not nice to start background processes on users machines without their explicit permission.  Of course many windows programs do just that - but that is no reason to do it as well.