Armory - Discussion Thread

[Raoul Duke]

Ever since you added the version check Armory is always losing the connection to the bitcoin client.
It can't ever hold it for a couple minutes straight.

Strange.  Does this happen with 0.81 also?  0.82.1?  Can you send me a log file?  Just like for wachtwood, I will release a temporary version just for logging network messages (or maybe it will become part of the release, hmm).  Hopefully that will help answer some questions about networking issues.

Yes, it already happened with both those versions. Or at least with 0.82.1. But if I'm not mistaken I've already updated it twice since that started happening. My last update(the second) was today, to 0.82.2.

I'll fire it up and collect a log file to see if there is something useful there.

[payb.tc]

i was getting this error in an older version (can't remember which one), so I decided to upgrade to the latest, but still get the same error on startup:
...
edit 2: before trying the win32 version, i just deleted a few things out of the appdata folder such as mempool.bin and settings file, and it worked.

I don't suppose you have the settings file in your recycle bin or something, do you?  I just looked at how that error is triggered, and it's by the "LastFilterState" variable in the settings file being a string (supposed to be an integer).  I can see why deleting that file fixed it, but I'm curious what it got set to that caused the problem.  I can't see in the code how it could get set to non-integer value.

no i don't i'm sorry. if it ever happens again, i'll be sure to keep a copy for you.

[Raoul Duke]

Code:

2012-08-13 12:39 (INFO) -- ArmoryQt.py:350 - Loading blockchain took 278.5 seconds
2012-08-13 12:39 (INFO) -- ArmoryQt.py:403 - Usermode: Expert
2012-08-13 12:39 (INFO) -- ArmoryQt.py:704 - Changing usermode:
2012-08-13 12:39 (INFO) -- ArmoryQt.py:705 -    From: Expert
2012-08-13 12:39 (INFO) -- ArmoryQt.py:713 -      To: Expert
2012-08-13 12:39 (INFO) -- ArmoryQt.py:818 - You are running the latest version!
2012-08-13 12:39 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:40 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:40 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:41 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:45 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...

Those are the only messages on the logfile.
Not very useful it seems.

[etotheipi]

Code:

2012-08-13 12:39 (INFO) -- ArmoryQt.py:350 - Loading blockchain took 278.5 seconds
2012-08-13 12:39 (INFO) -- ArmoryQt.py:403 - Usermode: Expert
2012-08-13 12:39 (INFO) -- ArmoryQt.py:704 - Changing usermode:
2012-08-13 12:39 (INFO) -- ArmoryQt.py:705 -    From: Expert
2012-08-13 12:39 (INFO) -- ArmoryQt.py:713 -      To: Expert
2012-08-13 12:39 (INFO) -- ArmoryQt.py:818 - You are running the latest version!
2012-08-13 12:39 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:40 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:40 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:41 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:45 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...

Those are the only messages on the logfile.
Not very useful it seems.

Does it happen with version 0.81?

[Raoul Duke]

Code:

2012-08-13 12:39 (INFO) -- ArmoryQt.py:350 - Loading blockchain took 278.5 seconds
2012-08-13 12:39 (INFO) -- ArmoryQt.py:403 - Usermode: Expert
2012-08-13 12:39 (INFO) -- ArmoryQt.py:704 - Changing usermode:
2012-08-13 12:39 (INFO) -- ArmoryQt.py:705 -    From: Expert
2012-08-13 12:39 (INFO) -- ArmoryQt.py:713 -      To: Expert
2012-08-13 12:39 (INFO) -- ArmoryQt.py:818 - You are running the latest version!
2012-08-13 12:39 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:40 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:40 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:41 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...
2012-08-13 12:45 (ERROR) -- armoryengine.py:9123 - ***Connection to Satoshi client LOST!  Attempting to reconnect...

Those are the only messages on the logfile.
Not very useful it seems.

Does it happen with version 0.81?

I've been looking trough my logfiles and the oldest transaction that I remember it happening was already using 0.82.1.

[wachtwoord]

The compiling is only half of it.  I can't compile the 32-bit version on a 64-bit machine without have the supporting libraries installed as 32-bit (such as python).  If I install those packages as 32-bits, everything breaks, or weird things start happening.  I'm sure I could do it, but I have tried in the past and I could not resolve it in a timely manner.  A second VM seemed easier.

The other half (which wouldn't benefit from cross-compiling) is the process for creating the installer.  The program has to be setup for each new version, and for whatever reason the executable always comes out without an icon, so I have to modify it (using Resource Hacker).  It's not terrible, but if you consider that I do it about 5 times per night (per OS) when tweaking, testing, updating and trying to get out a release, it's pretty annoying.

Can't help you with the first bit, but resHack can be invoked on the command line and so you could probably create a batch file to automate this quite easily.

[etotheipi]

The compiling is only half of it.  I can't compile the 32-bit version on a 64-bit machine without have the supporting libraries installed as 32-bit (such as python).  If I install those packages as 32-bits, everything breaks, or weird things start happening.  I'm sure I could do it, but I have tried in the past and I could not resolve it in a timely manner.  A second VM seemed easier.

The other half (which wouldn't benefit from cross-compiling) is the process for creating the installer.  The program has to be setup for each new version, and for whatever reason the executable always comes out without an icon, so I have to modify it (using Resource Hacker).  It's not terrible, but if you consider that I do it about 5 times per night (per OS) when tweaking, testing, updating and trying to get out a release, it's pretty annoying.

Can't help you with the first bit, but resHack can be invoked on the command line and so you could probably create a batch file to automate this quite easily.

Oh good call.  I forgot that it can be scripted through some CLI calls.  If I figure out how to do it, I can add it to the "Post-Build Events" in MSVS.   

I'm using War Setup to make the .msi files.  It's not a great program, but it does work.  The problem is it's also little finnicky.  Sometimes I try making the MSI and I get cryptic errors, then I press the button again, and it works.  Not to mention I always forget to change settings for new versions and have to restart it.  On the other hand, I'm sure there's a way to script that too...

[chrisrico]

I have a server in my house which is connected to the Bitcoin network through a VPN all the time, with an empty wallet. When I start Bitcoin on my desktop, I use the -connect switch to connect only to my server's node. Unfortunately, this means I still need to catch up which can be rather slow. Is there a way I could tell armory on my desktop to connect to the Bitcoin instance running on the server and not use the block index file?

[etotheipi]

I have a server in my house which is connected to the Bitcoin network through a VPN all the time, with an empty wallet. When I start Bitcoin on my desktop, I use the -connect switch to connect only to my server's node. Unfortunately, this means I still need to catch up which can be rather slow. Is there a way I could tell armory on my desktop to connect to the Bitcoin instance running on the server and not use the block index file?

Unfortunately, there is no way to do that, yet.  I was just posting a week ago about how I wanted Armory to maintain its own blockchain files, and if I did that Armory would be able to connect to any trusted full node.  At the expense of having the blockchain data duplicated by Armory. 

So this is not currently possible, but I see it as a huge benefit to carrying through with this idea (and a drawback, a lot of users don't want the blockchain data duplicated in the case that Armory is connected to localhost).   

[Raoul Duke]

Unfortunately, there is no way to do that, yet.  I was just posting a week ago about how I wanted Armory to maintain its own blockchain files, and if I did that Armory would be able to connect to any trusted full node.  At the expense of having the blockchain data duplicated by Armory. 

So this is not currently possible, but I see it as a huge benefit to carrying through with this idea (and a drawback, a lot of users don't want the blockchain data duplicated in the case that Armory is connected to localhost).   

Why do you talk about blockchain data duplication?
I would use Armory exclusively if I could.
Between new wallet format not supported to import and the fact that I need bitcoind/bitcoin-qt running to use Armory I just use Bitcoin-qt.

[etotheipi]

Btw, anyone whose got network problems please either use the following version:

http://dl.dropbox.com/u/1139081/ArmoryTestingReleases/armory_0.82.5_win32.msi

or checkout the "networklog" branch.  Then run Armory using the " --netlog" option.  It will record all non inv/tx/block packets which will hopefully help me identify why the handshake isn't completing.  I'd really like to nail this down...

Also, I also just experienced the bug:

Adding new block data to memory pool failed!Block data did not extend the main chain!
***ERROR:  parseNewBlockData did not get enough data...
***ERROR:  parseNewBlockData did not get enough data...

Unfortunately, that was after like 5 days of running Armory... I have recompiled and restarted it in the debugger, but who knows if it will ever trigger...

[payb.tc]

Unfortunately, there is no way to do that, yet.  I was just posting a week ago about how I wanted Armory to maintain its own blockchain files, and if I did that Armory would be able to connect to any trusted full node.  At the expense of having the blockchain data duplicated by Armory.  

So this is not currently possible, but I see it as a huge benefit to carrying through with this idea (and a drawback, a lot of users don't want the blockchain data duplicated in the case that Armory is connected to localhost).  

Why do you talk about blockchain data duplication?
I would use Armory exclusively if I could.
Between new wallet format not supported to import and the fact that I need bitcoind/bitcoin-qt running to use Armory I just use Bitcoin-qt.

yeah, i have about 4 different installs of bitcoin... and 4 full copies of the blockchain... who cares? my 2tb of disk space can handle it like it's 1999.

i don't see armory having it's own copy as a problem at all.

i would also prefer if armory could drop it's dependance on bitcoin-qt.

[etotheipi]

Unfortunately, there is no way to do that, yet.  I was just posting a week ago about how I wanted Armory to maintain its own blockchain files, and if I did that Armory would be able to connect to any trusted full node.  At the expense of having the blockchain data duplicated by Armory.  

So this is not currently possible, but I see it as a huge benefit to carrying through with this idea (and a drawback, a lot of users don't want the blockchain data duplicated in the case that Armory is connected to localhost).  

Why do you talk about blockchain data duplication?
I would use Armory exclusively if I could.
Between new wallet format not supported to import and the fact that I need bitcoind/bitcoin-qt running to use Armory I just use Bitcoin-qt.

yeah, i have about 4 different installs of bitcoin... and 4 full copies of the blockchain... who cares? my 2tb of disk space can handle it like it's 1999.

i don't see armory having it's own copy as a problem at all.

i would also prefer if armory could drop it's dependance on bitcoin-qt.

I think everyone would prefer if I could drop the Bitcoin-qt dependence.  But that's a tall order among tall orders.  Right now, Bitcoin-qt is the most secure gateway available for Armory to talk to the Bitcoin network.  I would absolutely love to have that functionality directly integrated into Armory, but that's a multi-month process and fraught with bugs, security issues and stress.  On the other hand, if the Bitcoin-Qt networking could be functionally extracted & isolated so that I could essentially import it and hook up my program to it (which is still a lot of work, but do-able), I would make that one of my top priorities.    I understand that libcoin already has done this to some extent.  I don't know how good it is, though.  I was waiting for it to survive on the battlefield for a while before considering it.

Until then, I'm going to have to stick with Bitcoin-Qt.  On the other hand, switching to my own blockchain files would mean you could connect to any trusted full node.  It wouldn't have to be Bitcoin-Qt, and doesn't have to be on localhost.   I recognize some users might consider the duplication an issue, but I think the benefits outweigh the drawbacks -- and it's on the critical path to implementing a lite-mode option (and hopefully pruned mode).

In case you missed it above, if you are experiencing any of the network issues, please download and install the following:

http://dl.dropbox.com/u/1139081/ArmoryTestingReleases/armory_0.82.5_win32.msi

Run with the " --netlog" option for a while (through a couple blocks) and send me the log.  This is dual-testing for me if you are on a 64-bit system, as I need to find out if 32-bit version works reliably on 64-bits.  Especially because I can't compile a 64-bit version at the moment (damn you py2exe!)

[XertroV]

I've got a small bug report / request (it's only a bug in very particular network configurations, like my Uni's).

It seems that when Armory loads it now attempts to download a file (versions and whatnot)

2012-08-20 00:04 (ERROR) -- ArmoryQt.py:782 - Could not access latest Armory version information
2012-08-20 00:04 (ERROR) -- ArmoryQt.py:783 - Tried: http://bitcoinarmory.com/versions.txt

Setup for uni:
Need to VPN to get NAT, otherwise only web access is allowed.
This is done through proxies, but these proxies must be preset ALWAYS. Port 80 is always blocked out (the only exception is the wireless network, which has transparent proxies - now), this means that while Bitcoin-qt is happy and connected, and Armory can talk to Bitcoin-qt, it fails to connect to the internets and so fails to start.

2012-08-20 00:03 (INFO) -- ArmoryQt.py:836 - Setting up networking...
2012-08-20 00:04 (INFO) -- ArmoryQt.py:887 - Internet connection is Available: False
2012-08-20 00:04 (INFO) -- ArmoryQt.py:888 - Bitcoin-Qt/bitcoind is Available: True

With few exceptions every other port is available.

My 'feature request' I suppose would be to enable support for system proxies, or set up an arbitrary port (like 39462). I don't know which would provide greater compatibility.

I'm going to try my hand at creating a patch for ArmoryQt.py, and will post if successful.

(BTW, Thanks for Armory, Alan!)

[etotheipi]

I've got a small bug report / request (it's only a bug in very particular network configurations, like my Uni's).

It seems that when Armory loads it now attempts to download a file (versions and whatnot)

2012-08-20 00:04 (ERROR) -- ArmoryQt.py:782 - Could not access latest Armory version information
2012-08-20 00:04 (ERROR) -- ArmoryQt.py:783 - Tried: http://bitcoinarmory.com/versions.txt

Setup for uni:
Need to VPN to get NAT, otherwise only web access is allowed.
This is done through proxies, but these proxies must be preset ALWAYS. Port 80 is always blocked out (the only exception is the wireless network, which has transparent proxies - now), this means that while Bitcoin-qt is happy and connected, and Armory can talk to Bitcoin-qt, it fails to connect to the internets and so fails to start.

2012-08-20 00:03 (INFO) -- ArmoryQt.py:836 - Setting up networking...
2012-08-20 00:04 (INFO) -- ArmoryQt.py:887 - Internet connection is Available: False
2012-08-20 00:04 (INFO) -- ArmoryQt.py:888 - Bitcoin-Qt/bitcoind is Available: True

With few exceptions every other port is available.

My 'feature request' I suppose would be to enable support for system proxies, or set up an arbitrary port (like 39462). I don't know which would provide greater compatibility.

I'm going to try my hand at creating a patch for ArmoryQt.py, and will post if successful.

(BTW, Thanks for Armory, Alan!)

Unfortunately, I am not too adept with the networking side of things, and what's going on under-the-hood with proxies, VPN, etc.  I've been quite happy that python-twisted and urllib2 do a good job handling all that transparently, but it means I need some time to familiarize myself with how non-standard setups work in order to make it more flexible.

However, if I'm not mistaken, the real problem is Armory going into offline mode when it actually shouldn't.  You need a way to force it to think it's online, even though it can't access google.com to verify (it's got the bitcoind/-qt connection, so it doesn't matter).  For that reason, I will add to the next version a --force-online flag that allows you to over-ride that behavior.

For now, ArmoryQt.py:890 is the line where it determines whether you should be in online mode based on the previous method.  You can comment that out and replace it with "self.isOnline = True" and try rerunning it.  Please try that and tell me whether it works and/or causes any strange behavior!

[XertroV]

Unfortunately, I am not too adept with the networking side of things, and what's going on under-the-hood with proxies, VPN, etc.  I've been quite happy that python-twisted and urllib2 do a good job handling all that transparently, but it means I need some time to familiarize myself with how non-standard setups work in order to make it more flexible.

However, if I'm not mistaken, the real problem is Armory going into offline mode when it actually shouldn't.  You need a way to force it to think it's online, even though it can't access google.com to verify (it's got the bitcoind/-qt connection, so it doesn't matter).  For that reason, I will add to the next version a --force-online flag that allows you to over-ride that behavior.

For now, ArmoryQt.py:890 is the line where it determines whether you should be in online mode based on the previous method.  You can comment that out and replace it with "self.isOnline = True" and try rerunning it.  Please try that and tell me whether it works and/or causes any strange behavior!

Thanks, --force-online would be fantastic!

I ended up changing line 890 (I think, I added some proxy stuff to where it tries to get the HTTP_VERSION_FILE, which didn't seem to help that much; point being the below is 894 on mine. It's around there in any case) to be

self.isOnline = ((self.internetAvail or True) and self.satoshiAvail and not CLI_OPTIONS.offline)

That worked. Curiously, when I started it at some point I checked the log file and it was saying it could indeed connect to both bitcoin-qt and the interwebs. So I'm not sure what was going on there (I was fiddling with Ubuntu's system proxy settings, so that might have helped too.)

I didn't update at the time due to the market, but it was a quick and simple fix.
The other reason why --force-online would be nice is if you use connect= or addnode= in bitcoin.conf to access a node on your LAN with internet access / port forwarding if the client doesn't have direct access itself. (so Internet <-> gateway node <-> client node + armory). In those cases I imagine you could send/receive transactions fine, but you wouldn't have internet access yourself.

In any case, it's all good now.

I'm also running it in a VM because OS X isn't supported. I'm using Mountain Lion atm and will keep trying to compile Armory in my spare time (Lion instructions don't seem to be working too well ). If I can figure anything out on that front I'll be sure to post it (oh, how nice a .app would be! To the computer box!)

[ragnard]

I just installed 0.82.2 and, after encountering, and fixing, the error with connecting to bitcoin-qt which is running via TOR, I succesfully launched Armory Client.  I am running Win7 x86 on Virtual Box.  It took Armory quite some time to load up, as indicated on the Armory website.  Once it launched, it says it is Connected but only (117519 blocks).  There are currently over 195,000 blocks.  So, I sent a test .05 BTC from my main wallet (not in Armory) to the newly created wallet in Armory.  It came through nearly immediately.

I was still concerned with Armory still only showing 117519 blocks, so I closed it and rebooted Windows.  Now that everything is back up and running (bitcoin-qt and Armory), Armory still only shows 117519 blocks and my wallet balance is 0.  What happened to that .05 BTC?

Also, Armory somewhat regularly locks at 90-100% CPU usage for a minute or two.  Windows says it is Not Responding, but if I wait the couple minutes it comes back to life only to do it again later.

[etotheipi]

I just installed 0.82.2 and, after encountering, and fixing, the error with connecting to bitcoin-qt which is running via TOR, I succesfully launched Armory Client.  I am running Win7 x86 on Virtual Box.  It took Armory quite some time to load up, as indicated on the Armory website.  Once it launched, it says it is Connected but only (117519 blocks).  There are currently over 195,000 blocks.  So, I sent a test .05 BTC from my main wallet (not in Armory) to the newly created wallet in Armory.  It came through nearly immediately.

I was still concerned with Armory still only showing 117519 blocks, so I closed it and rebooted Windows.  Now that everything is back up and running (bitcoin-qt and Armory), Armory still only shows 117519 blocks and my wallet balance is 0.  What happened to that .05 BTC?

Also, Armory somewhat regularly locks at 90-100% CPU usage for a minute or two.  Windows says it is Not Responding, but if I wait the couple minutes it comes back to life only to do it again later.

Unfortunately, I can't comment on how Armory behaves behind proxies and/or Tor.  However, if it stops loading at the same block every time, my guess is it's reading an old blk0001.dat file, and actually has nothing to do with the proxy.  Do you keep your bitcoind/-qt datadir somewhere other than the default location?  If I had to guess, I'd say you used to have bitcoin-qt in the default location, and later moved it out. 

If so, use the " --satoshi-datadir=/path/to/bitcoinqt/homedir" to tell Armory where to find it.  This is because Armory requires the blk000X.dat files created by the currently-running version of bitcoin-qt. 

The transaction showed up in Armory because it saw the zero-confirmation tx forwarded by bitcoind.  But it will never see it enter the blockchain, because it's stuck on block 117,519, and obviously your tx is going to enter the blockchain much later than that.

[etotheipi]

Btw, for anyone having connection issues with (standard) bitcoind/-qt, I think I've found a bug in the connection code.  It will only fix startup-connection issues, but I'm hoping all the networking issues are related and will all go away with this (I can't replicate them, so I can't test it).  I have compiled a windows installer with the bug-fix:

http://dl.dropbox.com/u/1139081/ArmoryTestingReleases/armory_0.82.5-alpha_netfix.msi

If you are in Linux, you can check out the "dev" branch and recompile. 

[etotheipi]

Random update: after two months of working my ass off at work-work and getting engaged (), I am finally getting some time to dive back into Armory development.  For real.  And the first thing I'm doing is... not fixing startup times... but I'm fixing the startup experience by putting the blockchain loading into a background thread.   Armory will open immediately and operate like offline mode while the blockchain data is loading.  Then switch to online mode when it's done.  I already have a proof-of-concept for this, I just need to redesign some of the interface to accommodate mixed-online-offline mode.

This should be a massive, all-around usability improvement for Armory.  Especially if you only want to load it in order to get a new receiving address.   This will not only improve startup experience, but it'll be better for key-importing, too (which will probably switch you to offline mode while you wait for the scan to finish, but you can still use offline-capable features).

I'm going to spend a large chunk of this weekend doing that, and maybe I'll even be able to look at OSX packaging (pending py2app behaving itself).  Hell, if I got both those done, I think that would be enough for me to be comfortable declaring Beta!