Armory - Discussion Thread

[dave111223]

Is there any way to speed Armory up?  It takes hours to get online while showing the "Armory is offline while scanning the blockchain" message.

Also I have another question about using Armory.

I did the following:

1) Created a new armory wallet
2) Generated 1000 addresses/keys at www.bitaddress.org (so that I could have a whole bunch of address to use for different transactions)
3) Imported the keys into my armory wallet (imported not swept)
4) Sent some coins to one of these address (using bitcoin client)
5) Armory does it's scanning when opened

The transactions show as 100 confirmations already in the blockchain, but Armory is still shows the balance of both the wallet and the individual address as 0?

[etotheipi]

Is there any way to speed Armory up?  It takes hours to get online while showing the "Armory is offline while scanning the blockchain" message.

How much RAM do you have?  If you have sufficient RAM, it should be about 5-10min on Windows, 2-3 min on Linux.  The cases where I've seen it take longer than that is when the user has less than 2GB RAM.  For now, Armory requires that much RAM.  Blame SatoshiDice for growing the network faster than I could get around to fixing this...

Regardless, a fix is coming -- at the expense of duplicating blockchain data on disk.  I have to start maintaining my own blockchain data, but it will mean Armory's RAM usage will drop dramatically, and will also startup instantly after the first load.  Obviously a big improvement... but I don't know how long before I can have it implemented.  Definitely within the next couple months.  Until then, there's not much I can do for you

[cypherdoc]

Is there any way to speed Armory up?  It takes hours to get online while showing the "Armory is offline while scanning the blockchain" message.

shouldn't take hours.  takes me a few minutes with 8 GB RAM.

[dave111223]

I'm running 3GB of RAM.  Also I added some other questions to my original post, maybe you can help out?

[etotheipi]

I'm running 3GB of RAM.  Also I added some other questions to my original post, maybe you can help out?

If the addresses were imported successfully, Armory will see them after the next full scan.  I realize that's a lot to ask for in your case, but it has no way to know its balance until it does.  It should have seen it the first time.  If not, it's probably a bug, but it most definitely will see it after the next scan (if somehow the addresses didn't get registered in Armory before the last scan).  

3GB used to be enough, but I guess the blockchain has grown enough that even that is borderline, now.  All the more reason I need to get the file-based blockchain management integrated, ASAP.

On a side note:  I recommend not using bitaddress.org private keys.  You are nullifying many of the security benefits of Armory doing so, and it's not necessary -- Armory wallets will generate an infinite number of addresses for you, and you only have to print that one sheet of paper to protect all of them ("Wallet properties"-->"Print paper backup").  If you decide not to use Armory later (say, because I haven't fixed the RAM issue fast enough), you can go into the wallet and "Backup Individual Keys" and it will list all the keys that you have used so far -- then you can copy them out to import into another application.

In fact, importing those addresses won't do you much good anyway.  When you send coins, change is sent back to an address in that infinite chain.  And when you click "Receive Bitcoins", it will give you an address in that infinite chain (the one protected by your paper backup).  Importing keys is really more suited to having vanity addresses, or specific donation addresses you wanted to bring over from other applications, etc.  

[dave111223]

Thanks for the reply.  I've closed/re-opened Armory and will let it scan again, hopefully it picks up the coins on the second attempt.

With regards to the keys I didn't see any option is Armory to generate X number of keys? 

My goal was to have each order on my website use a different address, so that it's easy to see when an order has been paid.  I did not want to generate the keys on the webserver as this is apparently a security risk.  So instead need to pre-generate all the keys, then each time an order comes in I random assign it to one of the unused addresses.

If i'd have done this in Armory instead of bitaddress, wouldn't I have had to click "Receive Bitcoins" 1000 or so times? and then no easy way to export just a list of the addresses (without keys), for in my case importing into a mysql database.

[ErebusBat]

Thanks for the reply.  I've closed/re-opened Armory and will let it scan again, hopefully it picks up the coins on the second attempt.

With regards to the keys I didn't see any option is Armory to generate X number of keys? 

My goal was to have each order on my website use a different address, so that it's easy to see when an order has been paid.  I did not want to generate the keys on the webserver as this is apparently a security risk.  So instead need to pre-generate all the keys, then each time an order comes in I random assign it to one of the unused addresses.

If i'd have done this in Armory instead of bitaddress, wouldn't I have had to click "Receive Bitcoins" 1000 or so times? and then no easy way to export just a list of the addresses (without keys), for in my case importing into a mysql database.

Brainwallet allows you to generate armory codes en-mass for situations like yours: http://brainwallet.org/#chains

[etotheipi]

Thanks for the reply.  I've closed/re-opened Armory and will let it scan again, hopefully it picks up the coins on the second attempt.

With regards to the keys I didn't see any option is Armory to generate X number of keys?  

My goal was to have each order on my website use a different address, so that it's easy to see when an order has been paid.  I did not want to generate the keys on the webserver as this is apparently a security risk.  So instead need to pre-generate all the keys, then each time an order comes in I random assign it to one of the unused addresses.

If i'd have done this in Armory instead of bitaddress, wouldn't I have had to click "Receive Bitcoins" 1000 or so times? and then no easy way to export just a list of the addresses (without keys), for in my case importing into a mysql database.

You're in Armory territory now, and you don't even realize it.  This is exactly where Armory shines, if you are willing to consider a different route than you're taking right now.  I will explain the "correct" way to do this, and if you don't like it, I will tell you how to do it the way you are trying to do it, but I think you'll find your way excessively inconvenient.

The key here is to use  "watching-only wallets" which are copies of a regular Armory wallet that don't have the private keys.  It generates the same infinite chain of addresses as the full wallet, but without the risk of an attacker getting the private keys if they compromise your webserver.  And you can monitor the payments from multiple computers.  It's really got all the convenience and security you can ask for... if you do it the Armory way.

(1) On any computer other than the webserver (preferably offline), create a new Armory wallet.  No need to import any addresses.
(2) After creation print a paper backup!  It's the safest and most convenient way to backup your coins forever.  Digital media can fail, and printing thousands of keys will be a disaster to have to reimport later.
(3) In wallet properties, click "Create watching-only wallet".  (probably use USB key if offline)
(4) Transfer the watching-only wallet to webserver and if the original wallet is offline, also import the watching-only wallet into Armory on your online computer
(5) On the webserver, you can run a very simple script to generate as many addresses as you want, all protected by the paper backup, and without containing any private keys.  Install or download&compile Armory on your webserver, then run this script every time you want a new address (you may have to modify paths appropraitely):

Code:

import sys
sys.path.append('/usr/share/armory') # change to whereever armory is

from armoryengine import *
watchingWallet = PyBtcWallet().readWalletFile('/path/to/your/watch/only.wallet')
print watchingWallet.getNextUnusedAddress().getAddrStr() # or do something other than print it

The beauty of this is that the moment someone sends money to any address that your webserver generates, you will see it in Armory on your primary computer (using either watching-only or full wallet).  And if you keep the full wallet offline, you're about as secure as you can get.  If you want to create addresses manually on some occasions outside the webserver, just create a new wallet on your primary computer and use that (or rather, create a second offline wallet and import that one only on your primary computer).  

If you run a webstore, you absolutely should not be using online/web apps to generate addresses.  There's too many ways for that to be compromised, especially when there are tools like Armory available.  There's a JSON-RPC version of Armory (armoryd) available that will make this easier in the future, but it's not completely stable yet (crashes about once a week).  I'll let you know when it is stable if you want it.



EDIT: Duh!  If you want to access the Armory address chain through JSON-RPC, armoryd.py will work perfectly stable in offline mode (the instability comes from fighting with Bitcoin-Qt over the blk*.dat files when online).  If you, or anyone else reading, wants to access Armory watching-only wallet addresses using JSON-RPC and don't need network connectivity, I will commit an armoryd.py that works in offline mode, shortly.  It's basically already done, I just need to tweak it and test it.

EDIT 2:  So apparently it already works.  You just need to checkout the "testing" branch from the github repo.  On your webserver from the checkout directory:

Code:

$ sudo apt-get install git-core build-essential pyqt4-dev-tools swig libqtcore4 libqt4-dev python-qt4 python-dev python-twisted
$ git clone git://github.com/etotheipi/BitcoinArmory.git
$ cd BitcoinArmory
$ git checkout testing
$ make
$ python armoryd.py /path/to/watch/only.wallet --offline

Then setup your JSON-RPC process to use the username and password in /home/user/.armory/armoryd.conf (the file should have one line, "username:password").  
Then you simply keep sending "getnewaddress" commands and you'll be receiving new addresses every time.

[payb.tc]

(4) Transfer the watching-only wallet to webserver and if the original wallet is offline, also import it into Armory on your online computer

i feel the it here might be ambiguous for newbies, but if you think about it, 'it' refers to the watching only wallet, not the original wallet, as that would be pointless.

[dave111223]

I don't have root access to the webserver, site is just on a shared environment.  So can't install armory on the webserver (it's basically over my head at this point).

I'm still having trouble with my original issue.  I re-opened armory it scanned again, and still showing 0 balance, even though the transaction is now at 120 confirmations.

When I open the armory wallet the address is listed, but says "0" transactions and "..." balance?

And for the brainwallet suggestion, if I generate a whole bunch of addresses (say a 1000) using my Paper Backup key, will I need to import these addresses into my wallet or will they automatically appear in there once funds are sent to the address?

P.S.  Just re-opened and scanned Armory a 3rd time and still no sign of the missing bitcoins.

[etotheipi]

I don't have root access to the webserver, site is just on a shared environment.  So can't install armory on the webserver (it's basically over my head at this point).

I'm still having trouble with my original issue.  I re-opened armory it scanned again, and still showing 0 balance, even though the transaction is now at 120 confirmations.

When I open the armory wallet the address is listed, but says "0" transactions and "..." balance?

And for the brainwallet suggestion, if I generate a whole bunch of addresses (say a 1000) using my Paper Backup key, will I need to import these addresses into my wallet or will they automatically appear in there once funds are sent to the address?

P.S.  Just re-opened and scanned Armory a 3rd time and still no sign of the missing bitcoins.

The "..." balance is when Armory isn't done scanning and/or is offline.   If Armory truly believes balance is 0, it will show 0.0.  The "..." means it doesn't know yet.

Looks like you're stuck with the manual method... You can switch Armory to expert mode from main window in the the "User" menu (don't worry, you don't have to restart).  Then when you open wallet it will tell you how many addresses you have generated.  You can click on it to generate as many more as you want.  However... it's going to require a rescan afterwards, because this isn't a normal use case for Armory wallets (it's usually for searching for transactions that may have occurred beyond the last key generated so far from using the wallet on multiple systems).  But it doesn't matter ... you can still go "Backup Individual Keys" while it is scanning and click "Show unused keys" and print off a list of addresses to transfer to the webserver.  It's annoying to have to move the addresses back and forth, but if you can't put anything on the server, there's not much else you can do...

On the other hand, given the limitations you are experiencing... maybe Armory isn't the best choice for you at the moment.  And I'll take this as a nudge that I need to get the RAM thing sorted out...

[ErebusBat]

Looks like you're stuck with the manual method...

Not exactly true.  He could do a hybrid where he runs the script to get the addresses to import into his SQL table, but still have all the benefits of a deterministic wallet.

However as you said... if he is having trouble with Armory running to get the funds out then perhaps it is not the best way.

Is there a python script like the one you posted for backing up/extracting the private keys as well?  I always like to make sure that I can get access to the private keys in the event that a client goes belly up or simply does not work for some reason.  Right now the only way I know of is to either backup individual keys (which requires forethought) or using brainwallets chaincode feature.

[etotheipi]

Looks like you're stuck with the manual method...

Not exactly true.  He could do a hybrid where he runs the script to get the addresses to import into his SQL table, but still have all the benefits of a deterministic wallet.

However as you said... if he is having trouble with Armory running to get the funds out then perhaps it is not the best way.

Is there a python script like the one you posted for backing up/extracting the private keys as well?  I always like to make sure that I can get access to the private keys in the event that a client goes belly up or simply does not work for some reason.  Right now the only way I know of is to either backup individual keys (which requires forethought) or using brainwallets chaincode feature.

The "backup individual keys" doesn't require forethought.  You can always run Armory in offline mode and "backup individual keys' from there.  That requires no RAM, and will work on any computer that Armory can be installed on. 

If you really want a scripted way to do it, you can follow some of the example I posted above, except instead of getNextUnusedAddress() after loading the wallet file, you would iterate through wallet.addrMap and pull out  "addr.binPrivKey32_Plain".  If the wallet is encrypted, those variables will be empty unless you unlock it first: "wallet.unlock(SecureBinaryData('my_secur3_p4ssphr4se'))".  If you want an interactive script, you can use the "getpass" module to have the terminal request a password from you without echoing the chars.

[ErebusBat]

Thanks for the tips etotheipi.

I took your base script and hacked together this:

Code:

import sys
import getpass
sys.path.append('/usr/local/Cellar/armory-qt/0.86.3-beta/share/armory') # change to whereever armory is

from armoryengine import *
wallet = PyBtcWallet().readWalletFile('/path/to/my/armory.wallet')

def getNewPubAddr():
  return wallet.getNextUnusedAddress().getAddrStr()
  
def getPubAddrAtIndex(i):
  pub_addr = wallet.getAddress160ByChainIndex(i)
  return wallet.addrMap[pub_addr].getAddrStr()

def dumpPubAddrs():
  for i in range(0,25):
    # pub_addr = getNewPubAddr()
    pub_addr = getPubAddrAtIndex(i)
    print "%03d: %s" % (i,pub_addr)
    
def dumpPrivAddrs():
  k = SecureBinaryData(getpass.getpass('decrypt passphrase:'))
  wallet.unlock(k)  # Will throw on error
  for addrObj in self.addrMap.values():
    print "%s" % addrObj.binPrivKey32_Plain
   
dumpPubAddrs()
dumpPrivAddrs()

The public address part works like a champ; however when I specify the correct password I segfault with:

Code:

9   _CppBlockUtils.so             	0x00000001044588f9 CryptoPP::SimpleKeyingInterface::ThrowIfInvalidKeyLength(unsigned long) + 119
10  _CppBlockUtils.so             	0x0000000104458865 CryptoPP::SimpleKeyingInterface::SetKey(unsigned char const*, unsigned long, CryptoPP::NameValuePairs const&) + 31
11  _CppBlockUtils.so             	0x00000001044545f7 CryptoPP::CipherModeFinalTemplate_CipherHolder<CryptoPP::BlockCipherFinal<(CryptoPP::CipherDir)0, CryptoPP::Rijndael::Enc>, CryptoPP::ConcretePolicyHolder<CryptoPP::Empty, CryptoPP::CFB_DecryptionTemplate<CryptoPP::AbstractPolicyHolder<CryptoPP::CFB_CipherAbstractPolicy, CryptoPP::CFB_ModePolicy> >, CryptoPP::CFB_CipherAbstractPolicy> >::CipherModeFinalTemplate_CipherHolder(unsigned char const*, unsigned long, unsigned char const*) + 195
12  _CppBlockUtils.so             	0x0000000104449560 CryptoAES::DecryptCFB(SecureBinaryData&, SecureBinaryData&, SecureBinaryData) + 124
13  _CppBlockUtils.so             	0x0000000104526d21 _wrap_CryptoAES_DecryptCFB + 1125

If I specify a bad password then the verify catches it, does not segfault, and exits.

Given that 'Actual Armory' can unlock and export the wallet I am assuming that this is an issue of 1) me nothing knowing enough python, or 2) me not knowing / understanding the ArmoryEngine well enough.

I know you are busy, but if you could even give me pointers that would be great.  (yes i pointed it at my FULL (not watch only) wallet).

[etotheipi]

Thanks for the tips etotheipi.

I took your base script and hacked together this:

Code:

...

The public address part works like a champ; however when I specify the correct password I segfault with:

Code:

...

If I specify a bad password then the verify catches it, does not segfault, and exits.

Given that 'Actual Armory' can unlock and export the wallet I am assuming that this is an issue of 1) me nothing knowing enough python, or 2) me not knowing / understanding the ArmoryEngine well enough.

I know you are busy, but if you could even give me pointers that would be great.  (yes i pointed it at my FULL (not watch only) wallet).

Whoops, I gave bad advice:  if you don't supply any keywords to "wallet.unlock()" it treats the first argument as the "kdfOutput", instead of the "kdfPassphrase" (i.e. it's expecting a 32-byte decryption key where you put your passphrase).  In actuality, your decryption key is your passphrase passed through the KDF, so you need to call it like this:

Code:

wallet.unlock(securePassphrase=SecureBinaryData('p4$sph4S3'))

The function declaration is here.

It sounds like I need to put a check in there to make sure if someone calls with a non-32-byte input, it will bail instead of passing it to the C++ which will segfault (treats it as 32 bytes no matter what).

If you are going to do more scripting with armoryengine, checkout the "extras" directory.  Specifically, for this task, the command-line tx signing script (cli_sign_txdp.py) has most of what you need in it.

One day I'll be able to make developer docs...

[ErebusBat]

Whoops, I gave bad advice:  if you don't supply any keywords to "wallet.unlock()" it treats the first argument as the "kdfOutput", instead of the "kdfPassphrase" (i.e. it's expecting a 32-byte decryption key where you put your passphrase).  In actuality, your decryption key is your passphrase passed through the KDF, so you need to call it like this:

Code:

wallet.unlock(securePassphrase=SecureBinaryData('p4$sph4S3'))

The function declaration is here.

I had saw the function def; however it was a case of not knowing enough python (just enough to get into trouble).

However I am now running into the problem of converting it from a binary string into a base58 string.

I found the section of code where it outputs in the GUI, here; however despite having the "from armoryengine import *" line, I get the following error:

Code:

  File "export_keys.py", line 30, in dumpPrivAddrs
    priv_key = encodePrivKeyBase58(addrObj.binPrivKey32_Plain.toBinStr())
NameError: global name 'encodePrivKeyBase58' is not defined

Traceback (most recent call last):
  File "export_keys.py", line 34, in <module>
    dumpPrivAddrs()
  File "export_keys.py", line 30, in dumpPrivAddrs
    priv_key = encodePrivKeyBase58(addrObj.binPrivKey32_Plain.toBinStr())
NameError: global name 'encodePrivKeyBase58' is not defined

I can find the definition in armoryengine.py, but I am not sure why it is not importing into the namespace?  (Full script listing here.

Thanks for hand holding stupid noob python questions.

[etotheipi]

Whoops, I gave bad advice:  if you don't supply any keywords to "wallet.unlock()" it treats the first argument as the "kdfOutput", instead of the "kdfPassphrase" (i.e. it's expecting a 32-byte decryption key where you put your passphrase).  In actuality, your decryption key is your passphrase passed through the KDF, so you need to call it like this:

Code:

wallet.unlock(securePassphrase=SecureBinaryData('p4$sph4S3'))

The function declaration is here.

I had saw the function def; however it was a case of not knowing enough python (just enough to get into trouble).

However I am now running into the problem of converting it from a binary string into a base58 string.

I found the section of code where it outputs in the GUI, here; however despite having the "from armoryengine import *" line, I get the following error:

Code:

  File "export_keys.py", line 30, in dumpPrivAddrs
    priv_key = encodePrivKeyBase58(addrObj.binPrivKey32_Plain.toBinStr())
NameError: global name 'encodePrivKeyBase58' is not defined

Traceback (most recent call last):
  File "export_keys.py", line 34, in <module>
    dumpPrivAddrs()
  File "export_keys.py", line 30, in dumpPrivAddrs
    priv_key = encodePrivKeyBase58(addrObj.binPrivKey32_Plain.toBinStr())
NameError: global name 'encodePrivKeyBase58' is not defined

I can find the definition in armoryengine.py, but I am not sure why it is not importing into the namespace?  (Full script listing here.

Thanks for hand holding stupid noob python questions.

It doesn't work because I only recently added the "encodePrivKeyBase58" function and you are using an older version (up until now, I was basically rewriting the encode code everywhere I needed it).   It is available in the latest download, 0.87.

[ErebusBat]

It doesn't work because I only recently added the "encodePrivKeyBase58" function and you are using an older version (up until now, I was basically rewriting the encode code everywhere I needed it).   It is available in the latest download, 0.87.

Quite right.  I am on OSX so I leave the voodoo of making Armory work up to Red Emerald and the brew formula isn't updated for 0.87 yet.   I had thought that I checked my actual installed Armory code; however I must have been looking at my HEAD checkout, sorry about that.

I have pushed my script changes to github.  For anyone who is interested the (very rough) script can:

• Dump all 'created' public address
• Generate new public addresses
• Dump imported pub/priv keys
• Dump generated pub/priv keys

Right now is outputs in pseudo-csv format. I say pseduo because Armory logs to stdout so you have to do some 'magic' to get a plain CSV file:

Code:

# I am on OSX so I have to invoke python-for-armory in a voodoo way
PYTHONPATH=`brew --prefix`/lib/python2.7/site-packages /usr/bin/python export_keys.py | tail -n +10  > keys.txt

# Normal people would probably be able to just do:
/usr/bin/python export_keys.py | tail -n +10  > keys.txt

You can access the script here: https://github.com/ErebusBat/BitcoinArmory/blob/master/extras/export_keys.py with the same license as Armory.

Question:  I didn't use a test wallet for this and now my chain is extended by several hundred addresses.  Is there a (quick) way I can reset the highest address count back to something sane?  EDIT: nm, found wallet.rewindHighestIndex

Second question:  I appear to be getting differing results for the .chainIndex value, is that to be expected?  See below for an example of tailing output of a dump of the public keys, then a call to new which produces an index almost -100 behind.

EDIT 2: nm again... looks like i was using lastComputedChainIndex rather than highestUsedChainIndex.  Out of curiosity, what is the difference?

[ErebusBat]

It's me again.....

Is it possible to use Armory with testnet-in-a-box?

I tried but got the following output:

Code:

$ PYTHONPATH=`brew --prefix`/lib/python2.7/site-packages /usr/bin/python /usr/local/Cellar/armory-qt/0.86.3-beta/share/armory/ArmoryQt.py --testnet --satoshi-port=19000 --satoshi-datadir /Users/aburns/src/bitcoin/bitcoin-testnet-box/1/testnet3
********************************************************************************Loading Armory Engine:   Armory Version:       0.86.3
   PyBtcWallet  Version: 1.35
Detected Operating system: Mac/OSX
   User home-directory   : /Users/aburns/Library/Application Support
   Satoshi BTC directory : /Users/aburns/src/bitcoin/bitcoin-testnet-box/1/testnet3
   First blkX.dat file   : /Users/aburns/src/bitcoin/bitcoin-testnet-box/1/testnet3/blk0001.dat
   Armory home dir       : /Users/aburns/Library/Application Support/Armory/testnet3
Setting netmode: 1
Number of registered addr: 10
Opening file 1: /Users/aburns/src/bitcoin/bitcoin-testnet-box/1/testnet3/blk0001.dat
Highest blkXXXX.dat file: 1
Attempting to read blockchain from file: /Users/aburns/src/bitcoin/bitcoin-testnet-box/1/testnet3/blk0001.dat
/Users/aburns/src/bitcoin/bitcoin-testnet-box/1/testnet3/blk0001.dat is 0.550011 MB
***ERROR:  Block file is for the wrong network!
           MagicBytes of this file: fabfb5da

Obviously I can patch this check, but I just wanted to know if there was an easier way than hard patching the client?

[goatpig]

Today I tried to send btc. Had a hard time remembering the passphrase and that got me into a little bit of trouble

I was using an old version at that point, something like 0.7x, those that would give you the "loading blockchain info" splash screen still. So I start a transaction, put in the address, the amount, click send, confirm and here it asks for my passphrase. Here I wrote a wrong one and it froze armory. I killed it then restarted, tried again, same thing.

I then updated to 0.87 beta. I start it, let it read the block chain, procede to send coins, write in a wrong passphrase and it freezes on me again.

Round 3, start armory, let it load the chain, go to the wallet menu by double clicking on it and attempt the change the passphrase. Write in the wrong one, freeze.

Round 4, start armory, attempt the change passphrase before the block chain is loaded, write in the wrong one, long wait, then tells me the passphrase is wrong. Keep trying to change the passphrase until it finishes loading the chain. At that point I attempt the change the passphrase, expecting a crash, but it doesn't, and I manage to "brute force" through my passphrase and send my btc.