Armory - Discussion Thread

[etotheipi]

Quick question... I sent a transaction at roughly the same time my modem lost its connection, now it's sitting there in sent/unconfirmed status. Is there any way to force a transaction to be rebroadcast? That could be a useful advanced feature to have.

Also, it just so happens the transaction in question was to address 1ArmoryXcfq7TnCSuZa9fQjRYwJ4bkRKfv

Ack!  This is an unfortunate situation -- one I haven't figured out how to handle yet (gracefully) with Armory.  The problem is that Armory sends the tx to Bitcoin-Qt, which then adds it to its memory pool and "forwards" it to all its peers.  If that happens to be zero peers because connection is bad, then tough luck.

The issue is that if Armory tries to re-send the tx, it sends it to the Satoshi client which says "Oh, I've already seen this" and drops it immediately without re-forwarding.  So far, I haven't figured out a graceful way to get around this.  No matter what Armory does, ibitcoind/-qt won't re-broadcast it.

The non-graceful way is to restart Bitcoin-Qt (which should clear its memory pool), and stop Armory, delete armoryhomedir/mempool.bin and then restart Armory.  Then try again.

If you are familiar with the RPC interface to Bitcoin-Qt, I haven't tried the following, but I understand it should work:  double-click on the transaction in the Armory ledger, and if you are in advanced or expert user mode, there should be a "Copy Raw Transaction" button at the bottom.  You can copy this into the bitcoind/-qt RPC interface using "sendrawtransaction".  I've never done it, but gmaxwell tells me that will always re-broadcast, even if the tx is already in the bitcoind/-qt memory pool.

I really want to find a good way to resolve this issue, but I haven't found it yet...

[cypherdoc]

...

so the unsigned tx has is located in the unencrypted data portion of the USB stick and can be accessed after you've booted up a desktop computer from the encrypted Linux OS on the USB stick, correct?  but where is the Satoshi client/blockchain that you need Armory to access?

running two separate computers is the very idea of Armory (one for the offline wallet and one for the watching-only wallet).

With my USB stick I could do everything on the same PC:
- boot normally from harddisk: internet connection, bitcoind and Armory watching-only wallet. Create unsigned tx and save to the fat partition on the stick
- boot from USB, sign the tx with offline Armory (this needs no block chain)
- reboot from harddisk and send the signed transaction

whenever the system is booted from its harddisk the wallet on the encrypted part of the stick is securely hidden

now that is a cool solution.  just to make sure i understand; when you boot from the USB, its like a Live CD session where everything is in RAM.  nothing touches the hard drive, right?

[chrisrico]

All the instructions/files here
https://gist.github.com/3646033

Thanks for the awesome instructions. Now I, too have Armory running on my Pi! I sent half a bitcoin to your donation address.

etothepi, is it currently possible or would it difficult to implement command line transaction signing? For instance...

Code:

$ python armoryengine.py --sign-tx test.unsigned.tx

Prompting for the encryption password if encrypted, and wallet ID if it can't determine that on its own.

It could then either prompt to sign with information about the transaction, or just sign it and display that information.

Otherwise I have to get a USB hub, because with mouse and keyboard plugged in to the Pi, I have no USB ports left for transferring the files on USB drive.

[Herbert]

Is there a specific reason you link against the static python libs? On my Linux box (Arch Linux) only the dynamic libs are availble (*.so). I changed the Makefile to use the .so files and it seems to work so far.

[etotheipi]

All the instructions/files here
https://gist.github.com/3646033

Thanks for the awesome instructions. Now I, too have Armory running on my Pi! I sent half a bitcoin to your donation address.

etothepi, is it currently possible or would it difficult to implement command line transaction signing? For instance...

Code:

$ python armoryengine.py --sign-tx test.unsigned.tx

Prompting for the encryption password if encrypted, and wallet ID if it can't determine that on its own.

It could then either prompt to sign with information about the transaction, or just sign it and display that information.

Otherwise I have to get a USB hub, because with mouse and keyboard plugged in to the Pi, I have no USB ports left for transferring the files on USB drive.

That could probably be done.  I already have a python script posted in the Armory on web servers thread about creating and signing transactions.  It should be easy to modify it and use it for signing-only.  You will have to supply the wallet file, too, since armoryengine doesn't auto-detect wallets (that's the GUI).  But it's very doable.  Eventually, I can add some kind of functionality like that.

Is there a specific reason you link against the static python libs? On my Linux box (Arch Linux) only the dynamic libs are availble (*.so). I changed the Makefile to use the .so files and it seems to work so far.

It's because if I compile dynamically, the pre-compiled versions that I produce to distribute (the *.deb) only work on systems with the same python version as I used to compile it.  If you remember, I used to have an armory_0.XX.X-alpha_python2.6.deb and _python2.7.deb.  I solved that by static compiling.  I have been meaning to add something to the Makefile to autodetect when you don't have the .a, and look for the .so instead.  i.e. static compiling was done for me when producing the installers, but isn't necessary for those that are compiling it themselves.  I'll see if I can fix that

[etotheipi]

I was thinking more about the command-line signing.  It will actually be super easy.  I will come up with something more integrable later, but if you are running python on the offline computer and already have the wallet file and unsigned.tx file, you can make a short script like this.

NOTE: I haven't tested the script, so it probably has a few bugs in it.  I will test and update when I get home.  Though, if you are feeling adventurous, you are welcome to try it out for me  I will edit this post when I iron it out.

Code:

#! /usr/bin/python

import getpass
from sys import argv
import os

# Do not ever access the same wallet file from two different processes at the same time
raw_input('PLEASE CLOSE ARMORY BEFORE RUNNING THIS SCRIPT!  (press enter to continue)')

if len(argv)<3:
   print 'USAGE: %s <wallet file> <unsigned.tx file>' % argv[0]
   exit(0)

if not os.path.exists(argv[1]):
   print 'Wallet file was not found: %s' % argv[1]

if not os.path.exists(argv[2]):
   print 'Transaction file was not found: %s' % argv[2]

wlt = PyBtcWallet().readWalletFile(argv[1])

# If the wallet is encrypted, get the passphrase
if wlt.useEncryption:
   print 'Please enter your passphrase to unlock your wallet: '
   passwd = SecureBinaryData(getpass.getpass())
   while not wlt.verifyPassphrase(passwd):
      print 'Passphrase was incorrect!'
      passwd = SecureBinaryData(getpass.getpass())
   wlt.unlock(securePassrphase=passwd)
   passwd.destroy()

# Now read the txdp  
txdp = PyTxDistProposal().unserializeAscii( open(argv[2],'r').read())

try:
   wlt.signTxDistProposal(txdp)
except:
   print 'Error signing transaction.  Most likely this is not the correct wallet.'
   raise
     
if not txdp.checkTxHasEnoughSignatures():
   print 'Error signing transaction.  Most likely this is not the correct wallet.'
   exit(0)

outfile = open(argv[2].split('.')[:-2] + '.signed.tx', 'w')
outfile.write(txdp.serializeAscii)
outfile.close()

[chrisrico]

NOTE: I haven't tested the script, so it probably has a few bugs in it.  I will test and update when I get home.  Though, if you are feeling adventurous, you are welcome to try it out for me   I will edit this post when I iron it out.

I was actually teaching myself Python last night while hacking pieces of your code together for this purpose, but mine was ten times longer.

I made a few modifications to your script, I've tested it and it works. The new readWallet was so the wallet could be read from an open file, but isn't necessary (I just liked the type=argparse.FileType construct). The changes to pprint are so that a wallet can optionally be passed in to the method, and addresses in the wallet will be marked.

https://gist.github.com/3709993

[etotheipi]

OKAY!  A real script for signing offline transactions.  You could improve offline security by booting your offline system into runlevel 3, manually mounting your USB key, and then using this script to sign.  Without the desktop manager running, there's a whole lot less that could happen when you plug in that USB key...

I just committed the script to the master branch:  "extras/cli_sign_txdp.py".  Here's what the script looks like when you run it:

~/armory$ python extras/cli_sign_txdp.py
USAGE: extras/cli_sign_txdp.py <wallet file> <unsigned.tx file>

~/armory$ python extras/cli_sign_txdp.py   /home/user/.armory/armory_2DTq89wvw_.wallet   armory_test_cli_sign.unsigned.tx

PLEASE CLOSE ARMORY BEFORE CONTINUING!  (press enter to continue)
<enter>

********************************************************************************
PLEASE VERIFY THE TRANSACTION DETAILS BEFORE SIGNING
********************************************************************************
   INPUTS:  (10.01000000)
      1245Fve8ZgXY474JidB3jQQfGUKn6NhxUK          -->           5.01000000
      1245Fve8ZgXY474JidB3jQQfGUKn6NhxUK          -->           5.00000000
   OUTPUTS: (10.01000000)
      1Jkch9f4CR9ZaUk67sqBYoE52dSNG2CYi7          <--           4.00058564
      1338g4yeBt3AtevqsGjKKnHFuvp3UMubfX (change) <--           6.00941436
      Fee                                         <--           0.00000000

Does this look right?  [y/N]: y
Please enter your passphrase to unlock your wallet:
Wallet Passphrase: <NOECHO>
Correct Passphrase.  Unlocking wallet...

Signing was successful!  The signed transaction is located:
   armory_test_cli_sign.signed.tx

Would you like to delete the old unsigned transaction? [Y/n]: n

The *.signed.tx file can now be broadcast from any computer running
Armory in online mode.  Click "Offline Transactions" and "Broadcast".

You can go look at the source code on github.  It's about 100 lines now, because I added the transaction verification (important!) and some more error checking.  This script would also be useful for those who want to learn more about python, and leveraging armoryengine.py for other purposes!

NOTE: I haven't tested the script, so it probably has a few bugs in it.  I will test and update when I get home.  Though, if you are feeling adventurous, you are welcome to try it out for me  I will edit this post when I iron it out.

I was actually teaching myself Python last night while hacking pieces of your code together for this purpose, but mine was ten times longer.

I made a few modifications to your script, I've tested it and it works. The new readWallet was so the wallet could be read from an open file, but isn't necessary (I just liked the type=argparse.FileType construct). The changes to pprint are so that a wallet can optionally be passed in to the method, and addresses in the wallet will be marked.

Learn python.  It's delightful.  I'm convinced it's the only reason I was able to put as much stable functionality into Armory as I have:  I can so easily handle every error and every corner case without thousands of lines of code.  Well, there are thousands of lines of code... but it's not all error-checking  But there are dozens of reasons why everything is easier in python, and you can focus on functionality instead of syntax (don't get me wrong, though, I still personally enjoy C++ and rigorous, strongly-typed languages, but I don't always want to deal with it).

[etotheipi]

TEASER:

For those that don't actually care about the under-the-hood details, you can jump right to the screenshots.  The important part of the upcoming update is this:  
Armory opens immediately, runs everything in the background, and detects the various partial-functionality states and give information about what functionality is currently available and what will be available.  

Very close to done with this last upgrade to Armory before Beta.  I had to tear Armory apart, and rebuild much of the functionality in order to implement multi-threaded Blockchain management.  In case you're not familiar with what that means:  Armory used to be single-threaded -- every line of code I write will execute completely, and return whatever it's supposed to, then goes onto the next line.  The code is very simple, but the GUI will freeze if any heavy processing is performed.  This is why the splash screen seems so buggy, and why you can't touch Armory while it is sweeping a private key.

Multi-threaded means that the blockchain management happens in another thread (usually another CPU-core), and the GUI will keep operating smoothly.  This sounds great, but the complexity of code shoots way up.  No longer does each line of code execute immediately -- in many cases the line of code tells the blockchain manager to do something and then moves onto the next line before it's done (this is called "asynchronous programming").  This means that infrastructure had to be added to have a sane GUI that restricts access to various elements if the blockchain manager is busy, and I can't ever count on data being returned immediately when I ask for it.  I might have to remember that I asked for data, and check back with the blockchain manager later to see if it's available.  This why the upgrade took so long!












This is going to require a ton of testing before I can actually turn it into Beta.  I'm sure I broke a lot of things.  But my initial testing has been extremely positive so far.  Hopefully I'll have a testing release out by tomorrow!

P.S. -- I do not have any animated progress bars or anything like that, yet.  I wanted to get the functionality ironed out before focusing on aesthetics.  I will put that in while I wait for people to help test it!

[ragnard]

Wow!  This is a huge step forward!  Thanks etotheipi!

[Red Emerald]

I got the threading branch built on OSX without any difficulty.  I'll update the brew tap after I do some more testing.

I've gotten a couple errors and a few things don't seem to be working properly.

Code:

Wallet update requested
{'2LYwgBnHD': 0, '34LBoGEGF': 0, '2iDC1hYqd': 0, '2b1i32B43': 0}
  File "ArmoryQt.py", line 3124, in <module>
    os._exit(QAPP.exec_())
  File "/Users/bwstitt/src/BitcoinArmory/qt4reactor.py", line 231, in _iterate
    self.runUntilCurrent()
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/base.py", line 800, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "ArmoryQt.py", line 2834, in Heartbeat
    LOGINFO('New Block! : %d', self.currBlockNum)
  File "/Users/bwstitt/src/BitcoinArmory/armoryengine.py", line 324, in LOGINFO
    traceback.print_stack()
(ERROR) ArmoryQt.py:2875 - Error in heartbeat function
Traceback (most recent call last):
  File "ArmoryQt.py", line 2834, in Heartbeat
    LOGINFO('New Block! : %d', self.currBlockNum)
  File "/Users/bwstitt/src/BitcoinArmory/armoryengine.py", line 320, in LOGINFO
    logstr = msg if len(a)==0 else (msg%a)
TypeError: %d format: a number is required, not NoneType
BDMState: Scanning

My dashboard says "Armory is online!" however the bottom corner still says "Offline."

If I click "Offline Transcations" the "Create New Offline Transaction" button is greyed out.  If I click the Transactions tab, no transactions show up.  If I click "Wallet Properties" the transactions are listed.  I'm pretty sure this is because my bitcoind is behind by a week or so.

If I click "Send Bitcoins" it gives me an error saying "Armory is currently scanning..."  Maybe it should say "Armory is scanning!" on the dashboard.

I'll post more once I get bitcoind up to date.

[etotheipi]

I got the threading branch built on OSX without any difficulty.  I'll update the brew tap after I do some more testing.

I've gotten a couple errors and a few things don't seem to be working properly.

Code:

...

My dashboard says "Armory is online!" however the bottom corner still says "Offline."

If I click "Offline Transcations" the "Create New Offline Transaction" button is greyed out.  If I click the Transactions tab, no transactions show up.  If I click "Wallet Properties" the transactions are listed.  I'm pretty sure this is because my bitcoind is behind by a week or so.

If I click "Send Bitcoins" it gives me an error saying "Armory is currently scanning..."  Maybe it should say "Armory is scanning!" on the dashboard.

I'll post more once I get bitcoind up to date.

Ack!  Threading branch is not ready!  Don't use it!

And of course after I posted yesterday, I ran into a seg fault with the address book.  Strange...  but I've been struggling to get that fixed (I'm not as close to a testing release as I thought...)

I meant for the previous post to be a teaser, not a notification.  I still have quite a bit of functionality left to sort out -- it's nothing compared to the multi-threading infrastructure, but still not enough to be usable yet.

P.S. -- But thanks, RE, for being so proactive about helping me test!  I really do need the help testing, it's just not quite ready yet

[Red Emerald]

I got the threading branch built on OSX without any difficulty.  I'll update the brew tap after I do some more testing.

I've gotten a couple errors and a few things don't seem to be working properly.

Code:

...

My dashboard says "Armory is online!" however the bottom corner still says "Offline."

If I click "Offline Transcations" the "Create New Offline Transaction" button is greyed out.  If I click the Transactions tab, no transactions show up.  If I click "Wallet Properties" the transactions are listed.  I'm pretty sure this is because my bitcoind is behind by a week or so.

If I click "Send Bitcoins" it gives me an error saying "Armory is currently scanning..."  Maybe it should say "Armory is scanning!" on the dashboard.

I'll post more once I get bitcoind up to date.

Ack!  Threading branch is not ready!  Don't use it!

And of course after I posted yesterday, I ran into a seg fault with the address book.  Strange...  but I've been struggling to get that fixed (I'm not as close to a testing release as I thought...)

I meant for the previous post to be a teaser, not a notification.  I still have quite a bit of functionality left to sort out -- it's nothing compared to the multi-threading infrastructure, but still not enough to be usable yet.

P.S. -- But thanks, RE, for being so proactive about helping me test!  I really do need the help testing, it's just not quite ready yet

I did build the threading branch to test.  I realize it isn't production ready.  I guess I should have said, "I'll update the brew tap after more testing/fixes by etotheipi."

Any chance the OSX cppforswig makefile changes are merged into master?  If they are, I'll update my tap to that.

[etotheipi]

I did build the threading branch to test.  I realize it isn't production ready.  I guess I should have said, "I'll update the brew tap after more testing/fixes by etotheipi."

I've really only tested the internet/blockchain/satoshi detection and notifications.  Nothing else has been tested at all... it's possible that everything is broken!  I even made myself a checklist to make sure that I go through every dialog box, and every feature I can, because everything needs to be checked.  I don't recommend anyone use it at all until I've done that.

On that note, after spending hours chasing this seg-fault, I have identified what it is:  SWIG Bug 3403085.  It has to do with list & vector iteration when using the "-threads" option with SWIG (required by the upgrade).  Unfortunately, I use that kind of iteration all over the code base!  Since the bug isn't fixed (to my knowledge), I might just have to work around it, which may include going through all the python code and replacing all such list/vector iteration loops with a workaround. 

Alternatively, if someone is skillful enough to examine the CppBlockUtils_wrap.cxx produced by SWIG and figure out why the seg fault happens, that would be much better than working around it.  Extra credit if you can submit a patch to the SWIG project to fix it!  (I've tried digging into it, but it's all auto-generated code with lots of macros, so I can't even begin to figure out what's going on...)

Any chance the OSX cppforswig makefile changes are merged into master?  If they are, I'll update my tap to that.

What changes are those?  I thought I had already put them in and you had verified that it works...?  Or was it put in a different branch and never merged into master?

[dellech]

...

so the unsigned tx has is located in the unencrypted data portion of the USB stick and can be accessed after you've booted up a desktop computer from the encrypted Linux OS on the USB stick, correct?  but where is the Satoshi client/blockchain that you need Armory to access?

running two separate computers is the very idea of Armory (one for the offline wallet and one for the watching-only wallet).

With my USB stick I could do everything on the same PC:
- boot normally from harddisk: internet connection, bitcoind and Armory watching-only wallet. Create unsigned tx and save to the fat partition on the stick
- boot from USB, sign the tx with offline Armory (this needs no block chain)
- reboot from harddisk and send the signed transaction

whenever the system is booted from its harddisk the wallet on the encrypted part of the stick is securely hidden

now that is a cool solution.  just to make sure i understand; when you boot from the USB, its like a Live CD session where everything is in RAM.  nothing touches the hard drive, right?

it is not necessary to have evrything in RAM - when I boot from USB the encrypted partition may be a writable file system and even contain a swap file if used on a PC with too small RAM.
The only important thing is that this encrypted container is not accessible when the system runs an untrusted/networked OS and I plug in my USB stick.

[Red Emerald]

Any chance the OSX cppforswig makefile changes are merged into master?  If they are, I'll update my tap to that.

What changes are those?  I thought I had already put them in and you had verified that it works...?  Or was it put in a different branch and never merged into master?

I'm not sure what commit the change is from, but it appears that it is in both dev and threading and not master.

I think the part that matters is:

Code:

UNAME := $(shell uname)

ifeq ($(UNAME), Darwin)
   # This configuration assumes you have setup your 10.X.X system like
   # Red Emerald in this thread: 
   #    https://bitcointalk.org/index.php?topic=73648.msg976015#msg976015
    # The makefile updates he shows are already tied into this if/else branch
   SWIG_INC     += -I"/usr/include/python$(PYVER)"
   STATICPYTHON +=   "/usr/lib/python$(PYVER)/config/libpython$(PYVER).a"
else
   SWIG_INC     += -I"$(DEPSDIR)/include/python$(PYVER)"
   STATICPYTHON +=   "$(DEPSDIR)/lib/libpython$(PYVER).a"

endif

[cypherdoc]

...

so the unsigned tx has is located in the unencrypted data portion of the USB stick and can be accessed after you've booted up a desktop computer from the encrypted Linux OS on the USB stick, correct?  but where is the Satoshi client/blockchain that you need Armory to access?

running two separate computers is the very idea of Armory (one for the offline wallet and one for the watching-only wallet).

With my USB stick I could do everything on the same PC:
- boot normally from harddisk: internet connection, bitcoind and Armory watching-only wallet. Create unsigned tx and save to the fat partition on the stick
- boot from USB, sign the tx with offline Armory (this needs no block chain)
- reboot from harddisk and send the signed transaction

whenever the system is booted from its harddisk the wallet on the encrypted part of the stick is securely hidden

now that is a cool solution.  just to make sure i understand; when you boot from the USB, its like a Live CD session where everything is in RAM.  nothing touches the hard drive, right?

it is not necessary to have evrything in RAM - when I boot from USB the encrypted partition may be a writable file system and even contain a swap file if used on a PC with too small RAM.
The only important thing is that this encrypted container is not accessible when the system runs an untrusted/networked OS and I plug in my USB stick.

but once you unencrypt the partition can't malware on the computer reach your private keys?

[etotheipi]

I did build the threading branch to test.  I realize it isn't production ready.  I guess I should have said, "I'll update the brew tap after more testing/fixes by etotheipi."

Okay, threading branch is starting to look a lot better.  I've only played with it on testnet, but so far most things work.  Maybe now is a decent time for those who are anxious to play with it.  That would apply only to Linux users who are compiling it themselves.  I haven't touched this version with Windows yet...

There's some details I'm working out with restoring wallets -- because it may require many scans to figure out how many addresses you've created in your wallet and doing that asynchronously is a bit more complicated than I expected.  I might just have to add a button to increase the key pool, or set the default keypool size to super big (but this wallet format is kinda slow).

Either way, the restore does work if you let it scan right away.  And it looks like all importing and sweeping of keys works, and switched Armory modes correctly.  Some things probably don't work, so don't be surprised.  But you can help me figure out what those things are, while I deal with restoring wallets from paper backup   Be sure to copy log messages for me!

[Joe200]

Could you please add an option to Armory to allow the user to refuse to accept certain coins. If he receives certain coins, he sends them back right away.

Here is the full proposal: https://bitcointalk.org/index.php?topic=110749.0

Thanks.

[etotheipi]

Could you please add an option to Armory to allow the user to refuse to accept certain coins. If he receives certain coins, he sends them back right away.

Here is the full proposal: https://bitcointalk.org/index.php?topic=110749.0

Thanks.

This is not feasible.  99% of users have encrypted wallets, which require unlocking to move any coins once they've been received. Many wallets are offline, and require a bit more work than typing a passphrase.

Second of all, this is not a feature for any baseline Bitcoin client.  This violates one of the core metrics of suitability for a currency: fungibility.  If I have 10 BTC, it will buy me 20 cheeseburgers, regardless of the transaction history of those 10 BTC.  No coins are worth more or less, any more so than $100 bill from my wallet is different than a $100 bill from a bank vault.

In the end, there's no benefit anyway.  If someone steals 1000 BTC they're not going to spend them directly.  They will mix them through 100 addresses with "non-tainted" coins, and 99% of users don't care anyway, so they will accept them as payment and mix them with more good coins from legitimate users.  The end result is that your client no longer believes that any coins in circulation are legitimate, and the only person you've inconvenienced is yourself.

If you would like to explore this idea, modifications could be made to notify you that somehow there's a relevant history to the coins you just received.  I'm sure someone could help you modify Armory to explore this, but I'm up to my eyeballs in core feature development, and making Armory usable at all.  This particular feature would be very low on the priority list.