Bitcoin Forum
April 26, 2024, 11:35:03 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 4 5 »  All
  Print  
Author Topic: [ANN] BitcoinArmory-Daemon - armory on web servers  (Read 20535 times)
mav (OP)
Full Member
***
Offline Offline

Activity: 169
Merit: 107


View Profile
July 10, 2012, 05:09:49 AM
Last edit: September 17, 2012, 04:53:03 AM by mav
Merited by ABCbits (6)
 #1

Armory Daemon is now available on github:

https://github.com/thedawnrider/BitcoinArmory-Daemon

More info on page 2 of this thread - https://bitcointalk.org/index.php?topic=92496.msg1197291#msg1197291

-------------------------

I thought I'd start this thread as a place to collate information about using Armory client on a webserver, since the only info I found so far was hidden on page 16 of the Armory Discussion topic.

The info below is about getting an Armory address on a webserver and it comes from https://bitcointalk.org/index.php?topic=56424.msg765880#msg765880

Anyone who has information or questions about using Armory on a webserver, please post it here so webdevs can make the most of Armory, which currently seems to be focusing on GUI interaction rather than API (understandably).




I put some thought into how Armory could be used for a webserver application, and I realized that with just a couple lines of python, you can generate addresses no problem on your webserver.  I added a couple convenience functions and committed them to the master branch of Armory. Here's what you do:

(1) Load Armory on your computer.  
(2) Create a regular wallet.  
(3) In Wallet Properties, "Create Watching-Only Copy"
(4) Upload watchonly wallet to webserver
(5) Setup webserver to load wallet and call wlt.getNextUnusedAddress() for each request
(6) Sit at home with the full wallet and watch/confirm/send transactions while the webserver does the work!

Demonstration code to see how to access/manipulate the wallet from a python script:

Code:
from armoryengine import *

wlt = PyBtcWallet().readWalletFile('/home/server/web_server.watchonly.wallet')
wlt.setAddrPoolSize(1000)

# Quick printing of addr info, and shows how to access various props of the addr
def pprint(addr):
   print 'The new address: '
   print '   Hash160:  ', binary_to_hex(addr.getAddr160(), BIGENDIAN)
   print '   Base58:   ', addr.getAddrStr()
   print '   Index:    ', addr.chainIndex
   print '   Have Pub: ', addr.hasPubKey()
   print '   Have Priv:', addr.hasPrivKey()
  
print 'Get the next unused address...'
addr = wlt.getNextUnusedAddress()
pprint(addr)

print 'Generating 10 more addresses:'
for i in range(10):
   addr = wlt.getNextUnusedAddress()
   print '\t', addr.chainIndex, '\t', addr.getAddrStr()

print 'Get addr #15'
a160 = wlt.getAddress160ByChainIndex(15)
print binary_to_hex(a160, BIGENDIAN)
addr = wlt.addrMap[a160]
pprint(addr)

print 'This wallet has %d addresses computed' % wlt.getHighestComputedIndex()
print 'Addresses used: %d ' % wlt.getHighestUsedIndex()

Here's the output on a watching-only wallet (this is actually the output of the second time I ran it):

Code:
Get the next unused address...
The new address:
   Hash160:   728b0b8cc930cb4756328e743b7b2e7dde93a35f
   Base58:    mpEeSrEr3EiyCMRskT5VrELkY8X9ZwT3BV
   Index:     11
   Have Pub:  True
   Have Priv: False

Generating 10 more addresses:
12 mrhzBHQcn7jmhvpbcfW6ebiAFEm9qZF6AL
13 moPM3KPygygsWzvUHfeDsmPChL5xWycc5y
14 n4ZtG47TNMifGJ57msGAs8ZD1HeVqSqRyx
15 mw3cxXKCaKitV8w1wBRrpjQNWu4od5Nd5H
16 n1UGPkUgUXiwsotzfpAezpy6NkRg1fGMv5
17 muFPi3pAw7Rbb9aGc9UJPX9m6JT1VGHf9J
18 n3TRNc86Vmi2EygvjeMsAXXAYhPygR2sQK
19 mikSy5FjeSPVBSaXDs8ihMKrMgZsoohsjz
20 mwb8h2PJvGGGQTA7QBgvPVrWowQZi43ZD2
21 mpBFZ1H6AcRqHUZ7ETz1Xh1PqaPiMvdQDM

Get addr #15
c42ea65330263fd9c02fe5ca2e3a1c44dca356aa
The new address:
   Hash160:   c42ea65330263fd9c02fe5ca2e3a1c44dca356aa
   Base58:    mw3cxXKCaKitV8w1wBRrpjQNWu4od5Nd5H
   Index:     15
   Have Pub:  True
   Have Priv: False

This wallet has 1021 addresses computed
Addresses used: 21

Your webserver can run blissfully ignorant about what is actually going on with the wallet, it just gives out a new address on every request.  You can sit at home on your computer with Armory running and the full wallet there, with full control.  But someone who compromises the webserver will not get anything!

If you want the webserver to track incoming transactions, it will require loading the blockchain and checking for new blocks periodically.  It's not a lot of code, but I'll save it for later if you are interested.

One other thing:  Armory uses the crypto++ library, which is not particularly fast at ECDSA operations (and my key generation algorithm is kinda slow, which is why I'll be upgrading to a new one, soon).   My [decent] system can generate about 200 addr/sec.  So this script takes a few sec to run.  But all keys are stored in the wallet, so they don't have to be recomputed when you restart the script.
1714174503
Hero Member
*
Offline Offline

Posts: 1714174503

View Profile Personal Message (Offline)

Ignore
1714174503
Reply with quote  #2

1714174503
Report to moderator
1714174503
Hero Member
*
Offline Offline

Posts: 1714174503

View Profile Personal Message (Offline)

Ignore
1714174503
Reply with quote  #2

1714174503
Report to moderator
Transactions must be included in a block to be properly completed. When you send a transaction, it is broadcast to miners. Miners can then optionally include it in their next blocks. Miners will be more inclined to include your transaction if it has a higher transaction fee.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1714174503
Hero Member
*
Offline Offline

Posts: 1714174503

View Profile Personal Message (Offline)

Ignore
1714174503
Reply with quote  #2

1714174503
Report to moderator
mav (OP)
Full Member
***
Offline Offline

Activity: 169
Merit: 107


View Profile
July 12, 2012, 08:15:57 AM
Last edit: July 12, 2012, 09:40:49 AM by mav
 #2

Here is some more info which I found from testing today.

I wanted to cut the armory source down to remove dependency on qt, and thus run just a simple script to access the watchonly wallet on my web server without installing the whole of armory.

My goal in this post is to get new addresses from a watch-only wallet using python, on a server without qt installed.

Steps:

From the source at https://github.com/etotheipi/BitcoinArmory I took only the following file:

  • armoryengine.py

There were two more files I needed, which are not in the github repo (presumably compiled on install), so I got them from my armory install, located in /usr/share/armory and added them to the folder containing the above three files.

  • CppBlockUtils.py
  • _CppBlockUtils.so

Note there is no dependency on qt for any of these files.

I copied a watchonly wallet from armory into the folder containing the specially selected files above, and created a test.py file as below. It worked! The test script read the wallet file and created / printed the details of the next address.

Code:
from armoryengine import *
wlt = PyBtcWallet().readWalletFile('test.watchonly.wallet')
addr = wlt.getNextUnusedAddress()

print 'The new address: '
print '   Hash160:  ', binary_to_hex(addr.getAddr160(), BIGENDIAN)
print '   Base58:   ', addr.getAddrStr()
print '   Index:    ', addr.chainIndex
print '   Have Pub: ', addr.hasPubKey()
print '   Have Priv:', addr.hasPrivKey()

This allows the web server to run a light-weight non-gui version of the watch-only capabilities of armory, which utilises the watch-only wallet.

Note that this code does not seem to allow interaction with the bitcoin network, only with the wallet. Thus, getting the balance of an armory address from the watchonly wallet is not possible with this code (which is my next task to figure out). Any suggestions how to go about it are most welcome. I imagine this functionality should be possible from the satoshi bitcoin client, but I have yet to figure out how to get the balance of an address not in the current (satoshi) wallet. I'm also looking into how armory connects to bitcoind and how it gets balances for addresses.

Any suggestions or comments are most welcome, this is all new to me.
davout
Legendary
*
Offline Offline

Activity: 1372
Merit: 1007


1davout


View Profile WWW
July 12, 2012, 09:20:42 AM
 #3

I would totally donate a few coins for an Armory port to Ruby

mav (OP)
Full Member
***
Offline Offline

Activity: 169
Merit: 107


View Profile
July 12, 2012, 10:06:20 AM
 #4


Note that this code does not seem to allow interaction with the bitcoin network, only with the wallet. Thus, getting the balance of an armory address from the watchonly wallet is not possible with this code (which is my next task to figure out). Any suggestions how to go about it are most welcome. I imagine this functionality should be possible from the satoshi bitcoin client, but I have yet to figure out how to get the balance of an address not in the current (satoshi) wallet. I'm also looking into how armory connects to bitcoind and how it gets balances for addresses.


I found an answer for these questions and have posted some code about how to get the balance of an address.

Calling BDM_LoadBlockchainFile is very slow (tens of seconds on my machine) since I'm guessing it loads and parses the entire blockchain. I'm going to look into how to incorporate that command (probably with some others to make it useful) and create a kind of 'bitcoind' for armory, which will be able to access the watch-only armory wallet, and also be able to get balances for that wallet. It seems from the comments in the armory source code that any address balance can be easily fetched, not just addresses in the wallet (can't find the comment sorry).

Code:
from armoryengine import *

wlt = PyBtcWallet().readWalletFile('armory_2hdJwWB5h_.watchonly.wallet')

addr = wlt.getNextUnusedAddress()
print 'The new address: '
print '   Hash160:  ', binary_to_hex(addr.getAddr160(), BIGENDIAN)
print '   Base58:   ', addr.getAddrStr()
print '   Index:    ', addr.chainIndex
print '   Have Pub: ', addr.hasPubKey()
print '   Have Priv:', addr.hasPrivKey()

BDM_LoadBlockchainFile()

print wlt.getAddrBalance(addr.getAddr160())  #This prints the address balance from the address above

As for a Ruby port, sorry I'm not going to be much help cause I don't know any ruby. Perhaps someone else will step up and investigate. Possibly if a bitcoind style implementation of armory can be developed, with a json api, perhaps your ruby could access armory through that interface rather than as a direct language port. Time will tell.
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
July 13, 2012, 04:58:00 AM
 #5

Hah!  I didn't even notice this thread.  I'll offer lots of advice for you...

Most importantly, checkout extras/sample_armory_code.py.   That has a lot of details about how to load the blockchain and access address balances.  Constructing transactions isn't so obvious, but it's do-able.

I'm kinda swamped this month, but I'm subscribed now and will provide tips on using the armoryengine.py where I can.

Btw, I did intend for armoryengine.py + _CppBlockUtils.so/.dll/.pyd to be a nice standalone, full-featured library.  I'm pretty sure that all the ArmoryQt.py and qtdialogs.py code only uses armoryengine.py, but does not expand it (thus, anything that Armory does, you can do with just those files).  There's no JSON interface yet, but I think I can add it...

 
P.S. -- The dependencies for just armoryengine+cppBlockUtils is:

Code:
build-essential 
swig
python-dev
python-twisted

All of it is in the Ubuntu repos.  Doing this in Windows is far from easy, but I'm pretty sure the original instructions posted 6 months ago still work.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
mav (OP)
Full Member
***
Offline Offline

Activity: 169
Merit: 107


View Profile
July 13, 2012, 11:38:58 AM
 #6

Anticipating that servers often create multiple threads to handle concurrent requests, I wanted to get a feel for how armoryengine would respond to attempts at fetching many addresses at the same time. I ran the following test code and it borked my watch-only wallet file, to the point where I couldn't even get an address out of it without threading.

Code:
from armoryengine import *
from threading import Thread

def get_addr():
    wlt = PyBtcWallet().readWalletFile('test.watchonly.wallet')
    addr = wlt.getNextUnusedAddress()
    print addr.getAddrStr()

for a in range(10):
    Thread(target=get_addr, args=()).start()

Whenever I call readWalletFile on a borked wallet file, with or without threading, I get the following error:

Code:
***ERROR:  Requested wallet is for a different blockchain!
           Wallet is for:   

.../armoryengine.py", line 6907, in unpackHeader
    print '           Wallet is for:   ', BLOCKCHAINS[self.magicBytes]


The purpose of this post is just to gather the info, I will look more into how to handle this in the near future. Any suggestions from wise heads are welcome.
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
July 13, 2012, 07:40:28 PM
 #7

This is why I prevent Armory from opening twice.  Wallet operations are not threadsafe. Not at all!  

However, one benefit of python-twisted is that it is guaranteed to run as a single-thread, so as long as there is only one instance of Armory, it's perfectly safe.

For your purposes, not using python-twisted, need to do something else.  Actually, I would recommend using python-twisted to create a server/daemon process to handle (serially) all requests for wallet interaction.  Make sure everything goes through that interface.  The wallet design has been remarkably robust (I even test it with mid-write interrupts and it recovers fine).  But as you can see, two simultaneous operations can break it pretty easily.  


Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
mav (OP)
Full Member
***
Offline Offline

Activity: 169
Merit: 107


View Profile
July 16, 2012, 02:21:14 AM
 #8

Indeed, twisted seems to be the way to go. I've written a twisted web server which allows for threaded interaction with the armory wallet and blockchain.

I'm looking for feedback about what functionality to incorporate (and in what format) from people who may want to use this.

Currently I've got a basic web server with the following url structure (only basic calls for my needs so far but I plan to extend it further)

URLDESCRIPTIONARGSOPTIONAL defaultCONSIDERATIONS
/wallet/getAddressgets the next unused address from the watch-only wallet
/blockchain/getAddressBalancegets the balance of an address (not necessarily in your wallet)address=XbalType=Spendable | unconfirmed | ultimateUnsure to use balType from armory client or minconf from satoshi client

My doubt with the url design is whether to emulate as closely as possible the internal class structure of armoryengine or whether to design an entirely new abstraction with wallet and blockchain etc (like above table) which is independent of the wallet type (satoshi / armory etc). My thoughts right now are to replicate the internal structure of armory, since this is a wrapper for armory and not a generic application. But then I think perhaps a generic web interface would be good to have...

Any particular feature requests will be given priority over general coverage (once I've satisfied my own needs), so make them in this thread.

Suggestions about how the implementation are most welcome.




Also, I am unsure about the difference between spendable, unconfirmed and ultimate in address balances. Can someone clear this up for me? It didn't behave how I expected. eg:

Into one address I had 1.2 btc transferred with 8 confirmations and 2.1 btc with 1 confirmation, and I got this result:
spendable - 3.3
unconfirmed - 0
ultimate - 3.3
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
July 16, 2012, 02:33:53 AM
Last edit: July 16, 2012, 05:38:45 PM by etotheipi
 #9


Also, I am unsure about the difference between spendable, unconfirmed and ultimate in address balances. Can someone clear this up for me? It didn't behave how I expected. eg:

Into one address I had 1.2 btc transferred with 8 confirmations and 2.1 btc with 1 confirmation, and I got this result:
spendable - 3.3
unconfirmed - 0
ultimate - 3.3


(1) Ultimate/Full includes every, confirmed and not confirmed
(2) Spendable includes everything (SentToSelf || >1 confirmation)
(3) Unconfirmed is (NotSentToSelf && <6 confirmations)

EDIT:  I realize now it would've been clearer to say avoid "Spendable" and just say "Confirmed".  Though maybe not: you send coins to yourself in Armory and they show with 0 confirmations, yet it still shows up as confirmed.  That doesn't fit well with folks conditioned to believe that nothing is confirmed until 1+.  Ehhh... whatever.  I'm leaving it.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
Clark
Hero Member
*****
Offline Offline

Activity: 548
Merit: 502


So much code.


View Profile WWW
August 12, 2012, 09:37:02 PM
 #10

I have been looking for this information on the Armory website (like a developer API section), but I'm sure it will be up there at some point.

Thanks for this! It will be highly useful.

etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
August 12, 2012, 10:22:12 PM
 #11

I have been looking for this information on the Armory website (like a developer API section), but I'm sure it will be up there at some point.

Thanks for this! It will be highly useful.

That kind of documentation has not made it into my priority list, yet.  However, threads like this are specifically for users to share ideas about how they successfully leveraged armoryengine for their purposes.  And while I don't have time to make such documentation (yet), I will respond to such threads and help folks accomplish their goals. 

Also, if you checkout the git project, there is a file called "extras/sample_armory_code.py" which has a lot of example code.  It's not as good as actual documentation, but working example code is quite useful.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
Herbert
Hero Member
*****
Offline Offline

Activity: 488
Merit: 500



View Profile WWW
August 21, 2012, 08:30:08 PM
 #12

So i can basically create as many receiving addresses on the webserver as I want, but without any risk in case the server gets compromised? Awesome!

www.bitcoinmonitor.net - Free payment notification via email, newsfeed, xpmm/jabber, url callback and full API access!
Send SMS with www.txt4coins.net! No registration, pay-per-use, full API access, bulk messages - All inclusive!
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
August 22, 2012, 07:12:53 PM
 #13

So i can basically create as many receiving addresses on the webserver as I want, but without any risk in case the server gets compromised? Awesome!

Exactly.

I recommend using an offline computer to make the full wallet, then make a watching-only copy.  Put it on your webserver, and also import it into Armory on your primary system.  Then your webserver will distribute addresses, and you can see the full transaction history from your desktop.  If someone gets the watching-only wallet... well all they can do is watch, but they can't take any funds!

I don't recommend generating addresses from the watching-only wallet, while the webserver is doing the same.  There won't be any fireworks, but you might end up re-using an address by accident.  If you want to also be able to distribute addresses from your primary system, I recommend making a second wallet on the offline computer just for that purpose.  You can watch them both from Armory, but only distribute addresses from the one the server isn't using.  Then you can also execute offline transactions from either wallet using the online computer.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
mav (OP)
Full Member
***
Offline Offline

Activity: 169
Merit: 107


View Profile
August 23, 2012, 03:45:55 AM
 #14

So i can basically create as many receiving addresses on the webserver as I want, but without any risk in case the server gets compromised? Awesome!

This functionality is very simple to achieve and is outlined in the first post thanks to etotheipi providing the example. This speaks volumes about how nicely armory is designed, it's very easy to work with. I have started on an api interface which currently is able to get the next address in the watch-only wallet.

I have had difficulty in achieving my next two goals with armory which is to get the balance of an address, and to make a transaction for signing by the offline wallet. Unlike generating the next address, both of these require the BDM to be loaded (ie the blockchain to be buffered), which I have been able to achieve. However, implementing the functionality I desire has not yet happened, I have had too many other things which have taken priority, and I have not looked closely enough at the source to realise these goals.

If anyone is able to make this extra functionality work I would love to hear how you did it. I have written a very basic foundation which allows interface to armory wallets, however it's so basic that I have not yet uploaded it for public consumption. The project deserves a more complete implementation than what I have started. I don't seem to recall anything being in the example file about generating transactions.

Delay in upload is also due to considering whether to implement a satoshi-clone api or to create an api specific to armory. Any suggestions on this are also welcome.
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
August 23, 2012, 12:40:09 PM
Last edit: September 12, 2012, 03:45:14 AM by etotheipi
 #15

I mentioned in my original post about this that this part is not trivial.  It's because selecting coins for a transaction is not a one-shot process -- you have to declare what your desired output amount is and your desired fee, then Armory will create the best transaction it can based on those parameters.  If the transaction has properties that require a bigger fee, then you need to re-select coins with the new fee.  Theoretically, this should be part of a while-loop or something (in case the new selection requires an even-bigger fee), but I designed the SelectCoins algorithm to use a selection that leaves some margin (if possible) so that a higher fee can be included without changing the coin selection.

The functions for reference are:
validateInputsGetTxDP() at qtdialogs.py:4867,
createTxAndBroadcast() at qtdialogs.py:4811
(Maybe)broadcastTransaction() in ArmoryQt.py:1889).  
PySelctCoins() in armoryengine.py:5049.  

There's a lot of code here, because it is also validating the data was entered properly on the Send-Tx dialog.  But what you need is there.  I'll try to summarize it here.

PySelectCoins arguments:
  • unspentTxOutInfo: a list of unspent TxOuts, retrieved via wallet.getTxOutList("Spendable") (qtdialogs.py:4968)
  • targetOutVal: total amount to send to (all) recipients(s), in satoshis  (i.e. 10.0*ONE_BTC for 10BTC)
  • minFee: you can always set this to 0 the first time, and re-call it with higher if the output suggests more
  • numRand (optional): number of randomized SelectCoins solutions to try (more about this in the appendix, below)
  • margin (optional): how much margin you'd like to build into the selection, for the reasons stated above

Actually executing the transaction -- you must get all the spendable coins, make a coin selection, then calculate the change and generate the recipient list (you will need to wallet.getNextUnusedAddress() generate a change address, or you can do something else).  Shuffle the list.   Then create the "PyTxDistProposal" -- this is an intermediate step where the transaction is constructed as a BIP 0010 packet, in case it needs to be signed by an offline computer.  In the case that you have the full wallet -- this TxDP will be signed immediately, then converted to a standard transaction and broadcast.

And finally, here's the code sample you've been waiting for:  Create one transaction from a wallet, to send 10.0 BTC to one recipient

Code:
       # Get unspent TxOutList and select the coins
      totalSend, fee = long(10.0 * ONE_BTC), 0
      spendBal = wallet.getBalance('Spendable')
      utxoList = wallet.getTxOutList('Spendable')
      utxoSelect = PySelectCoins(utxoList, totalSend, fee)

      # Check the minimum fee for the coin selection
      minFeeRec = calcMinSuggestedFees(utxoSelect, totalSend, fee)[1]
      if fee<minFeeRec:
           if totalSend + minFeeRec > spendBal:
               raise NotEnoughCoinsError, "You can't afford the fee!"
           utxoSelect = PySelectCoins(utxoList, totalSend, minFeeRec)
           fee = minFeeRec

      if len(utxoSelect)==0:
         raise CoinSelectError, "Somehow, coin selection failed.  This shouldn't happen"

      # We have a good coinselection, now compute change and create recip list
      totalSelected = sum([u.getValue() for u in utxoSelect])
      totalChange = totalSelected - (totalSend  + fee)

      outputPairs = []
      outputPairs.append( [addr160_recipient, totalSend] )
      if totalChange > 0:
         outputPairs.append( [wallet.getNextUnusedAddress().getAddr160(), totalChange] )

      # Shuffle the list, create the TxDP
      random.shuffle(outputPairs)
      txdp = PyTxDistProposal().createFromTxOutSelection(utxoSelect, outputPairs)
      
      # Sign the TxDP
      wallet.signTxDistProposal(txdp)
      finalTx = txdp.prepareFinalTx()
      wallet.setComment(finalTx.getHash(), "This was the example tx created by the python script")
      finalTx.pprint()
      print "Hexified Tx: ", binary_to_hex(finalTx.serialize())
      
      # Broadcast the transaction
      ...  (will fill this part in later)
      

I suppose I could encapsulate much of this code in a more friendly way.  On the other hand, you can pretty much copy this code and make the appropriate adjustments, and you'll get a signed transaction.  You can modify it for multiple recipients, handling fees differently (but you should trust the calcMinSuggestedFee[1] value), and saving off copies of the transaction before sending.

I will include more about actually broadcasting the transaction, in a bit.  The problem is that you need to start the networking engine in order to actually broadcast it.  On the other hand, you have a PyTx object (finalTx), so you can call binary_to_hex(finalTx.serialize()) to get the raw hex version of it, which you might be able to broadcast in other ways.  (like "sendrawtransaction" RPC call with bitcoind)

I know you'll have questions.  Fire away!



Appendix: The way PySelectCoins works:  a bunch of deterministic coin-selections (solutions to the pseudo-knapsack problem) are generated based on a variety of standard coin pools (the types and sizes of unspent TxOuts for your wallet).  A few random solutions are thrown on top to generate some variety.  Usually, the deterministic algorithms find the best answer, but for strange coin-pool make-ups, some semi-randomized selections will produce decent answers.  In all, we get about a couple dozen possible solutions, and then each one is evaluated for "fitness" ... how big is the fee?  how many addresses are combine?  how obvious is it which output is the recipient and which one is change?  The relative weightings of each of these is defined in armoryengine.py:5000 -- the WEIGHTS variable defines how much relative weighting to use for each factor.  There's six of them, though it really could've been reduced to 3 important ones:  tx-fee/allow-free, input-anonymity, and output-anonymity.  

Code:
WEIGHTS[IDX_ALLOWFREE]  =  100000
WEIGHTS[IDX_NOZEROCONF] = 1000000  
WEIGHTS[IDX_PRIORITY]   =      50
WEIGHTS[IDX_NUMADDR]    =  100000
WEIGHTS[IDX_TXSIZE]     =     100
WEIGHTS[IDX_OUTANONYM]  =      30

You can manually modify these weightings, or find a way to inject a new weights matrix into the PyEvalCoinSelect (I apologize, I didn't make that easy, you might be better to just manually change armoryengine.py, for now).  What do these mean?  The default behavior is to avoid using zero-conf coins at all costs.  Any solution with no zero-conf coins will be scored higher than any solution with zero-conf.  The next two biggest weights are "ALLOWFREE" and "NUMADDR".  This means that equal weighting is given to "please give me a transaction that can be free" and "please use as few addresses as possible on the input side".   "TXSIZE" and "PRIORITY" are next, but probably shouldn't even be there, because they are already tied into the ALLOWFREE and NOZEROCONF scores.  

Finally, OUTANONYM is the last critieria.  Given a set of coin-selects that are basically all the same for the other 5 criteria, it will select the solution that has better output-anonymity.  i.e. -- if the outputs are 10.0 and 1.3482023, it's fairly obvious to someone observing the blockchain which one is the recipient, which is the change-back-to-self (1.3482023).  However, if a solution is found that gives 11.34 and 5.11 -- it's not so obvious anymore, and that solution will get a higher score.  It gets extra credit if it's deceptive:  if the recipient is getting 1.384 and it finds a solution that has a change of 8.1:  then most would observers would make the wrong conclusion.

The point of this description was not only to describe the weightings, but suggest you change them for your own usage.  I had planned to make this customizable in Armory, I just hadn't gotten around to it.  If all you care about is input-anonymity and output-anonymity, I suggest changing NUMADDR and OUTANONYM to high numbers, and change the other 4 to something tiny.  You might pay a fee when you otherwise wouldn't have to, but it will be more anonymous.

DISCLAIMER:  I make no guarantees about transaction anonymity at all.  Use of this feature is completely at your own risk.  This is implemented solely for the purposes of declaring a rough ordering for an under-defined set of solutions.  Setting any particular weight to 1010 and others to 0 does not mean you are guaranteed a "good" solution -- there may only be one solution.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
runeks
Legendary
*
Offline Offline

Activity: 980
Merit: 1008



View Profile WWW
August 25, 2012, 12:54:01 AM
 #16

Another possible web server setup that is possible with Armory (that I've actually thought of setting up myself), is having the web server connect via USB to a small Linux ARM device (just something cheap) that has a USB OTG port (allowing the device to act as a USB client). This USB client device stores the private keys and only responds to request over USB. With this setup, it's possible to simply send transaction requests to the web server that are signed with a certain private key, let the web server forward the request to the device, and then let the device - connected only via USB and not connected to the Internet - verify that request with the corresponding public key, and only return a signed transaction over USB if the signature on the request can be verified with the public key on the device.

This should - if implemented correctly - make the wallet unhackable. The only thing an attacker can do - even if he gains full access to the web server - is send request over the USB connection to the device that hold the keys, but without the proper private key the device will ignore the requests.

This could be useful for use cases such as an exchange needing to transfer coins from its cold storage wallet to it's online wallet. The exchange owner simply has to make sure that the private key used to sign requests is inaccessible to attackers.
Another use case could be a company exchanging fiat currency for bitcoins. Here the bank is in possession of the private key. It promises to sign all incoming wire transfers with this private key. If the wire transfer - in a comments-field of some sort - includes a receiving address, it's possible to let someone wire a certain amount of fiat currency to the business owners account, and let the business send out coins to that person - completely securely - given that the customer includes the send-to address in the wire transfer. If the bank can keep the private key secure, this - again - prevents an attacker from being able to do anything were he to gain full control of the web server.
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
August 25, 2012, 03:16:19 AM
 #17

Another possible web server setup that is possible with Armory (that I've actually thought of setting up myself), is having the web server connect via USB to a small Linux ARM device (just something cheap) that has a USB OTG port (allowing the device to act as a USB client). This USB client device stores the private keys and only responds to request over USB. With this setup, it's possible to simply send transaction requests to the web server that are signed with a certain private key, let the web server forward the request to the device, and then let the device - connected only via USB and not connected to the Internet - verify that request with the corresponding public key, and only return a signed transaction over USB if the signature on the request can be verified with the public key on the device.

This should - if implemented correctly - make the wallet unhackable. The only thing an attacker can do - even if he gains full access to the web server - is send request over the USB connection to the device that hold the keys, but without the proper private key the device will ignore the requests.

This could be useful for use cases such as an exchange needing to transfer coins from its cold storage wallet to it's online wallet. The exchange owner simply has to make sure that the private key used to sign requests is inaccessible to attackers.
Another use case could be a company exchanging fiat currency for bitcoins. Here the bank is in possession of the private key. It promises to sign all incoming wire transfers with this private key. If the wire transfer - in a comments-field of some sort - includes a receiving address, it's possible to let someone wire a certain amount of fiat currency to the business owners account, and let the business send out coins to that person - completely securely - given that the customer includes the send-to address in the wire transfer. If the bank can keep the private key secure, this - again - prevents an attacker from being able to do anything were he to gain full control of the web server.

This seems like a good idea to post on the Improving Offline Wallets Thread.

I like the hardware idea, except I don't understand where the private key & signature comes from?  It's not stored on the online system, is it? 

Why do you need any signing?  Why not just use that device to facilitate passing ASCII-only between the two systems?  Any non-ASCII buffers are rejected, and the offline computer will do nothing except collect the data and wait for the user to act on the received data.  Really, it would just behave like a serial port.

Damnit!  I wish I could make the serial port idea work...

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
runeks
Legendary
*
Offline Offline

Activity: 980
Merit: 1008



View Profile WWW
August 25, 2012, 04:03:55 AM
Last edit: August 25, 2012, 04:44:28 AM by runeks
 #18

Another possible web server setup that is possible with Armory (that I've actually thought of setting up myself), is having the web server connect via USB to a small Linux ARM device (just something cheap) that has a USB OTG port (allowing the device to act as a USB client). This USB client device stores the private keys and only responds to request over USB. With this setup, it's possible to simply send transaction requests to the web server that are signed with a certain private key, let the web server forward the request to the device, and then let the device - connected only via USB and not connected to the Internet - verify that request with the corresponding public key, and only return a signed transaction over USB if the signature on the request can be verified with the public key on the device.

This should - if implemented correctly - make the wallet unhackable. The only thing an attacker can do - even if he gains full access to the web server - is send request over the USB connection to the device that hold the keys, but without the proper private key the device will ignore the requests.

This could be useful for use cases such as an exchange needing to transfer coins from its cold storage wallet to it's online wallet. The exchange owner simply has to make sure that the private key used to sign requests is inaccessible to attackers.
Another use case could be a company exchanging fiat currency for bitcoins. Here the bank is in possession of the private key. It promises to sign all incoming wire transfers with this private key. If the wire transfer - in a comments-field of some sort - includes a receiving address, it's possible to let someone wire a certain amount of fiat currency to the business owners account, and let the business send out coins to that person - completely securely - given that the customer includes the send-to address in the wire transfer. If the bank can keep the private key secure, this - again - prevents an attacker from being able to do anything were he to gain full control of the web server.

I like the hardware idea, except I don't understand where the private key & signature comes from?  It's not stored on the online system, is it?
The private key (let's call it the master private key) would basically belong to the person who needs this setup (web server plus key-storage box (the latter having the private keys for the wallet (lets call these wallet private keys))) to produce and send out valid transactions, spending coins from the wallet(s) on the key-storing box. The key-storing box would then have the corresponding "master" public key, and it will only sign a transaction sent to it over the USB connection if it is signed using the master private key.

My initial use case was, as mentioned, me selling bitcoins for fiat currency. I have a site that informs the user to send an amount of fiat currency to my bank account, and include the address - to which the bitcoins are to be sent - in the "comments" field, when sending the wire transfer via his or her e-banking site. The bank I would use to provide my account would have the master private key. Whenever a new bank transfer to my account takes place, the bank sends me an email containing information on the transfer - including the destination bitcoin-address that the buyer included in the "comments" field - signed with the master private key. Then it's simply a matter of having the online web server read new incoming mails from the bank, and pass on the content of the mails to the key-storage box. The key-storage box checks that the wire transfer message is signed with the master private key, and if it is, constructs a transaction that sends an amount of bitcoins to the address specified in the wire transfer message. It passes this to the web server which publishes it. All automated, and - as far as I can see - unhackable, provided the software on the key storage box is trusted.
The only thing that struck me while writing this, is that we need some way of telling the storage box the amount to send. This can't really be included in the signed wire transfer message, as the customer shouldn't be in charge of this. It seems this would be an unsafe operation. Ie. a customer could place an order, hack into the web server, and change the amount being sent to his address, since the storage box can't know the bitcoin exchange rate, since it isn't online. I guess we'd need Mt. Gox to sign a piece of data containing the current time and exchange rate, so the box can know how many coins to send.

Quote
Why do you need any signing?  Why not just use that device to facilitate passing ASCII-only between the two systems?  Any non-ASCII buffers are rejected, and the offline computer will do nothing except collect the data and wait for the user to act on the received data.  Really, it would just behave like a serial port.
Are you imagining three devices? I only imagine two: the web server and the device. The device stores the keys, and does the signing - provided it receives a properly signed message that tells it where to send the coins.

Quote
Damnit!  I wish I could make the serial port idea work...
You and me both! I think I'll be looking into getting the Raspberry Pi to act as a USB client. It would then be necessary to use the USB protocol, which is a lot more complicated than just serial, but would not - as far as I know - allow login to the offline machine.

EDIT: I just discovered that the Raspberry Pi Model A (as opposed to Model B that is out now) will support acting as a USB device/client, because it uses a different communication-chip than Model B. Model A doesn't have an Ethernet port, and only has a single USB port, but I figure this might even be an advantage, since after the software has been installed on the device (using, for example, some USB-to-Ethernet type device to update packages), the Ethernet port will have to be removed because the USB connection needs to be used to communicate with the online computer. Perhaps one will be less tempted - and feel safer - if the device doesn't even have a port to plug in an Ethernet cable.
Additionally, it costs only $25. That's really not much for the ultimate in safe Bitcoin wallets. That being said, Model A isn't out now. And USB client mode won't necessarily work out-of-the-box. This is a software issue, as far as I can gather. Not OS level, I think, but firmware level - so it might not be that easy to implement. But still, looks like a device is coming to the market that fulfils the requirements of an offline hardware wallet. Off the top of my head, I can't see what else one would need other than a Raspberry Pi Model A, and perhaps a USB cable. But most people have USB cables lying around. Sure beats that $40 2xUSB-to-serial-plus-null-modem solution.
ErebusBat
Hero Member
*****
Offline Offline

Activity: 560
Merit: 500

I am the one who knocks


View Profile
August 26, 2012, 01:34:31 AM
 #19

I would like to point out that you do not have to use USB. Ethernet would actually work fine for this. Just have to cards in the webserver; one for Internet and the other for the key-box.

Then you can just set static addresses (or even run a dhcp server on the keybof, but that increases your attack surface).

Anyway you achieve the same goal. There is only one connection to the keybox and it is not on the interwebz.

░▒▓█ Coinroll.it - 1% House Edge Dice Game █▓▒░ • Coinroll Thread • *FREE* 100 BTC Raffle

Signup for CEX.io BitFury exchange and get GHS Instantly!  Don't wait for shipping, mine NOW!
runeks
Legendary
*
Offline Offline

Activity: 980
Merit: 1008



View Profile WWW
August 30, 2012, 07:29:45 PM
 #20

That's true. The thing is though, that I imagine IP connectivity opens up a lot more potential attack surfaces than a USB connection. I mean, any program on the key box listening for incoming connections will suddenly be an attack surface. And it's really hard to control which programs do this; a lot will. With the device acting as a USB client this won't be an issue. You can't - not as far as I know at least - "listen" for connections via USB, and so, there will only be one central point - one attack surface - that we need to secure, which is the USB driver and the program that communicates over this connection. If this is secure, the key box should be secure.
Pages: [1] 2 3 4 5 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!