Bitcoin Forum
May 29, 2024, 01:16:58 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: SOLVED - MADE SCRIPT: signing raw transactions from bitcoind with armory  (Read 4375 times)
rbdrbd (OP)
Sr. Member
****
Offline Offline

Activity: 462
Merit: 250



View Profile
February 04, 2014, 08:25:32 AM
Last edit: February 07, 2014, 05:23:24 AM by rbdrbd
 #1

Quick question:

I was wondering if it was possible to sign an unsigned transaction (hex encoded) essentially produced by bitcoind's createrawtransaction (i.e. long hex string starting with 010000000) in offline armory, assuming that my offline armory wallet has the necessary private keys of course.

I tried producing one using and pasting it into armory's Sign and/or Broadcast Transaction text box, but it says "Unrecognized", along with "Transaction data is invalid and cannot be shown". The txn is valid (if signed in bitcoind it will broadcast fine, and it decodes properly with https://blockchain.info/decode-tx). I've noticed the format that armory appears to produce for its raw transactions has header and footer lines, as well as _TXDIST/_TXINPUT, and the hex encoded transaction is wrapped at a certain line width. Do I need to somehow encode the raw bitcoin transaction to match this form for armory to be able to work with it?

Thanks!

EDIT: So I figured it out and wrote a script to do this, which will take a bitcoind hex raw txn and convert to an armory style txn, which should be good for signing and/or broadcasting (if already signed) I would think. I couldn't get the system path hack to fully work... it still complained about not being able to import a module in the jsonrpc folder. That being the case, you could just throw this script in /usr/lib/armory on the online armory box (with a full or watch-only wallet) and run it. Starts armory up, loads the blockchain, and spits out the armory-formatted txn.

Any suggestions to the script are welcome (and feel free to add this to the armory extras folder):

Code:
import sys
sys.path.append("/usr/lib/armory")
from armoryengine import *

#See https://bitcoinarmory.com/developers/python-scripting/

if len(sys.argv) != 3:
    print "Please pass wallet file, followed by hex encoded unsigned raw txn"
    sys.exit(2)

walletPath = sys.argv[1]
hexRawTxn = sys.argv[2]

wlt = PyBtcWallet().readWalletFile(walletPath)
# Register wallet and start blockchain scan
TheBDM.registerWallet(wlt)
TheBDM.setBlocking(True)
TheBDM.setOnlineMode(True)  # will take 20 min for rescan
# Need "syncWithBlockchain" every time TheBDM is updated
wlt.syncWithBlockchain()

#Translate raw txn
pytx = PyTx()
print("Encoding raw txn: %s" % hexRawTxn)
binTxn = hex_to_binary(hexRawTxn)
pytx.unserialize(binTxn)
tx = PyTxDistProposal(pytx)
print("\n\nOutput is:\n%s" % tx.serializeAscii())

TheBDM.execCleanShutdown()

fthoughts
Newbie
*
Offline Offline

Activity: 21
Merit: 0


View Profile
April 25, 2014, 03:33:42 AM
 #2

this just saved me so hard.

thanks bootloads rbdrbd, keep doin yer thing,

great dev work
gokudev
Newbie
*
Offline Offline

Activity: 30
Merit: 0


View Profile
May 22, 2014, 08:22:05 PM
 #3

wondering why you need to go in online mode and sync with blockchain?
fthoughts
Newbie
*
Offline Offline

Activity: 21
Merit: 0


View Profile
May 22, 2014, 09:06:58 PM
 #4

cause you have to let armory sort out whether the input you selected is a valid (unspent) one or not
gokudev
Newbie
*
Offline Offline

Activity: 30
Merit: 0


View Profile
May 22, 2014, 09:18:47 PM
 #5

And this can't be done with when creating raw unsigned transaction with the online bitcoin daemon ?
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
May 24, 2014, 05:05:45 PM
 #6

You don't have to go online to sign transactions created by Armory, because Armory bundles in extra information needed for the offline computer to be able to securely review and sign the transaction.  Just a raw unsigned transaction is not enough, and Bitcoin Core doesn't have any way to provide that information (without modifying it).  You should create the unsigned transaction (TxDP) with Armory online, and then it can be signed with Armory without going online. 

That's the entire purpose of this thread:  https://bitcointalk.org/index.php?topic=181734.0.  Requesting to piggyback on the next hard fork to make it possible to supply all the secure threshold of data without having to bundle potentially MB of supporting transactions.  However, even if that was fixed, you still need the previous TxOut scripts in order to execute signing, which have to be retrieved from the blockchain -- again something else that Armory's TxDP structure bundles with it, but a raw unsigned tx from Bitcoin Core does not have.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
fthoughts
Newbie
*
Offline Offline

Activity: 21
Merit: 0


View Profile
May 24, 2014, 06:41:41 PM
 #7

"However, even if that was fixed, you still need the previous TxOut scripts in order to execute signing, which have to be retrieved from the blockchain -- again something else that Armory's TxDP structure bundles with it, but a raw unsigned tx from Bitcoin Core does not have."

I think  you could make an additional query to the blockchain from bitcoin-core to find this info, AFAIK that is what we are doing now, but the answer to the above question is basically Armory has metadata that has to be fetched while online first, if going from bitcoin-core tx -> Armory tx

etothepi, could you let us know what exactly that info is?
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
May 24, 2014, 10:19:23 PM
Last edit: May 25, 2014, 12:10:14 AM by etotheipi
 #8

"However, even if that was fixed, you still need the previous TxOut scripts in order to execute signing, which have to be retrieved from the blockchain -- again something else that Armory's TxDP structure bundles with it, but a raw unsigned tx from Bitcoin Core does not have."

I think  you could make an additional query to the blockchain from bitcoin-core to find this info, AFAIK that is what we are doing now, but the answer to the above question is basically Armory has metadata that has to be fetched while online first, if going from bitcoin-core tx -> Armory tx

etothepi, could you let us know what exactly that info is?

https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki

With multi-sig coming, that whole message format changed, and BIP 10 is obsolete.  But at least right now, everything you need to know is there.  It's "draft" because I was the only one doing offline tx and no one wanted to contribute to making a  standard out of it.

The new version for multi-sig also natively handles offline tx (it's just a 1-of-1 "multi"-sig).  I'll document it once design flux settles down.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
PhantomPhreak
Sr. Member
****
Offline Offline

Activity: 476
Merit: 300

Counterparty Chief Scientist and Co-Founder


View Profile
July 09, 2014, 01:59:15 AM
 #9

"However, even if that was fixed, you still need the previous TxOut scripts in order to execute signing, which have to be retrieved from the blockchain -- again something else that Armory's TxDP structure bundles with it, but a raw unsigned tx from Bitcoin Core does not have."

I think  you could make an additional query to the blockchain from bitcoin-core to find this info, AFAIK that is what we are doing now, but the answer to the above question is basically Armory has metadata that has to be fetched while online first, if going from bitcoin-core tx -> Armory tx

etothepi, could you let us know what exactly that info is?

https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki

With multi-sig coming, that whole message format changed, and BIP 10 is obsolete.  But at least right now, everything you need to know is there.  It's "draft" because I was the only one doing offline tx and no one wanted to contribute to making a  standard out of it.

The new version for multi-sig also natively handles offline tx (it's just a 1-of-1 "multi"-sig).  I'll document it once design flux settles down.

Hi. I don't mean to necro-bump! I'm trying to implement bip-0010 with no signing and no multi-sig. Here's what I have now (Python 3):

Code:
txdp = []                                                           
dpid = base58_encode(hashlib.sha256(unsigned_tx).digest())[:8]     
txdp.append(('-----BEGIN-TRANSACTION-' + dpid + '-----').ljust(80,'-'))
                                                                               
magic_bytes = binascii.hexlify(config.MAGIC_BYTES).decode('utf-8') 
varIntTxSize = binascii.hexlify(len(unsigned_tx).to_bytes(2, byteorder='big')).decode('utf-8')
txdp.append('_TXDIST_{}_{}_{}'.format(magic_bytes, dpid, varIntTxSize))
tx_list = unsigned_tx_hex                                           
for coin in inputs:                                                 
        tx_list += get_raw_transaction(coin['txid'], json=False)       
for byte in range(0,len(tx_list),80):                               
        txdp.append(tx_list[byte:byte+80] )                             
                                                                               
for index, coin in enumerate(inputs):                               
        index_fill = str(index).zfill(2)                               
        value = '{0:.8f}'.format(coin['amount'])                       
        txdp.append('_TXINPUT_{}_{}'.format(index_fill, value))         
                                                                               
txdp.append(('-----END-TRANSACTION-' + dpid + '-----').ljust(80,'-'))
output = '\n'.join(txdp)                                   

And here is some example output:
Code:
-----BEGIN-TRANSACTION-ENeepxUG-------------------------------------------------
_TXDIST_f9beb4d9_ENeepxUG_0164
0100000003f4c768004d9cbf49b408d6f25397a2a9cab849e5765acf50bea755c2ea434f8c000000
001976a914f8195d523aa0c10d9d20eca785041815257f3ec888acffffffffef3c6c5629ece542e9
1b1bd1e77608227c1609e55e1e644ed5767cc1118ed0aa000000001976a914f8195d523aa0c10d9d
20eca785041815257f3ec888acffffffff59f6458fbdb0b891318d04cab86972df09672a30cb14d8
a5869223e82fd50823000000001976a914f8195d523aa0c10d9d20eca785041815257f3ec888acff
ffffff03771e0000000000001976a914f8195d523aa0c10d9d20eca785041815257f3ec888ac771e
000000000000475121035bceeb417f25beaa28d133ee7b28faa1e4f5c2f76b8daf12c3fab1826171
8790211c434e545250525459000000000000000000000001000000001dcd65000000000052ae451b
0000000000001976a914f8195d523aa0c10d9d20eca785041815257f3ec888ac0000000001000000
0178671be6c7687711cb89f86855b4dda19b29c37e3400a93edabfb1340a06d231010000006a4730
44022014356ce70459773236bc5005128856342a8d8c74fc3c2d48ac7b1ca80ec259f402207492c2
e0816ce7bc15c61f649edfac504382ece9aace3f0a96664fae9bb443010121035bceeb417f25beaa
28d133ee7b28faa1e4f5c2f76b8daf12c3fab18261718790ffffffff036c2a0000000000001976a9
14f8195d523aa0c10d9d20eca785041815257f3ec888ac6c2a000000000000475121035bceeb417f
25beaa28d133ee7b28faa1e4f5c2f76b8daf12c3fab18261718790211c434e545250525459000000
000000000000062fac00000000000186a00000000052aebd92bf01000000001976a914f8195d523a
a0c10d9d20eca785041815257f3ec888ac00000000010000000176ebe92afc372a7ce8f6d5ca18c6
fc801ebd1671f5894b2513453e183de191df020000006b483045022100e876eebed951dd4c7f6076
bf3df9a91c276e1ad6538e65ea0c2a4e44d849799d02203b50c8ce3fbf440b94fb6cb028f7be49c7
5595111787e7fdaf841118eccf21520121035bceeb417f25beaa28d133ee7b28faa1e4f5c2f76b8d
af12c3fab18261718790ffffffff036c2a0000000000001976a914f8195d523aa0c10d9d20eca785
041815257f3ec888ac6c2a000000000000475121035bceeb417f25beaa28d133ee7b28faa1e4f5c2
f76b8daf12c3fab18261718790211c434e545250525459000000000000000000000001000000001d
cd65000000000052aec1168900000000001976a914f8195d523aa0c10d9d20eca785041815257f3e
c888ac0000000001000000014dc91b94b0812f68c4df320ace1b061310aa30b8e26a161318bf6686
22b24c5e040000006a47304402207fc35dfea6232a859f80009d285e14b60df33f1f585bb4c2775a
8961752a7d3d022025cbe954f9edea01bca313a08a73948a4b86f587d9612ad177dc88730032738d
0121035bceeb417f25beaa28d133ee7b28faa1e4f5c2f76b8daf12c3fab18261718790ffffffff03
6b2a0000000000001976a914f8195d523aa0c10d9d20eca785041815257f3ec888ac6b2a00000000
0000475121035bceeb417f25beaa28d133ee7b28faa1e4f5c2f76b8daf12c3fab18261718790211c
434e545250525459000000000000000000000001000000001dcd65000000000052aec8dc85000000
00001976a914f8195d523aa0c10d9d20eca785041815257f3ec888ac00000000
_TXINPUT_00_0.00010860
_TXINPUT_01_0.00010860
_TXINPUT_02_0.00010859
-----END-TRANSACTION-ENeepxUG---------------------------------------------------


When I paste this into Armory, running offline with --debug, I don't get any error messages except the following:

Code:
X Error: BadMatch (invalid parameter attributes) 8
  Major opcode: 42 (X_SetInputFocus)
  Resource id:  0x2600043

The signing window reads only "Unrecognized!". Thoughts on what I'm doing wrong? (It must be something obvious.) Any help would be greatly appreciated!
fthoughts
Newbie
*
Offline Offline

Activity: 21
Merit: 0


View Profile
July 09, 2014, 09:17:56 PM
 #10

Hm, I'm curious, how or where are you getting the data to reconstruct the Armory tx? If you haven't looked at their code, I think this approach is perilous, since Armory's custom format will surely throw your code in disarray.

For example, I'm not sure if the bytes from utxo's are appended like this:

Code:
for coin in inputs:                                                 
        tx_list += get_raw_transaction(coin['txid'], json=False)       
for byte in range(0,len(tx_list),80):                               
        txdp.append(tx_list[byte:byte+80] )                         

PhantomPhreak
Sr. Member
****
Offline Offline

Activity: 476
Merit: 300

Counterparty Chief Scientist and Co-Founder


View Profile
July 10, 2014, 04:11:35 AM
 #11

Hm, I'm curious, how or where are you getting the data to reconstruct the Armory tx? If you haven't looked at their code, I think this approach is perilous, since Armory's custom format will surely throw your code in disarray.

For example, I'm not sure if the bytes from utxo's are appended like this:

Code:
for coin in inputs:                                                 
        tx_list += get_raw_transaction(coin['txid'], json=False)       
for byte in range(0,len(tx_list),80):                               
        txdp.append(tx_list[byte:byte+80] )                         



I'm looking at https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki, and, yes, at the code itself. These lines, in particular, are what I'm working off of: https://github.com/etotheipi/BitcoinArmory/blob/master/armoryengine/Transaction.py.
rbdrbd (OP)
Sr. Member
****
Offline Offline

Activity: 462
Merit: 250



View Profile
July 11, 2014, 02:39:02 PM
 #12

Hm, I'm curious, how or where are you getting the data to reconstruct the Armory tx? If you haven't looked at their code, I think this approach is perilous, since Armory's custom format will surely throw your code in disarray.

For example, I'm not sure if the bytes from utxo's are appended like this:

Code:
for coin in inputs:                                                 
        tx_list += get_raw_transaction(coin['txid'], json=False)       
for byte in range(0,len(tx_list),80):                               
        txdp.append(tx_list[byte:byte+80] )                         



I'm looking at https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki, and, yes, at the code itself. These lines, in particular, are what I'm working off of: https://github.com/etotheipi/BitcoinArmory/blob/master/armoryengine/Transaction.py.

I'm actually xnova (another one of the Counterparty devs). Any assistance here would be very much welcome. Getting this working would allow us to add Armory offline signing support to counterpartyd and Counterwallet, which would be great for our users and probably add dozens of  more new armory users, at the very least.
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
July 11, 2014, 03:01:19 PM
 #13

Hm, I'm curious, how or where are you getting the data to reconstruct the Armory tx? If you haven't looked at their code, I think this approach is perilous, since Armory's custom format will surely throw your code in disarray.

For example, I'm not sure if the bytes from utxo's are appended like this:

Code:
for coin in inputs:                                                 
        tx_list += get_raw_transaction(coin['txid'], json=False)       
for byte in range(0,len(tx_list),80):                               
        txdp.append(tx_list[byte:byte+80] )                         



I'm looking at https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki, and, yes, at the code itself. These lines, in particular, are what I'm working off of: https://github.com/etotheipi/BitcoinArmory/blob/master/armoryengine/Transaction.py.

I'm actually xnova (another one of the Counterparty devs). Any assistance here would be very much welcome. Getting this working would allow us to add Armory offline signing support to counterpartyd and Counterwallet, which would be great for our users and probably add dozens of  more new armory users, at the very least.


Guys, I apologize for not being responsive here.  I'm up to my eyeballs in a variety of other things.   After 0.92, I will have some time to discuss this further, though I would then encourage you to switch to the new format we are using which handles offline transactions and multi-sig alike (you don't have to use the multi-sig stuff, it's simply a lot cleaner and more flexible).

Until then, everything that you need should be in serializeAscii() method:  https://github.com/etotheipi/BitcoinArmory/blob/master/armoryengine/Transaction.py#L1246

From a glance, I'm not sure that you should be using .decode('utf-8') when doing the serializations.  This is binary data, not unicode.  Construct all pieces as raw binary and convert to hex.  There shouldn't be any utf-8 calls in it.

In the next release, we switch to UnsignedTransaction, which handles more general transaction types and securely signing with offline-multi-sig devices.  But it will become the new standard for Armory offline transactions and multi-sig.   And serialization is simpler:  create one giant binary string and convert it all to base64. 


Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
PhantomPhreak
Sr. Member
****
Offline Offline

Activity: 476
Merit: 300

Counterparty Chief Scientist and Co-Founder


View Profile
July 11, 2014, 03:23:57 PM
 #14

Hm, I'm curious, how or where are you getting the data to reconstruct the Armory tx? If you haven't looked at their code, I think this approach is perilous, since Armory's custom format will surely throw your code in disarray.

For example, I'm not sure if the bytes from utxo's are appended like this:

Code:
for coin in inputs:                                                 
        tx_list += get_raw_transaction(coin['txid'], json=False)       
for byte in range(0,len(tx_list),80):                               
        txdp.append(tx_list[byte:byte+80] )                         



I'm looking at https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki, and, yes, at the code itself. These lines, in particular, are what I'm working off of: https://github.com/etotheipi/BitcoinArmory/blob/master/armoryengine/Transaction.py.

I'm actually xnova (another one of the Counterparty devs). Any assistance here would be very much welcome. Getting this working would allow us to add Armory offline signing support to counterpartyd and Counterwallet, which would be great for our users and probably add dozens of  more new armory users, at the very least.

From a glance, I'm not sure that you should be using .decode('utf-8') when doing the serializations.  This is binary data, not unicode.  Construct all pieces as raw binary and convert to hex.  There shouldn't be any utf-8 calls in it.

binascii.hexlify() returns ASCII-encoded hex only, at least in Python 3:

Code:
Python 3.4.1 (default, May 19 2014, 17:23:49) 
[GCC 4.9.0 20140507 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import binascii
>>> foo = binascii.hexlify(b'\x00')
>>> foo
b'00'
>>>
>>> foo.decode('utf-8')
'00'
>>>
PhantomPhreak
Sr. Member
****
Offline Offline

Activity: 476
Merit: 300

Counterparty Chief Scientist and Co-Founder


View Profile
July 11, 2014, 03:28:21 PM
 #15

Hm, I'm curious, how or where are you getting the data to reconstruct the Armory tx? If you haven't looked at their code, I think this approach is perilous, since Armory's custom format will surely throw your code in disarray.

For example, I'm not sure if the bytes from utxo's are appended like this:

Code:
for coin in inputs:                                                 
        tx_list += get_raw_transaction(coin['txid'], json=False)       
for byte in range(0,len(tx_list),80):                               
        txdp.append(tx_list[byte:byte+80] )                         



I'm looking at https://github.com/bitcoin/bips/blob/master/bip-0010.mediawiki, and, yes, at the code itself. These lines, in particular, are what I'm working off of: https://github.com/etotheipi/BitcoinArmory/blob/master/armoryengine/Transaction.py.

I'm actually xnova (another one of the Counterparty devs). Any assistance here would be very much welcome. Getting this working would allow us to add Armory offline signing support to counterpartyd and Counterwallet, which would be great for our users and probably add dozens of  more new armory users, at the very least.


Guys, I apologize for not being responsive here.  I'm up to my eyeballs in a variety of other things.   After 0.92, I will have some time to discuss this further, though I would then encourage you to switch to the new format we are using which handles offline transactions and multi-sig alike (you don't have to use the multi-sig stuff, it's simply a lot cleaner and more flexible).

[snip]

In the next release, we switch to UnsignedTransaction, which handles more general transaction types and securely signing with offline-multi-sig devices.  But it will become the new standard for Armory offline transactions and multi-sig.   And serialization is simpler:  create one giant binary string and convert it all to base64. 

Is the new format stable now? For how long will the current format be supported?
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
July 11, 2014, 03:34:08 PM
 #16

Is the new format stable now? For how long will the current format be supported?

Yes, we've done 5-way simulfunding and 7-of-7 multi-sig spending with this format and we plan to move forward with our release in its current form.  We expect that in the future, a variety of multi-sig services will standardize these formats to allow interoperability, but many such services are scrambling to get anything functional and standardizing would inhibit the design process.  So far, we see no reason that we will have to change the format at all, until that standardization occurs -- in which case everyone will be changing.  (much like how Armory and Electrum had their own deterministic wallet algorithms, then BIP32 was standardized and now everyone is moving torwards that).

From a glance, I'm not sure that you should be using .decode('utf-8') when doing the serializations.  This is binary data, not unicode.  Construct all pieces as raw binary and convert to hex.  There shouldn't be any utf-8 calls in it.

binascii.hexlify() returns ASCII-encoded hex only, at least in Python 3:

Code:
Python 3.4.1 (default, May 19 2014, 17:23:49) 
[GCC 4.9.0 20140507 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import binascii
>>> foo = binascii.hexlify(b'\x00')
>>> foo
b'00'
>>>
>>> foo.decode('utf-8')
'00'
>>>

Why do you need the last line?  Wouldn't you end up with some of the binary chars being converted to ASCII/unicode characters?   My apologies for being naive about python3 ... I'm not actually that familiar with it, beyond the fact that if I were to rewrite Armory from scratch, I'd use python3 from the start. 

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
PhantomPhreak
Sr. Member
****
Offline Offline

Activity: 476
Merit: 300

Counterparty Chief Scientist and Co-Founder


View Profile
July 11, 2014, 04:09:12 PM
Last edit: July 11, 2014, 04:22:44 PM by PhantomPhreak
 #17

binascii.hexlify() returns ASCII-encoded hex only, at least in Python 3:

Code:
Python 3.4.1 (default, May 19 2014, 17:23:49) 
[GCC 4.9.0 20140507 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import binascii
>>> foo = binascii.hexlify(b'\x00')
>>> foo
b'00'
>>>
>>> foo.decode('utf-8')
'00'
>>>

Why do you need the last line?  Wouldn't you end up with some of the binary chars being converted to ASCII/unicode characters?   My apologies for being naive about python3 ... I'm not actually that familiar with it, beyond the fact that if I were to rewrite Armory from scratch, I'd use python3 from the start.  

Going back and forth between Python 2 and Python 3 is a mindfuck. Wink

binascii.hexlify() returns ASCII-encoded binary of the hexadecimal string (I don't know why). When you try to print that directly, Python prints the binary as binary (EDIT: implicitly converting to ASCII literal characters, wherever it can, just for display).

Applying this to the code I wrote...

With:
Code:
_TXDIST_f9beb4d9_ENeepxUG_0164

Without:
Code:
_TXDIST_b'f9beb4d9'_ENeepxUG_b'0164'
xnova
Sr. Member
****
Offline Offline

Activity: 390
Merit: 254

Counterparty Developer


View Profile
July 12, 2014, 10:14:36 PM
 #18

Alan,

Will the new version of armory still support the BIP10 format for signing transactions offline, or is that going to be totally removed?

Also, with this new format, why use binary? Why not have a JSON-formatted object of some kind thats produced (even if it's then encoded to base64 as a final step). A format like that could be much more easily read and verified via a standard JSON schema (http://json-schema.org/), instead of custom byte-packing code. That will make it especially easier for external tool creators/integrators like us to support this new format, especially, and make it easier for developers in general to debug issues surrounding it.

I think another one of the issues we see with this new format is that adding any support for it (at least at this time) involves delving into the guts of numerous armory objects which have their own esoteric methods and functioning. It just feels that could be changing a lot between now and the beta release, given how tightly coupled to the implementation it appears to be, at first glance. (If that's not the case at all and the format is pretty well stabilized, that would be great though.)

Visit the official Counterparty forums: http://counterpartytalk.org
etotheipi
Legendary
*
Offline Offline

Activity: 1428
Merit: 1093


Core Armory Developer


View Profile WWW
July 12, 2014, 10:46:46 PM
 #19

Alan,

Will the new version of armory still support the BIP10 format for signing transactions offline, or is that going to be totally removed?

Also, with this new format, why use binary? Why not have a JSON-formatted object of some kind thats produced (even if it's then encoded to base64 as a final step). A format like that could be much more easily read and verified via a standard JSON schema (http://json-schema.org/), instead of custom byte-packing code. That will make it especially easier for external tool creators/integrators like us to support this new format, especially, and make it easier for developers in general to debug issues surrounding it.

I think another one of the issues we see with this new format is that adding any support for it (at least at this time) involves delving into the guts of numerous armory objects which have their own esoteric methods and functioning. It just feels that could be changing a lot between now and the beta release, given how tightly coupled to the implementation it appears to be, at first glance. (If that's not the case at all and the format is pretty well stabilized, that would be great though.)

Unfortunately, we removed BIP 10 because it was not used by anyone else, and the new format completely encapsulated everything we needed for both regular offline and multisig.  Hence, if you upgrade your online computer to 0.92+ you'll have to do the same for the offline computer.  It's the first time in 2-3 years that we've broken compatibility with older offline systems, but we felt we didn't have a choice.  Everything is far simpler for us this way.

As for the binary formats:  most of what's moving between systems is binary data.  And all systems have the standard set of get/put VAR_INT, VAR_STR, etc, the same as serializing and unserializing other Bitcoin objects.   We had pondered protobuf, but we ran into some issues getting it working cross-platform.  Maybe in the future we will.

As I mentioned above: we've done 5-way simulfunding and 7-of-7 lockbox spends with the new format, and have no issues.  Our barrier to release is stabilizin the GUI -- we see no reason we will have to further modify those formats.  Also, a lot of things in that new format are forward-thinking, like space for payment protocol or other web-of-trust mechanisms (authMethod & authData), and wallet-defined strings to help lite-wallets find their own addresses (wltLocators).   Those can be left blank right now (as Armory does).  The rest of it is just raw scripts, raw public keys, raw transactions, some metadata & labels, etc.  The only thing that might be unusual is the way we compute lockbox IDs, but it's not complex.

I do admit it may have to be updated in the future -- usually something this complex you can't get everything right the first time.  But at least for now, this works for 100% of Armory's simulfunding and lockbox activities, securely works with offlien devices, and even has a bit of foresight built-in to handle lite wallets and authentication of addresses.  I expect this will be sufficient for a very long time, maybe even until the community determines how they want to standardize this, and then we'll upgrade (as well everyone else)

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!