Armory - Discussion Thread

[mav]

I would have thought there's away to convert those floats to the correct decimal values, although maybe that was just a client... I'll look into it tonight cause those floats are horrid! Also I'll look at bitcoind listtransactions output if I get a chance.

Sounds like it's going well so far for the most part.

[etotheipi]

I would have thought there's away to convert those floats to the correct decimal values, although maybe that was just a client... I'll look into it tonight cause those floats are horrid! Also I'll look at bitcoind listtransactions output if I get a chance.

Sounds like it's going well so far for the most part.

The problem is that those values should be transferred as either long-integer (satoshis) or string... in which case the value can be coded exactly.  But someone decided that they'll have bitcoind should transfer them as non-exact floating point (doubles), and 0.0005000000001 for the fee is the closest float that can be encoded.  I'm fairly confident it's really just a display issue, not an armoryd.py issue.

Luckily, round()'ing will always work because doubles have enough bits to represent 8 places without rounding errors.  But I wouldn't mind if we can find a way to have the json library print out the rounded version.  I noticed in the few lines of code I stole from jgarzik (and now part of armoryd.py), he defined a "UniversalEncoder" and was able to pass it to the json.dumps() method to explain how to handle decimal.Decimal objects.  Though I mostly removed that in favor of following  Proper money handling in JSON-RPC, even though it should all give the same answers (it's less characters and more readable).  Either way, maybe there's a hint there about how to tell UniversalEncoder to print "float" values.

Anyone think it's a bad/good idea to do what I recommended:  "python armoryd.py setamountencoding COIN" or "python armoryd.py setamountencoding STRING" to set the daemon to transfer long-int-satoshis or exact strings, respectively?  That way default behavior matches bitcoind that most are expecting, but they have the option to change it with one line at the top of their script.

-----

By the way (mostly for mav), I added the capability for armoryd.py to detect if there's already an instance of armoryd.py running, and if there is, it will connect to it, pass the command line arguments to it, print the reply, then disconnect.  I believe that's exactly what bitcoind does.

Also for anyone who wants to use this, you need to have a "/home/username/.armory/armoryd.conf" file or "C:\Users\username\AppData\Roaming\Armory\armoryd.conf" in order to use it.  The file contains a single line:  "username:password".  Make sure to "chmod 600 armoryd.conf"

[mav]

Have a look at this gist for my idea about using Decimal values (or converting ints to decimals first) and then dumping in the json so it doesn't have floaty rounding display happening

https://gist.github.com/4536693

Will get back in a few hours with the other stuff I'm looking into.

edit: I am inclined to agree with you on the using integers and correct money handling rather than shitty floats or even decimals. At least in this case, when the value is received and loaded into a Decimal, it will be displayed and loaded completely accurately and not rely on the client correctly implementing rounding, which we cannot assume. As I have previously stated, I think following bitcoind api where possible is best since it makes for the least dicking around when migrating to use armory from bitcoind.

edit2: The option to set integer / decimal / string encoding is a great idea. Don't forget to allow users to set it back to the default for decimal if they desire.

edit3: I had a crack at getting the listtransactions data for the various scenarios you describe (and thus the 'way it works'), however I couldn't get any results. Frustratingly, I got raw transactions to sign (which is the second-last step of many) and then the last step is to broadcast the transaction, but my transactions were always getting rejected. I've got my progress thus far well documented and is definitely repeatable so I might try to work it out later in the week. It's worth asking on the development subform for clarification on the way the listtransactions call works, they will hopefully be able to answer it. Anyway I was creating raw transactions to match your scenarios based off this guide and testnet-in-a-box - https://people.xiph.org/~greg/signdemo.txt

[etotheipi]

edit3: I had a crack at getting the listtransactions data for the various scenarios you describe (and thus the 'way it works'), however I couldn't get any results. Frustratingly, I got raw transactions to sign (which is the second-last step of many) and then the last step is to broadcast the transaction, but my transactions were always getting rejected. I've got my progress thus far well documented and is definitely repeatable so I might try to work it out later in the week. It's worth asking on the development subform for clarification on the way the listtransactions call works, they will hopefully be able to answer it. Anyway I was creating raw transactions to match your scenarios based off this guide and testnet-in-a-box - https://people.xiph.org/~greg/signdemo.txt

I'm not sure I understand what you tried.  Were you trying to test creating unsigned transactions and then sign and broadcast with Armory?   Here's what I would do to test (and maybe this is what you did):  open ArmoryQt.py, create a new wallet, send it some money (testnet would be fine).   Create watching-only copy and save it in a different directory.  Now run armoryd.py on that wallet, and use the "sendtransaction" or "sendmany" to create the unsigned transaction.  Then you can go back to ArmoryQt and use the "Offline Transactions" button and select "Sign and/or Broadcast".  You can copy it into the window* and it should immediately recognize it, and allow you to examine and sign it.   Since you're online, you can also broadcast it right away, but I'm not sure why I didn't try that.  I got through signing with my testnet wallet while testing, and everything looked perfect, up to that point.

Also, I will integrate the cli_sign_script.py code into armoryd.py so that signing can be enabled, too.  My only question is how to send the passphrase to JSON-RPC... is it sent cleartext over the socket/JSON interface?  I looked at the Bitcoin code, and it appears that's what they do, though I'm not positive.  This seems okay for localhost (though I don't like it), but not for any remote connections.  Or maybe it just shouldn't be allowed on remote connections (actually, for now there are no remote connections, so I guess it doesn't matter).



*What I found is that if you use armoyd to send the command, it spits out the python-formatted string:  you cannot just copy this into ArmoryQt, because it has "\n" raw characters in there, etc.  You can open python and type "pring <copy string here>" and it will print it out with proper formatting.  Then you can copy it into ArmoryQt.

[etotheipi]

Anyone still having persistent crashing problems in Windows?  (or Linux?)

I finally fixed the two readBlkFileUpdate() bugs.  I made an elaborate test by taking slices of blk files, and copying them into a fake blkfile directory between readBlkFileUpdate calls.  In the end it was a tiny fix, but it should've resolved a variety of issues -- I'm hoping also the Windows crashing issue...

I created a testing version with the fix and even took it to the offline system to get signatures on the sha256 hashes.  As usual, if you are testing in Linux, just switch to the "testing" branch.   

Testing version 0.87.1-testing for Windows 64-bit
Testing version 0.87.1-testing for Windows 32- and 64-bit
Testing version 0.87.1-testing SHA256 hashes of installers

I've had Windows crashing before, but it has sometimes taken days between crashes, so I need someone who has more-frequent crashes to help test it.  I'd say there's a 50-50 chance it's fixed... but if not I have another idea...

[mav]

edit3: I had a crack at getting the listtransactions data for the various scenarios you describe...[/url]

I'm not sure I understand what you tried.  Were you trying to test creating unsigned transactions and then sign and broadcast with Armory?...

I tried getting some info about bitcoind listtransactions behaviour so you had something to go off when emulating it. It was nothing to do with armory. Actually I haven't even got the latest version of armory to look at yet, been crackers busy lately, but will definitely try to do so on the weekend.

Offline transactions with bitcoind are actually quite interesting. I still prefer armory because of the deterministic and watch-only wallet features, that's really a remarkable feature to have.

[wachtwoord]

@wachtwoord
The alternative you describe is really not much better than just maintaining an encrypted wallet on your hot computer.  What it prevents is someone who has gotten remote access to your machine and manually digging around your filesystem looking for wallet files.  That's not to say that such things don't happen, but I believe the real threats are viruses that get on your computer and siphons off data and send it back to "home base" when it finds it (it will farm the data and send it back the next time it's online).  In this case it probably doesn't even have to send any data back... it just reads your wallet when decrypted and creates a transaction sending all BTC to its owner the next time it's online.

Any computer with access to the internet is vulnerable to these viruses.  You can get them from various exploits usually relating to your browser, or opening PDF/.xls documents with embedded exploits, etc.  However, in order to compromise a truly-offline computer, there's a couple orders-of-magnitude more work for the attacker to do.  They must (1) find a way to hide data on your USB key that (2) exploits an auto-run vulnerability when plugged into the offline computer, and then (3) be able to automatically find the data and copy it back, hidden, on the USB key to get it back to the offline computer.  And these exploits are much more complicated when they don't even know what OS the offline system is.  And if I can find another way to transfer data between systems not using USB keys, then this would even an order of magnitude better...

I would look into this idea posted by N.Z..   It's a bit more work but is definitely a very reasonable solution for a single-computer setup (in fact, I've been thinking about bundling the tails packages like N.Z. suggested, to make this easier).  This means that your wallet only ever touches this sandbox that has no access to internet, and is reset to default configuration on every boot.

As for question 3 (paper backups), you just said in question 2 that you believe flash storage is frail.  This is exactly why you make paper backups.  If you print a paper backup and put it in a safe-deposit box or fold it into a book on your bookshelf, it will still be readable 20+ years from now . It doesn't even have to survive "well", as you could pull the data out of a terribly-faded copy with a bit of work.  I mean, you've seen books that are 50+ years old and their pages are still readable.

Now compare that to any kind of digital media.  You can drop a flash drive, or sit on it or bend it, or put it too close to a magnet, or it just decays and it's no longer usable.  What's your confidence level that even a well-treated USB drive sitting in a metal box will still work 5 years from now?  20 years?    Even CDs and DVDs have expected lifetimes of 2-10 years, but that varies widely depending on lots of factors.  With the paper backup, unless it physically burns to ash, you know it will still provide the data you need to recover your wallet in 20 years.... probably much longer than that.

Thanks a lot for your reply. I have attempted to follow N.Z.'s method but have trouble comprehending how to make it work exactly so I created a thread and PM'ed N.Z. https://bitcointalk.org/index.php?topic=137091.0
Hopefully I'll be able to get this working and then I'll be much more secure

About the paper backups: I still think leaving paper backups is a huge security risk (I guess people will start searching for these). If someone finds my paper backup they can take all my coins while this is not the case for my flash drive (wallet files are encrypted!). With respect to the frailty of USB's, I have currently five nackups on different media and plan to backup it more when the media start failing.

[Daily Anarchist]

Fedora 64 bit make command gives me this error. I know this issue was addressed by mac osx builds, but was it ever addressed for linux builds?

$ make
cd cppForSwig; make swig
make[1]: Entering directory `/home/seth/BitcoinArmory/cppForSwig'
g++ -shared -lpthread  UniversalTimer.o BinaryData.o FileDataPtr.o BtcUtils.o BlockObj.o BlockUtils.o EncryptionUtils.o libcryptopp.a "/usr/lib/libpython`python -c 'import sys; print str(sys.version_info[0]) + "." + str(sys.version_info[1])'`.a" CppBlockUtils_wrap.o -o ../_CppBlockUtils.so
g++: error: /usr/lib/libpython2.7.a: No such file or directory
make[1]: *** [swig] Error 1
make[1]: Leaving directory `/home/seth/BitcoinArmory/cppForSwig'
make: *** [all] Error 2

[K1773R]

Fedora 64 bit make command gives me this error. I know this issue was addressed by mac osx builds, but was it ever addressed for linux builds?

$ make
cd cppForSwig; make swig
make[1]: Entering directory `/home/seth/BitcoinArmory/cppForSwig'
g++ -shared -lpthread  UniversalTimer.o BinaryData.o FileDataPtr.o BtcUtils.o BlockObj.o BlockUtils.o EncryptionUtils.o libcryptopp.a "/usr/lib/libpython`python -c 'import sys; print str(sys.version_info[0]) + "." + str(sys.version_info[1])'`.a" CppBlockUtils_wrap.o -o ../_CppBlockUtils.so
g++: error: /usr/lib/libpython2.7.a: No such file or directory
make[1]: *** [swig] Error 1
make[1]: Leaving directory `/home/seth/BitcoinArmory/cppForSwig'
make: *** [all] Error 2

install python?

[etotheipi]

Fedora 64 bit make command gives me this error. I know this issue was addressed by mac osx builds, but was it ever addressed for linux builds?

$ make
cd cppForSwig; make swig
make[1]: Entering directory `/home/seth/BitcoinArmory/cppForSwig'
g++ -shared -lpthread  UniversalTimer.o BinaryData.o FileDataPtr.o BtcUtils.o BlockObj.o BlockUtils.o EncryptionUtils.o libcryptopp.a "/usr/lib/libpython`python -c 'import sys; print str(sys.version_info[0]) + "." + str(sys.version_info[1])'`.a" CppBlockUtils_wrap.o -o ../_CppBlockUtils.so
g++: error: /usr/lib/libpython2.7.a: No such file or directory
make[1]: *** [swig] Error 1
make[1]: Leaving directory `/home/seth/BitcoinArmory/cppForSwig'
make: *** [all] Error 2

install python?

No that's not it.  It's a one-line makefile fix, but I needed someone who had this problem to help me test the fix for it.  I'll commit something to the "armoryd" branch (which is my current development branch), in about 10 minutes.  If that doesn't work, I'll just tell you how to fix it

[etotheipi]

No that's not it.  It's a one-line makefile fix, but I needed someone who had this problem to help me test the fix for it.  I'll commit something to the "armoryd" branch (which is my current development branch), in about 10 minutes.  If that doesn't work, I'll just tell you how to fix it

Gah!  Why's it so damned hard to check whether a file exists from within a makefile?  I can't use my old bash trick because it's a dynamic filename based on DEPSDIR.

The solution is to change "STATICPYTHON" to point to the libpython2.x.so file instead of the libpython2.x.a.   I want something like (Makefile:28) --

Code:

    SWIG_INC     += -I"$(DEPSDIR)/include/python$(PYVER)"
   STATICPYTHON +=   "$(DEPSDIR)/lib/libpython$(PYVER).a"

   ifneq (exists, $(shell [ -d $(STATICPYTHON) ]  && echo exists ))
      STATICPYTHON = "$(DEPSDIR)/lib/libpython$(PYVER).so"
   endif

But it doesn't seem to like the $(VAR) inside the shell...

You know the fix.  If you know a good way to resolve this, then please let me know.

[Daily Anarchist]

I'm confused by what should be added/replaced. This is what my Makefile looks like:

25    SWIG_INC     += -I"/usr/include/python$(PYVER)"
 26    STATICPYTHON +=   "/usr/lib/python$(PYVER)/config/libpython$(PYVER).a"
 27 else
 28    SWIG_INC     += -I"$(DEPSDIR)/include/python$(PYVER)"
 29    STATICPYTHON +=   "$(DEPSDIR)/lib/libpython$(PYVER).a"
 30
 31
 32    # A failed attempt at auto-detecting .a file and using .so if .a DNE
 33    #PYTHONA = "$(DEPSDIR)/lib/libpython$(PYVER).a"
 34    #ifeq ($(wildcard $(PYTHONA)),)
 35       #STATICPYTHON +=   "$(DEPSDIR)/lib/libpython$(PYVER).so"
 36    #else
 37       #STATICPYTHON +=   "$(DEPSDIR)/lib/libpython$(PYVER).a"
 38    #endif
 39 endif
 40

Can you give me a replace code with code for simplicty?

[etotheipi]

I'm confused by what should be added/replaced. This is what my Makefile looks like:

25    SWIG_INC     += -I"/usr/include/python$(PYVER)"
 26    STATICPYTHON +=   "/usr/lib/python$(PYVER)/config/libpython$(PYVER).a"
 27 else
 28    SWIG_INC     += -I"$(DEPSDIR)/include/python$(PYVER)"
 29    STATICPYTHON +=   "$(DEPSDIR)/lib/libpython$(PYVER).a"
 30
 31
 32    # A failed attempt at auto-detecting .a file and using .so if .a DNE
 33    #PYTHONA = "$(DEPSDIR)/lib/libpython$(PYVER).a"
 34    #ifeq ($(wildcard $(PYTHONA)),)
 35       #STATICPYTHON +=   "$(DEPSDIR)/lib/libpython$(PYVER).so"
 36    #else
 37       #STATICPYTHON +=   "$(DEPSDIR)/lib/libpython$(PYVER).a"
 38    #endif
 39 endif
 40

Can you give me a replace code with code for simplicty?

Sorry.  Modify line 29 at the end to .so instead .a

[Daily Anarchist]

No dice.

g++: error: /usr/lib/libpython2.7.so: No such file or directory
make[1]: *** [swig] Error 1
make[1]: Leaving directory `/home/seth/BitcoinArmory/cppForSwig'
make: *** [all] Error 2

[etotheipi]

No dice.

g++: error: /usr/lib/libpython2.7.so: No such file or directory
make[1]: *** [swig] Error 1
make[1]: Leaving directory `/home/seth/BitcoinArmory/cppForSwig'
make: *** [all] Error 2

Oh, have you installed python-dev?

Specfically, make sure you have all the dependencies listed on the compile-from-source page:

http://bitcoinarmory.com/index.php/building-armory-from-source

[Daily Anarchist]

For what it's worth, a couple of months ago somebody recommended removing a large chunk of line 15 and it allowed it to fully compile. But since I'm doing a fresh install I thought it would be a better idea to run it by you first.

[Daily Anarchist]

It's python-devel for us Red Hat folk, but yes it's installed, along with the rest of the dependencies.

$ yum list python-devel
Loaded plugins: langpacks, presto, refresh-packagekit
fedora/18/x86_64/metalink                                |  13 kB     00:00     
updates/18/x86_64/metalink                               | 8.4 kB     00:00     
Installed Packages
python-devel.x86_64                    2.7.3-13.fc18                     @fedora
Available Packages
python-devel.i686

[etotheipi]

For what it's worth, a couple of months ago somebody recommended removing a large chunk of line 15 and it allowed it to fully compile. But since I'm doing a fresh install I thought it would be a better idea to run it by you first.

Line 15 is what makes this makefile work regardless of what python version you have installed.  Unless you have a non-standard python installation.

What shows up when you type "ls -la /usr/lib | grep libpython"?  

You can change line 29 to just point to whatever .a or .so is there.  

[Daily Anarchist]

For what it's worth, a couple of months ago somebody recommended removing a large chunk of line 15 and it allowed it to fully compile. But since I'm doing a fresh install I thought it would be a better idea to run it by you first.

Line 15 is what makes this makefile work regardless of what python version you have installed.  Unless you have a non-standard python installation.

What shows up when you type "ls -la /usr/lib | grep libpython"?  

You can change line 29 to just point to whatever .a or .so is there.  

That command gives me nada.

[etotheipi]

For what it's worth, a couple of months ago somebody recommended removing a large chunk of line 15 and it allowed it to fully compile. But since I'm doing a fresh install I thought it would be a better idea to run it by you first.

Line 15 is what makes this makefile work regardless of what python version you have installed.  Unless you have a non-standard python installation.

What shows up when you type "ls -la /usr/lib | grep libpython"?  

You can change line 29 to just point to whatever .a or .so is there.  

That command gives me nada.

Okay, well figure out where your libpython.a or libpython.so file is.  That's what line 29 needs to reference.  I guess it's in another place in Red Hat...?