[ANN] BitcoinArmory-Daemon - armory on web servers

[pvz]

I have recently pushed changes to the daemon to bring it up to date with the latest armoryengine.py

Unfortunately I am unable to load the blockchain, and am getting these errors.

Code:

(ERROR) armoryengine.py:11362 - Error processing BDM input
(ERROR) armoryengine.py:11363 - Received inputTuple: StartScanRequested [8, 28474661, False]
(ERROR) armoryengine.py:11364 - Error processing ID (28474661)

MAV thanks!

I have a fully operational bitcoind daemon (Debian 6) and also get the same errors.

Does the CppBlockUtils.py and/or _CppBlockUtils.so can stay on the same version or do they need a more accurate version also?

[etotheipi]

I have recently pushed changes to the daemon to bring it up to date with the latest armoryengine.py

Unfortunately I am unable to load the blockchain, and am getting these errors.

Code:

(ERROR) armoryengine.py:11362 - Error processing BDM input
(ERROR) armoryengine.py:11363 - Received inputTuple: StartScanRequested [8, 28474661, False]
(ERROR) armoryengine.py:11364 - Error processing ID (28474661)

MAV thanks!

I have a fully operational bitcoind daemon (Debian 6) and also get the same errors.

Does the CppBlockUtils.py and/or _CppBlockUtils.so can stay on the same version or do they need a more accurate version also?

That's definitely going to have to be recompiled. 

On that note: mav, did you change any of the C++ code?  Or were you just using the stock armory code, and just putting a wrapper around it?  If you just wrapped it, it may be better to just fork the BitcoinArmory repo and add your .py files to it.  Then updates are just a "make" away, and goes pretty smoothly on all linux distros (few dependencies, and no version requirements on them).

[mav]

I have recently pushed changes to the daemon to bring it up to date with the latest armoryengine.py

Unfortunately I am unable to load the blockchain, and am getting these errors.

Code:

(ERROR) armoryengine.py:11362 - Error processing BDM input
(ERROR) armoryengine.py:11363 - Received inputTuple: StartScanRequested [8, 28474661, False]
(ERROR) armoryengine.py:11364 - Error processing ID (28474661)

MAV thanks!

I have a fully operational bitcoind daemon (Debian 6) and also get the same errors.

Does the CppBlockUtils.py and/or _CppBlockUtils.so can stay on the same version or do they need a more accurate version also?

That's definitely going to have to be recompiled. 

On that note: mav, did you change any of the C++ code?  Or were you just using the stock armory code, and just putting a wrapper around it?  If you just wrapped it, it may be better to just fork the BitcoinArmory repo and add your .py files to it.  Then updates are just a "make" away, and goes pretty smoothly on all linux distros (few dependencies, and no version requirements on them).

Aaaaack been a long time since I looked at this and forgot to copy the new _CppBlockUtils.so to the repo. I have not changed any c++ code, this is just a wrapper. I am going to put this on hold until the changes are made. Then I will properly re-assess the situation and put this in a fork of BitcoinArmory rather than as a standalone daemon with the .so binary included.

[etotheipi]

I have recently pushed changes to the daemon to bring it up to date with the latest armoryengine.py

Unfortunately I am unable to load the blockchain, and am getting these errors.

Code:

(ERROR) armoryengine.py:11362 - Error processing BDM input
(ERROR) armoryengine.py:11363 - Received inputTuple: StartScanRequested [8, 28474661, False]
(ERROR) armoryengine.py:11364 - Error processing ID (28474661)

MAV thanks!

I have a fully operational bitcoind daemon (Debian 6) and also get the same errors.

Does the CppBlockUtils.py and/or _CppBlockUtils.so can stay on the same version or do they need a more accurate version also?

That's definitely going to have to be recompiled. 

On that note: mav, did you change any of the C++ code?  Or were you just using the stock armory code, and just putting a wrapper around it?  If you just wrapped it, it may be better to just fork the BitcoinArmory repo and add your .py files to it.  Then updates are just a "make" away, and goes pretty smoothly on all linux distros (few dependencies, and no version requirements on them).

Aaaaack been a long time since I looked at this and forgot to copy the new _CppBlockUtils.so to the repo. I have not changed any c++ code, this is just a wrapper. I am going to put this on hold until the changes are made. Then I will properly re-assess the situation and put this in a fork of BitcoinArmory rather than as a standalone daemon with the .so binary included.

Actually, this may not be so bad.  I was busy preaching about all the changes and how it was all worth the effort given the usability improvement for Armory GUI ... but actually you may be relatively unaffected as long as you set blocking=True.  In general, the GUI is advancing, not the library, so you don't have to upgrade like this often.

I just browsed your code and didn't see a whole lot that has to be changed, except you need to replace "TheBDM.LoadBlockchain()" with "TheBDM.setBlocking(True);  TheBDM.setOnlineMode(True)".  There might be a couple minor things that you have to update, and please post here with any more issues and I bet we can clear them up quickly.  But overall you don't appear to be using many things that changed.

[pvz]

I just browsed your code and didn't see a whole lot that has to be changed, except you need to replace "TheBDM.LoadBlockchain()" with "TheBDM.setBlocking(True);  TheBDM.setOnlineMode(True)".  There might be a couple minor things that you have to update, and please post here with any more issues and I bet we can clear them up quickly.  But overall you don't appear to be using many things that changed.

With this change I get the following error:

Code:

(ERROR) armoryengine.py:11363 - Received inputTuple: GoOnlineRequested [13, 31616721, True]

If I set:

Code:

TheBDM.setBlocking(True);  TheBDM.setOnlineMode(FALSE)

There are no errors, also armory-daemon does not get started.
Well...I think it does not get started because I do not receive this message:

Code:

Loading blockchain
Initialising server
Server started

The process seems like to hang. Most of the time a ^C helps me out, now I have to shut down the terminal.

I expect the online mode needs a TheBDM.setOnlineMode(true) but I can't get to the code triggering the error.

By the way, I am not able to compile the CppBlockUtils.py or _CppBlockUtils.so code.

[etotheipi]

By the way, I am not able to compile the CppBlockUtils.py or _CppBlockUtils.so code.

Then how are you running with the new code?  The .so most definitely needs to be recompiled to use the new stuff.  I would expect seg faults, not hanging, but computers are weird....

I recommend checking out the current Armory branch to a different directory, type "make", and then copy the CppBlockUtils.py and _CppBlockUtils.so over to the BitcoinArmory-Daemon directory.  Then it should work.

[pvz]

The .so most definitely needs to be recompiled to use the new stuff.

You are right, this was the problem.
Thanks!

Status on my Debian 6 server:
- operational bitcoind daemon
- download BitcoinArmory-Daemon (https://github.com/thedawnrider/BitcoinArmory-Daemon.git)
  with latest armoryengine.py by mav
  (added Armory and Armory-Daemon dependencies)
- Replaced some code in armory-daemon.py (line 217 https://bitcointalk.org/index.php?topic=92496.msg1437378#msg1437378 by etotheipi)
- download and make BitcoinArmory (https://github.com/etotheipi/BitcoinArmory.git)
- replace CppBlockUtils.py and _CppBlockUtils.so on Armory-Daemon from BitcoinArmory
-> start Armory-Daemon ($ python armory-daemon.py)

Now starting to explore how the armory-daemon works...

[mav]

Great news, web-armory people! The daemon is being included with armory itself so will be up to date and more fully featured than I ever could have achieved.

see https://bitcointalk.org/index.php?topic=56424.msg1443054#msg1443054 for details.

I will keep this thread open to discuss the particulars of using armory on webservers, as the daemon is not strictly only for use on webservers (may be used for, say, running your uniquely common yet complex task easily, or remotely controlling your armory application).

[CashBitango]

Hi

I guess this is really basic, but I still need some help getting this script running.
So I'm running this on a linux machine, I've installed python, and I'm trying to run the script.

$ ./armoryd.py
from: can't read /var/mail/twisted.internet
from: can't read /var/mail/twisted.web
from: can't read /var/mail/txjsonrpc.web
from: can't read /var/mail/txjsonrpc.auth
from: can't read /var/mail/twisted.cred.checkers
from: can't read /var/mail/armoryengine
./armoryd.py: line 51: import: command not found
./armoryd.py: line 52: import: command not found
./armoryd.py: line 53: import: command not found
./armoryd.py: line 54: import: command not found
./armoryd.py: line 55: import: command not found
./armoryd.py: line 56: import: command not found
./armoryd.py: line 59: import: command not found
from: can't read /var/mail/jsonrpc
./armoryd.py: line 61: syntax error near unexpected token `('
./armoryd.py: line 61: `class UniversalEncoder(json.JSONEncoder):'
$

So then I installed twisted:
$ sudo apt-get install python-twisted

But I still get the same message.
Trying to run it on Windows I get the same import problem.

Any suggestions?

Question2
When I get the script running and can use it to create offline transactions etc, can I have it interact with another program in another language I've written?
If so, can I do it over localhost, and send RPC or something, getting replies in json?

[etotheipi]

You are trying to run it as a bash script.  You need to run it as a python script:

"python armoryd.py".

Also, as someone else noted, you'll need to download the txjsonrpc project and put it in your base directory (there should be a txjsonrpc dir next to armoryd.py).  Or actually get txjsonrpc installed -- it's an extra dependency not needed by regular Armory but needed for armoryd.py.

[CircusPeanut]

Question2
When I get the script running and can use it to create offline transactions etc, can I have it interact with another program in another language I've written?
If so, can I do it over localhost, and send RPC or something, getting replies in json?

I haven't had a need to use JSON across languages yet, but I understand that is one of it's features.

From http://en.wikipedia.org/wiki/JSON

Although JSON was originally based on a non-strict subset of the JavaScript scripting language (specifically, Standard ECMA-262 3rd Edition—December 1999[6]) and is commonly used with that language, it is a language-independent data format. Code for parsing and generating JSON data is readily available for a large variety of programming languages. JSON's Web site provides a comprehensive listing of existing JSON libraries, organized by language.

[CashBitango]

Hi

Thanks for the help thus far, but I still have problems. Sorry for this massive post, but I think it makes it easier to debug.

I'm trying to get this thing working on both windows and linux, but encounter difficulties on both.

1) Windows:
I put the  txjsonrpc into the suggested folder, I used this one: https://github.com/oubiwann/txjsonrpc
I installed twisted, and that required zope, so I installed that too.
Then I installed psutils-1.2.1 and confirmed that it's working.
I'm running this thing in Python 2.7.6, opening the file from the shell, then hitting run (F5)

So now I get this error:
----------------------
(CRITICAL) armoryengine.py:1043 - C++ block utilities not available.
(CRITICAL) armoryengine.py:1044 -    Make sure that you have the SWIG-compiled modules
(CRITICAL) armoryengine.py:1045 -    in the current directory (or added to the PATH)
(CRITICAL) armoryengine.py:1046 -    Specifically, you need:
(CRITICAL) armoryengine.py:1047 -        CppBlockUtils.py     and
(CRITICAL) armoryengine.py:1051 -        _CppBlockUtils.pyd

Traceback (most recent call last):
  File "X:\BitcoinArmory-master\armoryd.py", line 49, in <module>
    from armoryengine import *
  File "X:\BitcoinArmory-master\armoryengine.py", line 1039, in <module>
    import CppBlockUtils as Cpp
ImportError: No module named CppBlockUtils
----------------------

Any suggestions?
I'm using the latest source from https://github.com/etotheipi/BitcoinArmory, downloaded as zip.

2) Ubuntu 12.04
I put the txjsonrpc into the suggested folder, I used this one: https://github.com/oubiwann/txjsonrpc
I tried installing psutils-1.2.1:
----------------------
$ python setup.py  install
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'
  warnings.warn(msg)
running install
running build
running build_py
creating build
creating build/lib.linux-i686-2.7
creating build/lib.linux-i686-2.7/psutil
copying psutil/_pslinux.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psbsd.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_compat.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psosx.py -> build/lib.linux-i686-2.7/psutil
copying psutil/__init__.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_pssunos.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_error.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psmswindows.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_psposix.py -> build/lib.linux-i686-2.7/psutil
copying psutil/_common.py -> build/lib.linux-i686-2.7/psutil
copying psutil/error.py -> build/lib.linux-i686-2.7/psutil
running build_ext
building '_psutil_linux' extension
creating build/temp.linux-i686-2.7
creating build/temp.linux-i686-2.7/psutil
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c psutil/_psutil_linux.c -o build/temp.linux-i686-2.7/psutil/_psutil_linux.o
psutil/_psutil_linux.c:13:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------
Since install failed, I put psutils into the same folder that I'm trying to run the armoryd.py from, then try to run it:
----------------------
$ python armoryd.py
Traceback (most recent call last)
  File "armoryd.py", line 49, in <module>
    from armoryengine import *
  File "/home/linuxuser/Documents/arm/armoryengine.py", line 47, in <module>
    import psutil
  File "/home/linuxuser/Documents/arm/psutil/__init__.py", line 88, in <module>
    import psutil._pslinux as _psplatform
  File "/home/linuxuser/Documents/arm/psutil/_pslinux.py", line 20, in <module>
    import _psutil_linux
ImportError: No module named _psutil_linux
$
----------------------

More suggestions?

[CashBitango]

Question2
When I get the script running and can use it to create offline transactions etc, can I have it interact with another program in another language I've written?
If so, can I do it over localhost, and send RPC or something, getting replies in json?

I haven't had a need to use JSON across languages yet, but I understand that is one of it's features.

From http://en.wikipedia.org/wiki/JSON

Although JSON was originally based on a non-strict subset of the JavaScript scripting language (specifically, Standard ECMA-262 3rd Edition—December 1999[6]) and is commonly used with that language, it is a language-independent data format. Code for parsing and generating JSON data is readily available for a large variety of programming languages. JSON's Web site provides a comprehensive listing of existing JSON libraries, organized by language.

OK, I see I wasn't expressing myself clear.

The question wasn't about JSON per se, it was about how to interact with armory. I have written software in Java, and I need it to interact with Armory, to create an offline transaction.
I want my program to be able to connect for example to localhost:8999, send command jsonrpc_sendmany([array of bitcoin addresses : amount to send])
Then I'd like to get in return the offline transaction, unsigned, into my program, and my program stores file to disk/USB or whatever. Then I want to take that offline transaction to an offline computer and sign it before broadcasting it from an online computer.

Think it can be done?

[CircusPeanut]

...
1) Windows:
...
So now I get this error:
----------------------
(CRITICAL) armoryengine.py:1043 - C++ block utilities not available.
(CRITICAL) armoryengine.py:1044 -    Make sure that you have the SWIG-compiled modules
(CRITICAL) armoryengine.py:1045 -    in the current directory (or added to the PATH)
(CRITICAL) armoryengine.py:1046 -    Specifically, you need:
(CRITICAL) armoryengine.py:1047 -        CppBlockUtils.py     and
(CRITICAL) armoryengine.py:1051 -        _CppBlockUtils.pyd

Traceback (most recent call last):
  File "X:\BitcoinArmory-master\armoryd.py", line 49, in <module>
    from armoryengine import *
  File "X:\BitcoinArmory-master\armoryengine.py", line 1039, in <module>
    import CppBlockUtils as Cpp
ImportError: No module named CppBlockUtils
----------------------

Any suggestions?

You are missing _CppBlockUtils.pyd and CppBlockUtils.py. You can either get them from your armory installation or build cppForSwig to produce these files.

More detailed instructions are here:
https://bitcoinarmory.com/download/building-armory-from-source/

Re: Question 2...I don't have any experience building on linux so I can't help trouble shoot that.

....I have written software in Java, and I need it to interact with Armory, to create an offline transaction.
I want my program to be able to connect for example to localhost:8999, send command jsonrpc_sendmany([array of bitcoin addresses : amount to send])
Then I'd like to get in return the offline transaction, unsigned, into my program, and my program stores file to disk/USB or whatever. Then I want to take that offline transaction to an offline computer and sign it before broadcasting it from an online computer.

Think it can be done?

Yes that can definitely be done. Send your command from the Java program and the result of any JSON method you call will return a text representation of the returned value or values (as a name/value mapping).

[etotheipi]

Getting armoryd working on windows will be much more challenging than linux (which is five commands plus getting txjsonrpc).  I'm not sure we've even tried armoryd.py in Windows without having the whole build environment setup.   As CircusPeanut said, you might only need to get the _CppBlockUtils.pyd and .py from a preinstalled version of Armory, but there might be other .dlls you need as well.  Maybe just copy those two files plus all the .dlls from the installed Armory dir, into your checkout directory.  Or use Linux... which is dramatically easier to setup.


We should have a function in there that gets you an unsigned transaction for a list of recipients.  To make the signing process easier, you can look at extras/cli_sign_txdp.py which is a full-featured, command-line interface for signing offline transactions.  The two arguments for it are wallet file and a unsigned.tx files from Armory. 

Be aware that the master branch may not have all the armoryd.py features implemented.  The testing branch will have all the updated features, but of course is the unstable version.  Hopefully we'll have that ironed out in a couple weeks.

[halfawake]

I just now noticed this thread.  This is an amazing feature, given all the hacks / thefts that I've seen, this kind of webserver daemon seems like it'd be the holy grail of bitcoin daemons since it'd be effectively unhackable.

[CashBitango]

Hi guys

Thanks for all your support in trying to get Armory working on server, but it's still a bit difficult for me.

I need it to work easily in different environments... No hard setup or installs.

I have an idea though. If you guys tell me what classes and methods I should be looking at in the source, I might be able to write my own version in Java. So I'll look at your code, figure out what it does to produce the unsigned transaction, and then I'll write some code that replicates the output from the same input.

If I get it working, I'll send you the source files so you can let others use them too, if you want that.
I'm writing the tests as well of course, so that anyone can see that they get expected result.

I'm imagining something like creating a class and method in Java:

public String getArmoryUnsignedTransaction(String[] unspentInput, String[] addressesTo, long amountSatoshi){
           //Do some magic
           return theWholeTransactionAsASingleString;
}

Would you be able to point me to some folders, classes and methods in the source where I can see what's going on when creating the unsigned transaction?

[CircusPeanut]

Here is a link to the latest code:
https://github.com/etotheipi/BitcoinArmory/blob/0.91-dev/armoryd.py

[gokudev]

Damnit!  I wish I could make the serial port idea work...

You and me both! I think I'll be looking into getting the Raspberry Pi to act as a USB client. It would then be necessary to use the USB protocol, which is a lot more complicated than just serial, but would not - as far as I know - allow login to the offline machine.

EDIT: I just discovered that the Raspberry Pi Model A (as opposed to Model B that is out now) will support acting as a USB device/client, because it uses a different communication-chip than Model B. Model A doesn't have an Ethernet port, and only has a single USB port, but I figure this might even be an advantage, since after the software has been installed on the device (using, for example, some USB-to-Ethernet type device to update packages), the Ethernet port will have to be removed because the USB connection needs to be used to communicate with the online computer. Perhaps one will be less tempted - and feel safer - if the device doesn't even have a port to plug in an Ethernet cable.
Additionally, it costs only $25. That's really not much for the ultimate in safe Bitcoin wallets. That being said, Model A isn't out now. And USB client mode won't necessarily work out-of-the-box. This is a software issue, as far as I can gather. Not OS level, I think, but firmware level - so it might not be that easy to implement. But still, looks like a device is coming to the market that fulfils the requirements of an offline hardware wallet. Off the top of my head, I can't see what else one would need other than a Raspberry Pi Model A, and perhaps a USB cable. But most people have USB cables lying around. Sure beats that $40 2xUSB-to-serial-plus-null-modem solution.
[/quote]

Wonder where things are at with serial ports. I own a rasp pi model A and I like the idea of using it for running armory in offfline mode. The rasp pi model A can work with serial ports no problem using a shield. Has there been any development on armory for making serial ports work? I gather we have to send the 'unsigned transaction' over from online armory to an offline pc over a serial connection?

[gokudev]

This is why I prevent Armory from opening twice.  Wallet operations are not threadsafe. Not at all!  

However, one benefit of python-twisted is that it is guaranteed to run as a single-thread, so as long as there is only one instance of Armory, it's perfectly safe.

For your purposes, not using python-twisted, need to do something else.  Actually, I would recommend using python-twisted to create a server/daemon process to handle (serially) all requests for wallet interaction.  Make sure everything goes through that interface.  The wallet design has been remarkably robust (I even test it with mid-write interrupts and it recovers fine).  But as you can see, two simultaneous operations can break it pretty easily.  

Wallet operations are not threadsafe. Not at all!  
can someone please explain why armory wallets are not thread safe and why armory runs as a singleton? please explain in detail.