Armory - Discussion Thread

[etotheipi]

@Ente,

One other thing I could use is more testing.   I really haven't been getting the quantity of eyes on testing versions as I need to catch all these crazy things.  A lot of silly bugs get by me because I can only test such a small subset of everything.

On that note, I just figured out the tx/balance doubling bug:  it is only when you have freshly created a new wallet and receive coins.  It's because the wallet is somehow getting added twice to the application.  I think I can fix that bug and put it into a 0.86 release.  This seems important to fix for new users getting into Armory the first time...

[1713585484]

1713585484

[1713585484]

1713585484

[etotheipi]

I got it yesterday on my Mac, but first it printed that it had lost connection to the Satoshi client, so I assumed that was the problem.

I think losing connection is a symptom, not a cause.  I just put a ton of debug statements in and will start it up and hopefully get a crash in the not-too-distant future.

Got another one of those just now, two in one day. Could you make an executible with the added debug statements so we can help?

prezbo,

Are you compiling from source?  I just put the BlockUtils.cpp with debugging calls in my dropbox folder, and it can be copied right into the cppForSwig directory and recompiled.  It's a P.I.T.A. to debug this stuff, because I can't compile in MSVS in debug mode (because I get crazy linker errors for things like not having python_debug.dll, I haven't figured it out, yet), and I can debug in Linux, but the error doesn't seem to happen there.  

So, the best I can do at the moment, is have it spit output to the console every other line, to find out where the problem is (and the state of some variables when it happened).  I might be able to figure it out just from the output... at the very least I can do more intelligent debugging when I figure out the location.  I am putting it out there right now, because I just ran it on my Win7 VM for 36 hours and it hasn't crashed yet...


I found and fixed the bug causing double-display in new wallets.  That's a particularly important bug to fix with the recent influx of new users.  That along with coin control is worth a full release, even without this once-per-24hours-crash bug fixed.   I am in process right now, of compiling and signing the installers.  I will also see if I can get git installed on the offline computer and sign the "v0.86-beta" tag, too (after I create it).

I also figured out why so many users were having trouble with the installers in Windows -- I have been trying to push people onto the 32-bit version, and I think when it installs it skips files it thinks are already there -- including things like python.dll which is 64-bit but then replacing everything else with the 32-bit version.  This is also why a repair/reinstall always fixes it.  I'll see if I can figure out how to force the installer to replace everything.

[chrisrico]

Just ran into an issue with a few transactions which have been included in a block being stuck at 0 confirmations. I renamed mempool.bin, restarted, and then they weren't in the transaction list at all. What can I give you that would help you figure out the cause?

edit... I'll email you the log, a watching only copy of the wallet, and mempool.bin

[K1773R]

I got it yesterday on my Mac, but first it printed that it had lost connection to the Satoshi client, so I assumed that was the problem.

I think losing connection is a symptom, not a cause.  I just put a ton of debug statements in and will start it up and hopefully get a crash in the not-too-distant future.

Got another one of those just now, two in one day. Could you make an executible with the added debug statements so we can help?

prezbo,

Are you compiling from source?  I just put the BlockUtils.cpp with debugging calls in my dropbox folder, and it can be copied right into the cppForSwig directory and recompiled.  It's a P.I.T.A. to debug this stuff, because I can't compile in MSVS in debug mode (because I get crazy linker errors for things like not having python_debug.dll, I haven't figured it out, yet), and I can debug in Linux, but the error doesn't seem to happen there.  

So, the best I can do at the moment, is have it spit output to the console every other line, to find out where the problem is (and the state of some variables when it happened).  I might be able to figure it out just from the output... at the very least I can do more intelligent debugging when I figure out the location.  I am putting it out there right now, because I just ran it on my Win7 VM for 36 hours and it hasn't crashed yet...


I found and fixed the bug causing double-display in new wallets.  That's a particularly important bug to fix with the recent influx of new users.  That along with coin control is worth a full release, even without this once-per-24hours-crash bug fixed.   I am in process right now, of compiling and signing the installers.  I will also see if I can get git installed on the offline computer and sign the "v0.86-beta" tag, too (after I create it).

I also figured out why so many users were having trouble with the installers in Windows -- I have been trying to push people onto the 32-bit version, and I think when it installs it skips files it thinks are already there -- including things like python.dll which is 64-bit but then replacing everything else with the 32-bit version.  This is also why a repair/reinstall always fixes it.  I'll see if I can figure out how to force the installer to replace everything.

i can confirm this happens on linux too! it happens only if the harddisk is under full load and bitcoind dosnt respond fast enough (just a guess that this is the problem). Armory thinks he lost the connection due to a short timeout setting (just guessing). is this possible?

[etotheipi]

Just ran into an issue with a few transactions which have been included in a block being stuck at 0 confirmations. I renamed mempool.bin, restarted, and then they weren't in the transaction list at all. What can I give you that would help you figure out the cause?

I'm not sure how that's possible... if it's in the blockchain there's no way to not detect it on a restart.  Unless Bitcoin-Qt isn't fully synchronized.

To be clear, mempool.bin holds only zero-conf tx.  The zero-conf tx is deleted when the same tx is included in the blockchain.  If you delete mempool.bin before it makes it into the blockchain, then the tx won't show up at all, until it has 1+ confirmation.   So if it doesn't show up at all, then... are you sure it made it into a block?  Do you have access to the txid in order to look for it on blockchain.info.  I'd be very surprised if the tx is in the blockchain, is relevant to any your wallets, Bitcoin-Qt is sync'd, AND it doesn't show up. 

I guess an alternative explanation would be something to do with your wallet not recognizing that the tx is relevant to it:  perhaps if somehow your keypool wasn't big enough... did you generate 100s of empty addresses before this one?  However, expert mode now lets you extend the keypool manually.  I doubt that's your problem (especially because the tx showed up before, but it's worth trying if all else fails).

i can confirm this happens on linux too! it happens only if the harddisk is under full load and bitcoind dosnt respond fast enough (just a guess that this is the problem). Armory thinks he lost the connection due to a short timeout setting (just guessing). is this possible?

Hmm, I've never seen it happen in linux.  I guess my HDD just doesn't get overwhelmed like that.  However, the disconnect-reconnect fluctuations are a symptom of the problem, not the cause.  Perhaps I should add a mechanism to disable the flickering if there's too much...

The real problem is why it's taking longer than 10s to do something that normally takes 0.2 ms.  My guess is it isn't a "legitimate" failure (HDD taking 2500 times longer than normal), it could be an infinite loop in the readBlkFileUpdate method, or it's somehow inducing a rescan when it's only supposed to be doing a quick update.  The debug output should tell me the last line that was run in readBlkFileUpdate, and also tell me the state of a bunch of different variables at that time.

[K1773R]

Just ran into an issue with a few transactions which have been included in a block being stuck at 0 confirmations. I renamed mempool.bin, restarted, and then they weren't in the transaction list at all. What can I give you that would help you figure out the cause?

I'm not sure how that's possible... if it's in the blockchain there's no way to not detect it on a restart.  Unless Bitcoin-Qt isn't fully synchronized.

To be clear, mempool.bin holds only zero-conf tx.  The zero-conf tx is deleted when the same tx is included in the blockchain.  If you delete mempool.bin before it makes it into the blockchain, then the tx won't show up at all, until it has 1+ confirmation.   So if it doesn't show up at all, then... are you sure it made it into a block?  Do you have access to the txid in order to look for it on blockchain.info.  I'd be very surprised if the tx is in the blockchain, is relevant to any your wallets, Bitcoin-Qt is sync'd, AND it doesn't show up. 

I guess an alternative explanation would be something to do with your wallet not recognizing that the tx is relevant to it:  perhaps if somehow your keypool wasn't big enough... did you generate 100s of empty addresses before this one?  However, expert mode now lets you extend the keypool manually.  I doubt that's your problem (especially because the tx showed up before, but it's worth trying if all else fails).

i can confirm this happens on linux too! it happens only if the harddisk is under full load and bitcoind dosnt respond fast enough (just a guess that this is the problem). Armory thinks he lost the connection due to a short timeout setting (just guessing). is this possible?

Hmm, I've never seen it happen in linux.  I guess my HDD just doesn't get overwhelmed like that.  However, the disconnect-reconnect fluctuations are a symptom of the problem, not the cause.  Perhaps I should add a mechanism to disable the flickering if there's too much...

The real problem is why it's taking longer than 10s to do something that normally takes 0.2 ms.  My guess is it isn't a "legitimate" failure (HDD taking 2500 times longer than normal), it could be an infinite loop in the readBlkFileUpdate method, or it's somehow inducing a rescan when it's only supposed to be doing a quick update.  The debug output should tell me the last line that was run in readBlkFileUpdate, and also tell me the state of a bunch of different variables at that time.

drive the HD load to 100% (dd if=/dev/zero of=$HOME/junkfile bs=1024M count=10), at this time i always see lost and reconnect in a loop.

[etotheipi]

drive the HD load to 100% (dd if=/dev/zero of=$HOME/junkfile bs=1024M count=10), at this time i always see lost and reconnect in a loop.

That's an interesting observation on its own.  I'm not sure why HDD stress would cause networking hiccups.  Does it also cause Armory to crash (eventually)? 

However, I'm not convinced that is the source of the crashing problem.  I'm fairly confident that there is something else (i.e. a inf loop bug or something) causing readBlkFileUpdate() to take longer than 10s.  It just doesn't seem possible, even with a ridiculously slow HDD, that it would take more than 10s to read a few kB from disk.

Sorry, don't mean to discount your observation -- I do need to fix stupid networking problems like that.  But my gut tells me there's another, subtler issue...

[K1773R]

drive the HD load to 100% (dd if=/dev/zero of=$HOME/junkfile bs=1024M count=10), at this time i always see lost and reconnect in a loop.

That's an interesting observation on its own.  I'm not sure why HDD stress would cause networking hiccups.  Does it also cause Armory to crash (eventually)? 

However, I'm not convinced that is the source of the crashing problem.  I'm fairly confident that there is something else (i.e. a inf loop bug or something) causing readBlkFileUpdate() to take longer than 10s.  It just doesn't seem possible, even with a ridiculously slow HDD, that it would take more than 10s to read a few kB from disk.

Sorry, don't mean to discount your observation -- I do need to fix stupid networking problems like that.  But my gut tells me there's another, subtler issue...

no armory never crashed for me. u forget the cache from linux for the HDs, if you fill them up then you cant read until everything is written, if the cache is big enough (as for me) then it takes sometimes longer than 10s or if the HD is really slow. just try it yourself

[N.Z.]

etotheipi, I have a question...maybe you will find it strange and offensive, please excuse my ignorance.

Can I really trust Armory?
I mean, if I choose to use it as offline storage for relatively significant sum of bitcoins, what hypothetical risks will I face? Say I have paper backup and go to coma for 2-3-5 years, then I wake up. I can imagine some problems:
1. You will abandon developing
2. Some sort of attack to encryption
3. Attack to keys generation
4. Critical bug

Am I right? I think it is more philosophic question, or about intuition and professional experience. Is there a possibility that I will lose my bitcoins? Thanks.

[Red Emerald]

etotheipi, I have a question...maybe you will find it strange and offensive, please excuse my ignorance.

Can I really trust Armory?
I mean, if I choose to use it as offline storage for relatively significant sum of bitcoins, what hypothetical risks will I face? Say I have paper backup and go to coma for 2-3-5 years, then I wake up. I can imagine some problems:
1. You will abandon developing

Even if development is abandoned, the version you already have installed will probably still work.  If bitcoind (or even the entire bitcoin network) has changed in a way where Armory is not compatible, then you could still load the program offline, export your private keys and import them into newer software.

2. Some sort of attack to encryption
3. Attack to keys generation
4. Critical bug

Am I right? I think it is more philosophic question, or about intuition and professional experience. Is there a possibility that I will lose my bitcoins? Thanks.

These questions don't really have any more to do with Armory than they do with any piece of software in existence. 

tl;dr The armory software and the private keys are pretty separate, so no, there is no risk of losing coins even if etotheipi were to pull a Satoshi and disappear on us.  Barring some attack that completely destroys the bitcoin network, your funds are fine.

[etotheipi]

etotheipi, I have a question...maybe you will find it strange and offensive, please excuse my ignorance.

Can I really trust Armory?
I mean, if I choose to use it as offline storage for relatively significant sum of bitcoins, what hypothetical risks will I face? Say I have paper backup and go to coma for 2-3-5 years, then I wake up. I can imagine some problems:
1. You will abandon developing

Even if development is abandoned, the version you already have installed will probably still work.  If bitcoind (or even the entire bitcoin network) has changed in a way where Armory is not compatible, then you could still load the program offline, export your private keys and import them into newer software.

2. Some sort of attack to encryption
3. Attack to keys generation
4. Critical bug

Am I right? I think it is more philosophic question, or about intuition and professional experience. Is there a possibility that I will lose my bitcoins? Thanks.

These questions don't really have any more to do with Armory than they do with any piece of software in existence.  

tl;dr The armory software and the private keys are pretty separate, so no, there is no risk of losing coins even if etotheipi were to pull a Satoshi and disappear on us.  Barring some attack that completely destroys the bitcoin network, your funds are fine.

Exactly.  More explicitly: let's say I were to magically disappear and there was never another release of Armory AND Bitcoin-Qt changed in such a way that Armory didn't work anymore.  What do you do?

• (1) Fire up the last version of Armory you have
• (2) Go into your wallet and click on "Backup Individual Keys"
• (3) Copy and paste your private keys into any other program or service that let's you import keys (blockchain.info is one)
• (4) Continue using your Armory addresses from that other program or service.

So, there is no risk to your funds if I were to disappear -- you would just have to export all your keys to another app.  

And as RE said:  the question about security is really a risk of any program.  If Armory were designed poorly under-the-hood, it may be more susceptible than other programs.  I'd like to think Armory is far above average.  One thing users could do is donate to pay for third-party security auditing of the Armory code.  Even if it were to turn out that there was some problems, the audit would identify them and I could fix them.  But, as far as I know, I've followed most good practices for secure software -- making sure unencrypted data never touches the disk, using well-established encryption techniques, sufficient key-stretching, passing around private keys in RAM using self-destructing containers, etc.

Bugs are also a risk with any program -- especially financial software.  But two things about Armory in this respect:

(1) There's redundancy:  make a paper or digital backup, and you're protected against the 99% most common type of bug:  wallet corruption (or HDD loss)
(2) I have 1,000 lines of unit-testing (unittest.py) just for Armory wallets.  And it paid off: in the last 10 months, that code has hardly been touched because it's so stable and does what it's supposed to.

I hope you can sleep easier, now

[Red Emerald]

So bitcoin 0.8 has some changes for the blockchain storage IIRC.  Will this break your blockchain loading?  Have you started working on support for this?

[Foxpup]

So bitcoin 0.8 has some changes for the blockchain storage IIRC.  Will this break your blockchain loading?  Have you started working on support for this?

As I understand, the format of the blockchain data itself isn't changing, just the database that indexes it (which Armory doesn't use at all), so there should be no problems.

[N.Z.]

Red Emerald, etotheipi, I can sleep easier now, thanks a lot for explanation

[fabrizziop]

Hi. I'm having python27.dll error issues with Armory 0.86, it says it isn't found but I have python installed and those DLLs are in my system32 and syswow64 folders. Every time I install Armory I must copy the DLL from one of those folders and paste it on the armory program files directory. I don't know why does that happen.

[etotheipi]

So bitcoin 0.8 has some changes for the blockchain storage IIRC.  Will this break your blockchain loading?  Have you started working on support for this?

I'm ten steps ahead of you    I already talked to the Bitcoin-Qt devs and got the info, and accommodated it in the code.  You'll see in the 0.85-beta version notes "Preparation for Bitcoin-Qt 0.8+".  It was kind of a pain, because, Armory has to accommodate both pre-0.8 and post-0.8, so I had to autodetect the files and select the correct one. 

For reference, the current structure is that blocks are stored in ".bitcoin/blk1234.dat" of 2GB each.  0.8 will put them in ".bitcoin/blocks/blk01234.dat" of 128 MB each (now zero-indexed, and five digits instead of four).

Hi. I'm having python27.dll error issues with Armory 0.86, it says it isn't found but I have python installed and those DLLs are in my system32 and syswow64 folders. Every time I install Armory I must copy the DLL from one of those folders and paste it on the armory program files directory. I don't know why does that happen.

I believe the issue has to do with Armory shipping with it's own python27.dll.  If you switch between the 32-bit and 64-bit, I think the installer sees the .dll already there and doesn't replace it, not realizing it's leaving you with an incompatible dll.  If you uninstall-reinstall, or "repair" the installation, I think it should work.  Please let me know if that's not the case.

[chrisrico]

So I'm finally quite happy with my offline wallet setup, and I wanted to share it with others.

I have been using a Rapsberry Pi to run the offline instance of Armory. It was hooked up to my monitor through HDMI, and I had an extra keyboard around. When I wanted to perform an offline transaction, I would create it on my main computer, put it on a USB drive, plug the keyboard and USB drive into the Pi, run Armory, sign the transaction, transfer it back to the main computer, and broadcast it.

Last week, I purchased a USB to TTL, and that whole process changed for the better. The cable provides power to the Pi and allows for a terminal login to the device, so I no longer need an HDMI cable, power cable, or keyboard. I merely keep it unplugged, and plug it in when I want to perform a transaction. I log in using screen (screen /dev/ttyUSB0 115200), then run this script which does the following...

Opens a new file called armory.unsigned.tx in Nano, allowing me to paste in the ascii-serialized transaction
Runs this script against the unsigned file
If the file was signed, then display it using Less (at which point I copy it and paste into the broadcast transaction form)
Shutdown the Pi

I'm pretty sure that the TTY device only allows for a single login at a time, so I don't think it would be possible, and certainly not feasible, for an attacker to steal bitcoins from the Pi during the short time that it's connected to my computer. Any thoughts?

[Cryptoman]

Congratulations to etotheipi for reaching beta stage on this fantastic Bitcoin client.  What follows are build instructions for OpenSUSE 12.2 (64-bit).  I did a clean install of SUSE with default packages to find out which were needed for a plain-vanilla install.  If you've been using your system for development, chances are you won't need all of the packages.

1. Install the following packages using your preferred package manager (zypper, YaST, rpm, etc.):

Code:

git-core
swig
make
gcc-c++
python-devel
python-twisted

2. Open a terminal window and get the source:

Code:

git clone git://github.com/etotheipi/BitcoinArmory.git

3. Edit BitcoinArmory/cppForSwig/Makefile and change:

Code:

STATICPYTHON += "$(DEPSDIR)/lib/libpython$(PYVER).a"

to:

Code:

STATICPYTHON +=   "$(DEPSDIR)/lib64/libpython$(PYVER).so"

4. Build and run as described in the Ubuntu instructions, viz:

Code:

cd BitcoinArmory/cppForSwig
make swig
cd ..
python ArmoryQt.py

[etotheipi]

Congratulations to etotheipi for reaching beta stage on this fantastic Bitcoin client.  What follows are build instructions for OpenSUSE 12.2 (64-bit).  I did a clean install of SUSE with default packages to find out which were needed for a plain-vanilla install.  If you've been using your system for development, chances are you won't need all of the packages.

1. Install the following packages using your preferred package manager (zypper, YaST, rpm, etc.):

Code:

git-core
swig
make
gcc-c++
python-devel
python-twisted

2. Open a terminal window and get the source:

Code:

git clone git://github.com/etotheipi/BitcoinArmory.git

3. Edit BitcoinArmory/cppForSwig/Makefile and change:

Code:

STATICPYTHON += "$(DEPSDIR)/lib/libpython$(PYVER).a"

to:

Code:

STATICPYTHON +=   "$(DEPSDIR)/lib64/libpython$(PYVER).so.1.0"

4. Build and run as described in the Ubuntu instructions, viz:

Code:

cd BitcoinArmory/cppForSwig
make swig
cd ..
python ArmoryQt.py

Fantastic!  Now I have SuSE, Gentoo, Ubuntu, Fedora (somewhere).  Maybe I'll start a list of such instructions on the website.  

Two things:
(1) I don't think you need ".so.1.0", I think ".so" is sufficient (since it's just a symlink to .so.1.0, probably).  
(2) Everyone can avoid the Makefile modifications if I could just figure out how to autodetect the .a file, and switch the .so file if the .a does not exist.  Unfortunately, my bash scripting is garbage (my failed attempt is commented out in the Makefile).  Perhaps someone with more bash experience could just fix it for me and then I'll update the repo with it.

This is the biggest "hurdle" for people compiling on non-Ubuntu systems, and I could make it go away if I just got the bash commands figured out.  Then your compile instructions would pretty much be the same as I already have on the webpage.

Anyone want to help with that?

[justusranvier]

Now I have SuSE, Gentoo, Ubuntu, Fedora (somewhere).  Maybe I'll start a list of such instructions on the website.

I'm working on setting up a user overlay for Gentoo so that you'll be able to have proper installation instructions for Gentoo.