RAM-Reduction & Backup Center Testing (version 0.89.99.16)

[simonk83]

Good stuff, looking forward to the OSX release

[japerry]

Armory version 0.89.99.3-testing:

Windows:  Download the standalone executable bundle (click here)  
Just unpack and double-click on ArmoryQt.exe.  You may want to right-click and make a shortcut to it, in the same directory, then modify the properties->"Target:" to use non-standard "--datadir" and/or "--satoshi-datadir" paths (or any other flags you want, like "--testnet).

I'm trying to set up the new Armory and Bitcoin-QT to run completely off a flash drive. Oh!! BTW thanks for the new Armory! I actually can run it now! 

So what is the difference between the --datadir and --satoshi-datadir command line calls?

[etotheipi]

I'm trying to set up the new Armory and Bitcoin-QT to run completely off a flash drive. Oh!! BTW thanks for the new Armory! I actually can run it now!  

So what is the difference between the --datadir and --satoshi-datadir command line calls?

--datadir is the Armory home directory to use
--satoshi-datadir is the Bitcoin-Qt/bitcoind home directory to use (which should match the "-datadir" flag you give to Bitcoin software if run it manually)

[japerry]

I'm trying to set up the new Armory and Bitcoin-QT to run completely off a flash drive. Oh!! BTW thanks for the new Armory! I actually can run it now! 

So what is the difference between the --datadir and --satoshi-datadir command line calls?

--datadir is the Armory home directory to use
--satoshi-datadir is the [/i]Bitcoin-Qt/bitcoind[/i] home directory to use (which should match the "-datadir" flag you give to Bitcoin software if run it manually)

Thank you sir! I will report back to you soon on the totally flash drive test. BTW The flash drive is a TrueCrypt container.

[etotheipi]

I'm trying to set up the new Armory and Bitcoin-QT to run completely off a flash drive. Oh!! BTW thanks for the new Armory! I actually can run it now! 

So what is the difference between the --datadir and --satoshi-datadir command line calls?

--datadir is the Armory home directory to use
--satoshi-datadir is the [/i]Bitcoin-Qt/bitcoind[/i] home directory to use (which should match the "-datadir" flag you give to Bitcoin software if run it manually)

Thank you sir! I will report back to you soon on the totally flash drive test. BTW The flash drive is a TrueCrypt container.

What kind of flash drive are you using?  The Bitcoin and Armory directories together will probably consume 30 GB of disk space, and of course will grow with the blockchain.  Just a reminder.

[japerry]

I'm trying to set up the new Armory and Bitcoin-QT to run completely off a flash drive. Oh!! BTW thanks for the new Armory! I actually can run it now! 

So what is the difference between the --datadir and --satoshi-datadir command line calls?

--datadir is the Armory home directory to use
--satoshi-datadir is the [/i]Bitcoin-Qt/bitcoind[/i] home directory to use (which should match the "-datadir" flag you give to Bitcoin software if run it manually)

Thank you sir! I will report back to you soon on the totally flash drive test. BTW The flash drive is a TrueCrypt container.

What kind of flash drive are you using?  The Bitcoin and Armory directories together will probably consume 30 GB of disk space, and of course will grow with the blockchain.  Just a reminder.

Yes! I got too big for my britches!! LOL!! I'm testing it on a removable 1 TB drive with a 128G Truecrypt container now. LOL!! A 16G won't do the job! But at least the drive is removable.

[TierNolan]

My bad.  Sorry for the scavenger hunt!  Put in LOGERROR calls, which will guarantee it gets written out to the log file:

Code:

      LOGERROR('***BLKDIR VARIABLE:')
      LOGERROR('***Type:  %s' % str(type(blkdir)))
      LOGERROR('***Size:  %d' % len(blkdir))
      LOGERROR('***Value: %s' % str(blkdir))

Breaking them out like that will also make sure the type() is printed even if len() or str() throws an error.

It didn't crash until bitcoind had downloaded the blockchain.

Code:

(INFO) ArmoryQt.py:1346 - Setting netmode: 1
(INFO) armoryengine.py:12324 - Setting online mode: True (wait=False)
(INFO) armoryengine.py:13249 - Go online requested
(INFO) armoryengine.py:12766 - Called __startLoadBlockchain()
(ERROR) armoryengine.py:12808 - ***BLKDIR VARIABLE:
(ERROR) armoryengine.py:12809 - ***Type:  <type 'unicode'>
(INFO) armoryengine.py:10353 - Connection initiated.  Start handshake
(ERROR) armoryengine.py:12810 - ***Size:  22
(ERROR) armoryengine.py:12811 - ***Value: F:\bitcoin_data\blocks
(ERROR) armoryengine.py:13274 - Error processing BDM input
(ERROR) armoryengine.py:13275 - Received inputTuple: GoOnlineRequested [13, 40873630, False]
(ERROR) armoryengine.py:13276 - Error processing ID (40873630)
(ERROR) armoryengine.py:13277 - ERROR:
Traceback (most recent call last):
  File "G:\armory_testing\ArmoryStandalone_0.89.99.2_win_all\ArmoryStandalone\library.zip\armoryengine.py", line 13260, in run
  File "G:\armory_testing\ArmoryStandalone_0.89.99.2_win_all\ArmoryStandalone\library.zip\armoryengine.py", line 12812, in __startLoadBlockchain
  File "CppBlockUtils.pyc", line 1230, in SetBlkFileLocation
TypeError: in method 'BlockDataManager_LevelDB_SetBlkFileLocation', argument 2 of type 'string'

So, it looks like the string is of the wrong type.  None of the characters are non-ASCII, so it shouldn't be that hard to convert.

[etotheipi]

My bad.  Sorry for the scavenger hunt!  Put in LOGERROR calls, which will guarantee it gets written out to the log file:

Code:

      LOGERROR('***BLKDIR VARIABLE:')
      LOGERROR('***Type:  %s' % str(type(blkdir)))
      LOGERROR('***Size:  %d' % len(blkdir))
      LOGERROR('***Value: %s' % str(blkdir))

Breaking them out like that will also make sure the type() is printed even if len() or str() throws an error.

It didn't crash until bitcoind had downloaded the blockchain.

Code:

(INFO) ArmoryQt.py:1346 - Setting netmode: 1
(INFO) armoryengine.py:12324 - Setting online mode: True (wait=False)
(INFO) armoryengine.py:13249 - Go online requested
(INFO) armoryengine.py:12766 - Called __startLoadBlockchain()
(ERROR) armoryengine.py:12808 - ***BLKDIR VARIABLE:
(ERROR) armoryengine.py:12809 - ***Type:  <type 'unicode'>
(INFO) armoryengine.py:10353 - Connection initiated.  Start handshake
(ERROR) armoryengine.py:12810 - ***Size:  22
(ERROR) armoryengine.py:12811 - ***Value: F:\bitcoin_data\blocks
(ERROR) armoryengine.py:13274 - Error processing BDM input
(ERROR) armoryengine.py:13275 - Received inputTuple: GoOnlineRequested [13, 40873630, False]
(ERROR) armoryengine.py:13276 - Error processing ID (40873630)
(ERROR) armoryengine.py:13277 - ERROR:
Traceback (most recent call last):
  File "G:\armory_testing\ArmoryStandalone_0.89.99.2_win_all\ArmoryStandalone\library.zip\armoryengine.py", line 13260, in run
  File "G:\armory_testing\ArmoryStandalone_0.89.99.2_win_all\ArmoryStandalone\library.zip\armoryengine.py", line 12812, in __startLoadBlockchain
  File "CppBlockUtils.pyc", line 1230, in SetBlkFileLocation
TypeError: in method 'BlockDataManager_LevelDB_SetBlkFileLocation', argument 2 of type 'string'

So, it looks like the string is of the wrong type.  None of the characters are non-ASCII, so it shouldn't be that hard to convert.

In fact, there was no error when it printed the value of "str(blkdir)" which means it was safely cast to a string.  I don't know why you ended up with a unicode value, but I do know that the simply putting in that cast is not a good general solution.  Though it should work for you right now...

For now just cast the "unicode" object to a string.  I'll put that in the next version, since it is strictly an improvement over the current code (which will fail unless it's a string, anyway).  But I need to put more effort into the unicode handling there... probably one of need to call "toPreferred()" to get it to work with arbitrary path vars. 

[picobit]

I can report that the new version compiled just fine on OS X.  I had previously used HomeBrew to install all dependencies of the old Armory client, and did not have to install anything new to compile the new version.  The first tests on testnet look good.  I will test offline wallets (on testnet) soon.

As for packaging this as an .app, that is going to be harder, but I will have a look at that too.  But not this week, and probably not next week.

[picobit]

OK, first two strange behaviors:

I transferred some testnet coins from a testnet faucet.  When the first block arrived, Armory popped up a window saying that Bitcoin-QT was not up to date.  It also printed this around 10 times in the terminal.

At the same time this happened, I was transferring testnet coins from another faucet.  The amount appeared in the total, but this time the transaction did not appear in the list of transactions.  Restarting Armory made it appear (at 0 confirmations).  Restart was blazingly fast!


EDIT:

-INFO  - 1379965779: (BlockUtils.cpp:3748) Finished blockchain scan in 8e-06 seconds

I somehow doubt this timing

[japerry]

Armory version 0.89.99.3-testing:

Windows:  Download the standalone executable bundle (click here)  
Just unpack and double-click on ArmoryQt.exe.  You may want to right-click and make a shortcut to it, in the same directory, then modify the properties->"Target:" to use non-standard "--datadir" and/or "--satoshi-datadir" paths (or any other flags you want, like "--testnet).

I'm trying to set up the new Armory and Bitcoin-QT to run completely off a flash drive. Oh!! BTW thanks for the new Armory! I actually can run it now! 

So what is the difference between the --datadir and --satoshi-datadir command line calls?

OK! I have it running on a TrueCrypt container on a removable drive! So... what command line parameter specifies the location of the config file? My goal is to make a portable version with the command line parameters fully defined. Of course I want to read any docs explaining the parameters.

[etotheipi]

OK! I have it running on a TrueCrypt container on a removable drive! So... what command line parameter specifies the location of the config file? My goal is to make a portable version with the command line parameters fully defined. Of course I want to read any docs explaining the parameters.

See the bottom of the "troubleshooting" page:  http://bitcoinarmory.com/download/troubleshooting/

Not sure how it ended up there.  It might've gotten reshuffled with the new website.

[japerry]

OK! I have it running on a TrueCrypt container on a removable drive! So... what command line parameter specifies the location of the config file? My goal is to make a portable version with the command line parameters fully defined. Of course I want to read any docs explaining the parameters.

See the bottom of the "troubleshooting" page:  http://bitcoinarmory.com/download/troubleshooting/

Not sure how it ended up there.  It might've gotten reshuffled with the new website.

Wow! Thanks! That is exactly what I was looking for! I really appreciate your fast response time and I thank you for a wonderful program.

[Mooshire]

I check this subforum every once in a while, great work! I may finally be able to soon go back to using Armory!

[etotheipi]

I check this subforum every once in a while, great work! I may finally be able to soon go back to using Armory!

Well, please try it!  I need more people trying it!

[Mooshire]

I check this subforum every once in a while, great work! I may finally be able to soon go back to using Armory!

Well, please try it!  I need more people trying it!

I'll do that now. Specs before I try:
Windows 8
8 GB RAM
AMD 8120 8-core
NVIDIA GT 640

[etotheipi]

So far, I've seen a pattern that 32-bit Windows versions don't seem to work.  But 64-bit OSes do work.  And this strange unicode issue from TierNolan.

I tested it on my fiancée's computer:  Win7-64bit w/ 4 GB of RAM.  About 5 years old.  It took forever to download the blockchain (probably 24 hours).  And it took about 2-3 hours to build the the DB.  But once it finished scanning (I don't know the timing), it only takes 1.5 minutes to start up.  And most of that is waiting for bitcoind to initialize.  And somehow uses even less RAM than the other systems:  215 MB.  Not too shabby.  

Also had someone report success on a system with 2 GB of RAM!  Slow of course.  There is no doubt that Armory is still a pretty heavy program...

Given the issues with Windows 32-bit that I've seen, I'm going to try to rebuild everything on a native 32-bit VM.  Unfortunately, I tried installing MSVS 2012 Express on WindowsXP-32bit, but I got all sorts of crazy errors.  If anyone else has WinXP-32bit and feels like donating some time, please try downloading and installing MSVS 2012 Express for Windows Desktop.  I ask because, I assume it's possible that there is something wrong with my WinXP system, but I also wouldn't be surprised if Windows XP is just too old at this point for such a new piece of software.

Instead, I'm going to install Windows7-32bit and repeat the build process there.  If it works there, it obviously has a better chance of working on any Win7-32bit and Win8-32bit.  Fat chance for Windows XP, but maybe we'll get lucky.

Known Issues:

• Doesn't seem to work yet on 32-bit OS.  Haven't tried any *nix systems, yet.  But haven't heard any success stories from 32-bit Windows users.
• Unicode everything  Ugh.  Will work on that after it works for US users.  The next version will get some unicode-TLC (it will mostly be a bug-fix and polishing release, with nothing major except for message signing)
• About 10% of the time, I start the app and I get rapid disconnects/reconnects.  The log reports a huge string of "Marking Orphan Chain" messages (which indicates the headers are not being processed correctly).  I close it, and restart and it works fine.
• Super-slow and no feedback when it's updating a couple days' history after being shut down for a while.  The feedback is technically there, telling it to update the GUI every 25 MB... but the update is usually less than 25 MB, and even if it is >25 MB, that can take a couple minutes.  Not only do I need to improve the GUI-update cycle, I need to figure out why the initial build is 10x faster than updating on restart (even though they should be using the same code to do it)
• General speed:  Obviously some optimizations missed.  One of the reasons I picked levelDB was some testing early on where I found that doing a full blockchain scan was 30% faster than my super-optimized blockchain scanner.  Now it's 3x slower.  I have a good idea what it is -- I'll dedicate a day to tinkering and benchmarking, after the release

[SimonBelmond]

OK thanks for all the hard work Alan (and contributors and team):

Tested "Armory version 0.89.99.3-testing"
On: Windows 7 64 bit, 4 gb ram,
I had it scan the BC over night. This morning i got the message that bitcoin qt appears to be offline or something (sorry missed the screenshot). Armory was online though. I closed armory and reopened. I think it took even less than your 19 sec to download 1 block and go online straight away. very nice. Not a lot of testing apart from that. Will do more and also report on ram usage. First sight looks good. 

[etotheipi]

OK thanks for all the hard work Alan (and contributors and team):

Tested "Armory version 0.89.99.3-testing"
On: Windows 7 64 bit, 4 gb ram,
I had it scan the BC over night. This morning i got the message that bitcoin qt appears to be offline or something (sorry missed the screenshot). Armory was online though. I closed armory and reopened. I think it took even less than your 19 sec to download 1 block and go online straight away. very nice. Not a lot of testing apart from that. Will do more and also report on ram usage. First sight looks good. 

There appears to be an issue right after it finishes the full DB build.  But it seems to be solved by a restart.  So far, I've seen a few issues with Armory getting into a bad state like that, but it always resolves itself.  So far, I haven't had to "--rebuild" the DBs once. That's a good sign (though --rescanning sometimes seems necessary).

[justusranvier]

Leveldb/snappy is still broken:

Code:

(CRITICAL) armoryengine.py:1019 - C++ block utilities not available.
(CRITICAL) armoryengine.py:1020 -    Make sure that you have the SWIG-compiled modules
(CRITICAL) armoryengine.py:1021 -    in the current directory (or added to the PATH)
(CRITICAL) armoryengine.py:1022 -    Specifically, you need:
(CRITICAL) armoryengine.py:1023 -        CppBlockUtils.py     and
(CRITICAL) armoryengine.py:1025 -        _CppBlockUtils.so
(ERROR) Traceback (most recent call last):
  File "/usr/share/armory/ArmoryQt.py", line 30, in <module>
    from armoryengine import *
  File "/usr/share/armory/armoryengine.py",                                                                                                 line 1015, in <module>
    import CppBlockUtils as Cpp
  File "/usr/share/armory/CppBlockUtils.py", line 26, in <module>
    _CppBlockUtils = swig_import_helper()
  File "/usr/share/armory/CppBlockUtils.py", line 22, in swig_import_helper
    _mod = imp.load_module('_CppBlockUtils', fp, pathname, description)
ImportError: /usr/share/armory/_CppBlockUtils.so: undefined symbol: _ZN6snappy21GetUncompressedLengthEPKcmPm

Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/share/armory/armoryengine.py", line 595, in logexcept_override
    sys.__excepthook__(type, value, tback) 
AttributeError: 'NoneType' object has no attribute '__excepthook__'

Original exception was:
Traceback (most recent call last):
  File "/usr/share/armory/ArmoryQt.py", line 30, in <module>
    from armoryengine import *
  File "/usr/share/armory/armoryengine.py", line 1015, in <module>
    import CppBlockUtils as Cpp
  File "/usr/share/armory/CppBlockUtils.py", line 26, in <module>
    _CppBlockUtils = swig_import_helper()                                                                                                
  File "/usr/share/armory/CppBlockUtils.py", line 22, in swig_import_helper                                                                                                                                                               
    _mod = imp.load_module('_CppBlockUtils', fp, pathname, description)                                                                                                                                                                   
ImportError: /usr/share/armory/_CppBlockUtils.so: undefined symbol: _ZN6snappy21GetUncompressedLengthEPKcmPm

The pull request I sent you on GitHub fixes the problem for me. With that patch applied, the time from starting the program until balances are displayed is about 28 seconds.