Armory - Discussion Thread

[goatpig]

@ goatpig

I am trying to follow your solution on Windows 7, 32-bit.  Everything has been smooth except for ATL.  I installed the 7.1.0 WDK, but it still can't find atlbase.h.  I have searched the entire harddrive looking for it, but can't find it.  I reinstalled the WDK and rebooted and it's nowhere to be found.  I did some googling and found posts telling me to install the 7.1.0 WDK... I wonder if it has to do with 32-bit OS?  I can't imagine the header file would, but I'm grasping at straws here...  any advice?

C:\WinDDK\7600.16385.1\inc\atl71 -> this is my include folder for all header files
C:\WinDDK\7600.16385.1\lib\ATL\i386 -> this is where the libs are. I picked i386 cause leveldbwin only support x86. However x64 binaries and libs are available if you wanna try a x64 compile.


I'm curious about your install folder, namely its build version (mine is 7600.16385.1 as you can see)

Edit: I'm redownloading the wdk iso to check the installation process. Did you perform a custom or full install? There's only one option needed to install ATL binaries and headers

[1714127809]

1714127809

[1714127809]

1714127809

[etotheipi]

C:\WinDDK\7600.16385.1\inc\atl71 -> this is my include folder for all header files
C:\WinDDK\7600.16385.1\lib\ATL\i386 -> this is where the libs are. I picked i386 cause leveldbwin only support x86. However x64 binaries and libs are available if you wanna try a x64 compile.


I'm curious about your install folder, namely its build version (mine is 7600.16385.1 as you can see)

Ugh!  Why is Windows search so terrible?!  I did a full "search" of the C-drive and it came up with nothing.  But I just followed the path you mentioned and I found it.  Alright, I'll manually add that to the "Additional Include Directories" and see where that gets me.

[goatpig]

Ugh!  Why is Windows search so terrible?!  I did a full "search" of the C-drive and it came up with nothing.  But I just followed the path you mentioned and I found it.  Alright, I'll manually add that to the "Additional Include Directories" and see where that gets me.

Windows Search is based on preindexation performed by a back-end service. New folders may not be parsed right away in the temporary db.

You have to indicate both header and lib folder to msvc unless you install a pay version. You can register these folders globally in msvc options rather than just link them to each of your projects. Not sure how it's done is vc11 express. Ima look it up

[goatpig]

Ok my bad the global include folder registering through msvc GUI seems to be limited to pay versions. Have to do it through the registry.

[etotheipi]

You have to indicate both header and lib folder to msvc unless you install a pay version. You can register these folders globally in msvc options rather than just link them to each of your projects. Not sure how it's done is vc11 express. Ima look it up

It actually complied the leveldb.lib without linking the lib folder to the project.  I only included the header and it was happy.

[goatpig]

You have to indicate both header and lib folder to msvc unless you install a pay version. You can register these folders globally in msvc options rather than just link them to each of your projects. Not sure how it's done is vc11 express. Ima look it up

It actually complied the leveldb.lib without linking the lib folder to the project.  I only included the header and it was happy.

That's odd, i had to link atl.lib to it. Regardless, as long as it's built, the rest is irrelevant.

[etotheipi]

You have to indicate both header and lib folder to msvc unless you install a pay version. You can register these folders globally in msvc options rather than just link them to each of your projects. Not sure how it's done is vc11 express. Ima look it up

It actually complied the leveldb.lib without linking the lib folder to the project.  I only included the header and it was happy.

That's odd, i had to link atl.lib to it. Regardless, as long as it's built, the rest is irrelevant.

I selected "Use of ATL -- Yes".  Maybe it figured it out based on that?

[goatpig]

I selected "Use of ATL -- Yes".  Maybe it figured it out based on that?

Possibly. I never do that, I just link the include folder.

[etotheipi]

I'm curious to know if there is any push to improve the Armory interface? For a user like me it works just fine, but someone who is newer to Bitcoin, I could see how it could be very uninviting.

This is in our plans.  One of the investors has a software company and is planning to contribute some manpower to redesigning it.  Right now it uses all the vanilla PyQt buttons and widgets, etc.  But it should be easy to subclass everything and make prettier versions of them (like buttons with images instead of raw text), which will make it look better.  And also we will be redoing some of the dialogs and interfaces.  It's just not high priority for me or my new developers, but it is part of our plan going forward as we try to implement a lite-version that is better tailored to new users.

[super3]

I'm curious to know if there is any push to improve the Armory interface? For a user like me it works just fine, but someone who is newer to Bitcoin, I could see how it could be very uninviting.

This is in our plans.  One of the investors has a software company and is planning to contribute some manpower to redesigning it.  Right now it uses all the vanilla PyQt buttons and widgets, etc.  But it should be easy to subclass everything and make prettier versions of them (like buttons with images instead of raw text), which will make it look better.  And also we will be redoing some of the dialogs and interfaces.  It's just not high priority for me or my new developers, but it is part of our plan going forward as we try to implement a lite-version that is better tailored to new users.

Ok. Any timeline of when that might get done? Working on some Armory video tutorials with someone to reduce the technical hurdle for people who want to use Armory. These will of course have to be updated as the interface changes. If I can get a very rough idea then I can kinda plan ahead.

[etotheipi]

I'm curious to know if there is any push to improve the Armory interface? For a user like me it works just fine, but someone who is newer to Bitcoin, I could see how it could be very uninviting.

This is in our plans.  One of the investors has a software company and is planning to contribute some manpower to redesigning it.  Right now it uses all the vanilla PyQt buttons and widgets, etc.  But it should be easy to subclass everything and make prettier versions of them (like buttons with images instead of raw text), which will make it look better.  And also we will be redoing some of the dialogs and interfaces.  It's just not high priority for me or my new developers, but it is part of our plan going forward as we try to implement a lite-version that is better tailored to new users.

Ok. Any timeline of when that might get done? Working on some Armory video tutorials with someone to reduce the technical hurdle for people who want to use Armory. These will of course have to be updated as the interface changes. If I can get a very rough idea then I can kinda plan ahead.

I would guess it won't be in the next two months.  We have our plate pretty full, and we'd prefer to work out the higher priority stuff (like RAM reduction, new wallets, HW wallets) without dealing with the interface upgrades.  Well, and training our new employees

[etotheipi]

Updates:

Linux:  The app technically works, but it's got some serious usability bugs.  Importing, sweeping and restoring wallets causes the app to freeze until it finishes.  It also frequently freezes when new blocks come in or it checks for a new version.  Quite a few other quirks, too.  But all these issues are related to usability -- the core functionality seems to be in excellent shape!  The app successfully saves its entire state between loads, and only takes about 10-30 seconds to startup and pickup where it left off (longer if you haven't loaded it in a while and Bitcoin-Qt/bitcoind needs to catch up). 

Also, building and scanning (on all OS) is much slower than I was expecting.  Again, it's only a usability problem, and I am aware of multiple places that I missed optimizations that would improve all this.  I plan to dig into those when I get out a testing version. 

Windows:  The good news is that I got everything compiling and working in Windows using native python (thanks goatpig!).  I haven't gotten very far into Windows testing, but so far it appears to work the same as the Linux version though even slower (probably related to the ancient version of LevelDB that i hope goatpig will help me figure out how to upgrade for another 1 BTC ).

The bad news is that the process of creating a .exe file is failing, and may take a bit of work to fix.  py2exe has given me quite a bit of trouble in the past, and when I did get it working I had no idea what I did.  Therefore, it technically works on Windows, but only if you install python and like eight different python packages.  When I do finally get py2exe working, I can put out a pre-testing-testing version that should let people get access to their funds but not use it for regular use (it's going to be very rough around the edges). 

For those inclined to compile on Windows, I can tell you that part of the process has been restructuring everything.  All the old MSVS 2005/2008 projects have been destroyed, and rebuilt with MSVS 2012 Express.  This makes it free to compile (if you register MSVS w/ Microsoft), but it's still going to take a bit of tinkering.  If someone wants to try it, I can post a list of packages that need to be installed.

OSX:  If anyone wants to help with this, please let me know.  The new LevelDB stuff needs to be compiled & tested on the OSX side, and I have no idea how much of a PITA it is.  Perhaps Red Emerald is in good shape to try this out. 

[redphlegm]

Updates:

Linux:  The app technically works, but it's got some serious usability bugs.  Importing, sweeping and restoring wallets causes the app to freeze until it finishes.  It also frequently freezes when new blocks come in or it checks for a new version.  Quite a few other quirks, too.  But all these issues are related to usability -- the core functionality seems to be in excellent shape!  The app successfully saves its entire state between loads, and only takes about 10-30 seconds to startup and pickup where it left off (longer if you haven't loaded it in a while and Bitcoin-Qt/bitcoind needs to catch up). 

Also, building and scanning (on all OS) is much slower than I was expecting.  Again, it's only a usability problem, and I am aware of multiple places that I missed optimizations that would improve all this.  I plan to dig into those when I get out a testing version. 

Windows:  The good news is that I got everything compiling and working in Windows using native python (thanks goatpig!).  I haven't gotten very far into Windows testing, but so far it appears to work the same as the Linux version though even slower (probably related to the ancient version of LevelDB that i hope goatpig will help me figure out how to upgrade for another 1 BTC ).

The bad news is that the process of creating a .exe file is failing, and may take a bit of work to fix.  py2exe has given me quite a bit of trouble in the past, and when I did get it working I had no idea what I did.  Therefore, it technically works on Windows, but only if you install python and like eight different python packages.  When I do finally get py2exe working, I can put out a pre-testing-testing version that should let people get access to their funds but not use it for regular use (it's going to be very rough around the edges). 

For those inclined to compile on Windows, I can tell you that part of the process has been restructuring everything.  All the old MSVS 2005/2008 projects have been destroyed, and rebuilt with MSVS 2012 Express.  This makes it free to compile (if you register MSVS w/ Microsoft), but it's still going to take a bit of tinkering.  If someone wants to try it, I can post a list of packages that need to be installed.

OSX:  If anyone wants to help with this, please let me know.  The new LevelDB stuff needs to be compiled & tested on the OSX side, and I have no idea how much of a PITA it is.  Perhaps Red Emerald is in good shape to try this out. 

Great! Thanks for the update. Waiting for this with much anticipation (OSX and Windows).

[etotheipi]

Updates:

Linux:  The app technically works, but it's got some serious usability bugs.  Importing, sweeping and restoring wallets causes the app to freeze until it finishes.  It also frequently freezes when new blocks come in or it checks for a new version.  Quite a few other quirks, too.  But all these issues are related to usability -- the core functionality seems to be in excellent shape!  The app successfully saves its entire state between loads, and only takes about 10-30 seconds to startup and pickup where it left off (longer if you haven't loaded it in a while and Bitcoin-Qt/bitcoind needs to catch up). 

Also, building and scanning (on all OS) is much slower than I was expecting.  Again, it's only a usability problem, and I am aware of multiple places that I missed optimizations that would improve all this.  I plan to dig into those when I get out a testing version. 

Windows:  The good news is that I got everything compiling and working in Windows using native python (thanks goatpig!).  I haven't gotten very far into Windows testing, but so far it appears to work the same as the Linux version though even slower (probably related to the ancient version of LevelDB that i hope goatpig will help me figure out how to upgrade for another 1 BTC ).

The bad news is that the process of creating a .exe file is failing, and may take a bit of work to fix.  py2exe has given me quite a bit of trouble in the past, and when I did get it working I had no idea what I did.  Therefore, it technically works on Windows, but only if you install python and like eight different python packages.  When I do finally get py2exe working, I can put out a pre-testing-testing version that should let people get access to their funds but not use it for regular use (it's going to be very rough around the edges). 

For those inclined to compile on Windows, I can tell you that part of the process has been restructuring everything.  All the old MSVS 2005/2008 projects have been destroyed, and rebuilt with MSVS 2012 Express.  This makes it free to compile (if you register MSVS w/ Microsoft), but it's still going to take a bit of tinkering.  If someone wants to try it, I can post a list of packages that need to be installed.

OSX:  If anyone wants to help with this, please let me know.  The new LevelDB stuff needs to be compiled & tested on the OSX side, and I have no idea how much of a PITA it is.  Perhaps Red Emerald is in good shape to try this out. 

Great! Thanks for the update. Waiting for this with much anticipation (OSX and Windows).

I forgot to mention that RAM usage on Windows looks to be about the same as Linux.  600-700 MB while the DB is building, then about 300 MB under normal operation.  In exchanges for the improved load time and RAM usage is 14 GB of disk space used for databases (which will be reduced in the future, but this was the easiest implementation for now).  So it's only a tad more than Bitcoin-Qt itself uses, which is about 11-12 GB currently (i.e. the storage efficiency of all that data in Armory's LevelDB database is only about 10-20% overhead).

[halfawake]

Updates:

Linux:  The app technically works, but it's got some serious usability bugs.  Importing, sweeping and restoring wallets causes the app to freeze until it finishes.  It also frequently freezes when new blocks come in or it checks for a new version.  Quite a few other quirks, too.  But all these issues are related to usability -- the core functionality seems to be in excellent shape!  The app successfully saves its entire state between loads, and only takes about 10-30 seconds to startup and pickup where it left off (longer if you haven't loaded it in a while and Bitcoin-Qt/bitcoind needs to catch up). 

Also, building and scanning (on all OS) is much slower than I was expecting.  Again, it's only a usability problem, and I am aware of multiple places that I missed optimizations that would improve all this.  I plan to dig into those when I get out a testing version. 

Windows:  The good news is that I got everything compiling and working in Windows using native python (thanks goatpig!).  I haven't gotten very far into Windows testing, but so far it appears to work the same as the Linux version though even slower (probably related to the ancient version of LevelDB that i hope goatpig will help me figure out how to upgrade for another 1 BTC ).

The bad news is that the process of creating a .exe file is failing, and may take a bit of work to fix.  py2exe has given me quite a bit of trouble in the past, and when I did get it working I had no idea what I did.  Therefore, it technically works on Windows, but only if you install python and like eight different python packages.  When I do finally get py2exe working, I can put out a pre-testing-testing version that should let people get access to their funds but not use it for regular use (it's going to be very rough around the edges). 

For those inclined to compile on Windows, I can tell you that part of the process has been restructuring everything.  All the old MSVS 2005/2008 projects have been destroyed, and rebuilt with MSVS 2012 Express.  This makes it free to compile (if you register MSVS w/ Microsoft), but it's still going to take a bit of tinkering.  If someone wants to try it, I can post a list of packages that need to be installed.

OSX:  If anyone wants to help with this, please let me know.  The new LevelDB stuff needs to be compiled & tested on the OSX side, and I have no idea how much of a PITA it is.  Perhaps Red Emerald is in good shape to try this out. 

Great! Thanks for the update. Waiting for this with much anticipation (OSX and Windows).

I forgot to mention that RAM usage on Windows looks to be about the same as Linux.  600-700 MB while the DB is building, then about 300 MB under normal operation.  In exchanges for the improved load time and RAM usage is 14 GB of disk space used for databases (which will be reduced in the future, but this was the easiest implementation for now).  So it's only a tad more than Bitcoin-Qt itself uses, which is about 11-12 GB currently (i.e. the storage efficiency of all that data in Armory's LevelDB database is only about 10-20% overhead).

I have a question about the upcoming RAM-reduction update.  Will there be any option for people to turn off the more hard-drive intensive version of Armory and keep the RAM-intensive version for people who have a lot of RAM?  I don't know if this would be a major pain to code or not, but it'd be a nice feature to have.

[etotheipi]

I have a question about the upcoming RAM-reduction update.  Will there be any option for people to turn off the more hard-drive intensive version of Armory and keep the RAM-intensive version for people who have a lot of RAM?  I don't know if this would be a major pain to code or not, but it'd be a nice feature to have.

I guess, for now you could just keep using the old version.  But going forward, it will be kind of tough to do what you just suggested.  However, there will be lighter-weight modes implemented that will use less RAM and HDD, though, it might have slightly watered-down functionality (not being able view arbitrary transactions or fees on some incoming tx).  I'll keep it in mind, though.  I'm sure there's lot of server applications where the user has more than enough RAM, and would prefer the performance.

At the moment, while there's only one version of Armory that everyone uses (no super-node, regular-node, lite-node split), RAM is much more scarce than HDD space.  But these kinds of things are great to keep in mind so I don't burn any code bridges if there's a chance they will be in high demand in the future (when I've got different modes).

[goatpig]

I'm glad to hear the Windows build went through properly.

Regarding the speed issue and LevelDB: I suspect it's due to the implementation of atomic types and routines used in leveldbwin. That project is made for msvc 9 and 10, which don't support atomics and as such rely on full memory barriers and software mutexes, which are considerably slower than atomics. LevelDB is natively a C11 POSIX library.

I am going to try building the port folder on msvc11, with native atomics support. As I said before, I suspect compiling this part is all it'll take to get any version of LevelDB to build properly.

Regarding py2exe: I have no experience with Python, however Windows development is all about DLL compartmentalization, i.e. maintaining the armory core and the Python GUI part separately. I didn't get the impression that this is what SWIG does. I'd assume if there was a way to simply link the Python code to DLL binaries, it would considerably simplify the py2exe part, by limited its scope to the GUI code and an exporting header of sorts. Then again I may be totally out of my league and talking out of my a**.

[etotheipi]

I'm glad to hear the Windows build went through properly.

Regarding the speed issue and LevelDB: I suspect it's due to the implementation of atomic types and routines used in leveldbwin. That project is made for msvc 9 and 10, which don't support atomics and as such rely on full memory barriers and software mutexes, which are considerably slower than atomics. LevelDB is natively a C11 POSIX library.

I am going to try building the port folder on msvc11, with native atomics support. As I said before, I suspect compiling this part is all it'll take to get any version of LevelDB to build properly.

Regarding py2exe: I have no experience with Python, however Windows development is all about DLL compartmentalization, i.e. maintaining the armory core and the Python GUI part separately. I didn't get the impression that this is what SWIG does. I'd assume if there was a way to simply link the Python code to DLL binaries, it would considerably simplify the py2exe part, by limited its scope to the GUI code and an exporting header of sorts. Then again I may be totally out of my league and talking out of my a**.

The py2exe thing is not your problem.  Definitely mine.  And it's not related to the .dlls, I have all that under control.  It's that py2exe bundles up a working python project, and all its dependencies into a standalone directory with an .exe file.  I assume it embeds the python interpreter in there.  Technically the .exe works and links properly, but py2exe is not bundling some of the needed python libraries.  Armory successfully loads all the SWIG modules and all the related .dlls, but can't finish initializing because the py2exe never put the python-twisted libraries in the embedded environment and I get a python error that twisted is not found.

At the moment, I'm going to see if I can manually inject the twisted libraries into the output library.zip (which holds all the python dependencies).  If that works, it will give me an option for producing Windows .exe until I have time to fight py2exe.

[Raize]

The bad news is that the process of creating a .exe file is failing, and may take a bit of work to fix.  py2exe has given me quite a bit of trouble in the past, and when I did get it working I had no idea what I did.

Would cx_Freeze help here?

EDIT: Also, pyinstaller might be an option.

[etotheipi]

The bad news is that the process of creating a .exe file is failing, and may take a bit of work to fix.  py2exe has given me quite a bit of trouble in the past, and when I did get it working I had no idea what I did.

Would cx_Freeze help here?

EDIT: Also, pyinstaller might be an option.

cx_Freeze is an option.  But py2exe has been pretty darned reliable once I get it working.  Every Windows release I've made since Armory v0.5 has used py2exe and I'm happy with it.  And of course, it's already integrated into my build system.  If I'm desperate, I may investigate other freeze-app options, but I think I just need to figure out once and for all why py2exe is not behaving the same on all my build systems.