Armory - Discussion Thread

[etotheipi]

Got it!  The error was misreported making me think it was a problem with python-twisted, but it was a problem with a the zope.interface module which is used by python-twisted.   The "zope" module doesn't have an __init__.py file and py2exe can't detect it to put it into the exe bundle.  The solution was to make an empty __init__.py in the C:\Python27\Libs\site-packages\zope directory after installing zope.  That's it!  Will need to remember to put that in the build instructions...

248 MB RAM usage on Windows.  Solid.  I guess it's still high for a "regular" desktop app, but it's nothing out of this world anymore



Importing and sweeping appears to be working correctly.  Wallet restore -- it works, but you have to restart Armory, or do it in offline mode then go online.  There's also some important updates to the backup system that needs to put in before anyone goes and uses it.  So all that is in the works tonight and tomorrow, then I'll create a standalone executable (signed) that people can use for pre-testing and funds recovery.  Should be in the next 48 hours.

Andy (currently my first and only team member) is helping setup a new Windows installer (using NSIS) that will Do The Right Thing when it installs Armory -- no more mysterious errors that require uninstalling and reinstalling Armory.   It will also have more installation options.  And it will look slick

[halfawake]

I have a question about the upcoming RAM-reduction update.  Will there be any option for people to turn off the more hard-drive intensive version of Armory and keep the RAM-intensive version for people who have a lot of RAM?  I don't know if this would be a major pain to code or not, but it'd be a nice feature to have.

I guess, for now you could just keep using the old version.  But going forward, it will be kind of tough to do what you just suggested.  However, there will be lighter-weight modes implemented that will use less RAM and HDD, though, it might have slightly watered-down functionality (not being able view arbitrary transactions or fees on some incoming tx).  I'll keep it in mind, though.  I'm sure there's lot of server applications where the user has more than enough RAM, and would prefer the performance.

At the moment, while there's only one version of Armory that everyone uses (no super-node, regular-node, lite-node split), RAM is much more scarce than HDD space.  But these kinds of things are great to keep in mind so I don't burn any code bridges if there's a chance they will be in high demand in the future (when I've got different modes).

It's probably a seriously non-trivial thing to do, code wise.  Mostly the reason I was curious is that this kind of update would effectively double the hard drive demands of bitcoin since you'd have the 11.9 GB of the blockchain that bitcoin-qt manages, and the (presumably also) 11.9 GB of space that Armory would require from what you mentioned earlier.  24 GB is kind of a lot of space just to be able to keep Armory running happily, even in the era of hard drives with a lot of space.

[etotheipi]

It's probably a seriously non-trivial thing to do, code wise.  Mostly the reason I was curious is that this kind of update would effectively double the hard drive demands of bitcoin since you'd have the 11.9 GB of the blockchain that bitcoin-qt manages, and the (presumably also) 11.9 GB of space that Armory would require from what you mentioned earlier.  24 GB is kind of a lot of space just to be able to keep Armory running happily, even in the era of hard drives with a lot of space.

2.2x the blockchain size is a lot of space especially when the blockchain is 50 GB.  But if you're going to go put 128 GB of RAM in your server, it'll be a lot cheaper to just buy a single 2 TB HDD for $80 which will sustain you for another year or two even at absolute maximum blockchain growth.

Also, the alternative is that you only hold 1x the blockchain.  Obviously, it's less than 2.2x, but it's the same order of magnitude, which means if 2.2x is enough for you to be concerned, 1x is still a lot. 

A workaround that doesn't involve any work for me:  make yourself a RAM disk and put all the data on there.  It should be fast as hell.  But of course, everything goes poof on a power-cycle. 

As I said, I'll think about how it might be done.  Especially if DB performance becomes a real issue.  But I might wait until that happens before I put too much effort into the RAM solution.

[halfawake]

It's probably a seriously non-trivial thing to do, code wise.  Mostly the reason I was curious is that this kind of update would effectively double the hard drive demands of bitcoin since you'd have the 11.9 GB of the blockchain that bitcoin-qt manages, and the (presumably also) 11.9 GB of space that Armory would require from what you mentioned earlier.  24 GB is kind of a lot of space just to be able to keep Armory running happily, even in the era of hard drives with a lot of space.

2.2x the blockchain size is a lot of space especially when the blockchain is 50 GB.  But if you're going to go put 128 GB of RAM in your server, it'll be a lot cheaper to just buy a single 2 TB HDD for $80 which will sustain you for another year or two even at absolute maximum blockchain growth.

Also, the alternative is that you only hold 1x the blockchain.  Obviously, it's less than 2.2x, but it's the same order of magnitude, which means if 2.2x is enough for you to be concerned, 1x is still a lot. 

A workaround that doesn't involve any work for me:  make yourself a RAM disk and put all the data on there.  It should be fast as hell.  But of course, everything goes poof on a power-cycle. 

As I said, I'll think about how it might be done.  Especially if DB performance becomes a real issue.  But I might wait until that happens before I put too much effort into the RAM solution.

At the rate the bitcoin blockchain grows in size, 2.2x the growth worries me a lot more than 1x growth.  But you're right about cost, it's a lot cheaper to buy a really big hard drive than it is to buy a ton of RAM.  That said, I do think the hard drive solution is the right way to go since hard drive space is generally much more plentiful than RAM. 

[Raize]

Andy (currently my first and only team member) is helping setup a new Windows installer (using NSIS) that will Do The Right Thing when it installs Armory -- no more mysterious errors that require uninstalling and reinstalling Armory.   It will also have more installation options.  And it will look slick

Like an MSI package for silent installation? That sounds great!

[chrisrico]

At the rate the bitcoin blockchain grows in size, 2.2x the growth worries me a lot more than 1x growth.

It really shouldn't. Doubling the storage requirements is mathematically very little. If you can afford to store 10 GB, you can almost assuredly afford to store 20 GB. If you can afford to store 1 TB you can almost assuredly afford to store 2 TB.

[cp1]

At the rate the bitcoin blockchain grows in size, 2.2x the growth worries me a lot more than 1x growth.

It really shouldn't. Doubling the storage requirements is mathematically very little. If you can afford to store 10 GB, you can almost assuredly afford to store 20 GB. If you can afford to store 1 TB you can almost assuredly afford to store 2 TB.

Yes, while doubling RAM isn't so simple due to limited RAM slots on most personal computers, meaning you have to usually throw some sticks out to add more.  Most personal computers though have room to add an extra HDD.  And I don't think RAM prices scale linearly like HDD storage.

[Roy Badami]

At the rate the bitcoin blockchain grows in size, 2.2x the growth worries me a lot more than 1x growth.

It really shouldn't. Doubling the storage requirements is mathematically very little. If you can afford to store 10 GB, you can almost assuredly afford to store 20 GB. If you can afford to store 1 TB you can almost assuredly afford to store 2 TB.

It really isn't about what you can afford, it's about what you can do.  If you're on a laptop with an SSD, you have 128GTB, maybe 256GB if it's a new one.

Sure that's enough for now.... but what's this 2TB you're going to plug in to your laptop and how is it going to work?

roy

[etotheipi]

At the rate the bitcoin blockchain grows in size, 2.2x the growth worries me a lot more than 1x growth.

It really shouldn't. Doubling the storage requirements is mathematically very little. If you can afford to store 10 GB, you can almost assuredly afford to store 20 GB. If you can afford to store 1 TB you can almost assuredly afford to store 2 TB.

It really isn't about what you can afford, it's about what you can do.  If you're on a laptop with an SSD, you have 128GTB, maybe 256GB if it's a new one.

Sure that's enough for now.... but what's this 2TB you're going to plug in to your laptop and how is it going to work?

roy

The point is that increasing disk space on an arbitrary system is usually much easier then increasing RAM.  Most consumer motherboards currently max out at 32 GB, and the blockchain might be that big 12 months from now.  You can get yourself a crazy $10,000 server that is expandable up to 192 GB of RAM... but even many older consumer PCs now can hook up a HDD through (e)SATA and get 2 TB for less than $100.

[Roy Badami]

At the rate the bitcoin blockchain grows in size, 2.2x the growth worries me a lot more than 1x growth.

It really shouldn't. Doubling the storage requirements is mathematically very little. If you can afford to store 10 GB, you can almost assuredly afford to store 20 GB. If you can afford to store 1 TB you can almost assuredly afford to store 2 TB.

It really isn't about what you can afford, it's about what you can do.  If you're on a laptop with an SSD, you have 128GTB, maybe 256GB if it's a new one.

Sure that's enough for now.... but what's this 2TB you're going to plug in to your laptop and how is it going to work?

roy

The point is that increasing disk space on an arbitrary system is usually much easier then increasing RAM.  Most consumer motherboards currently max out at 32 GB, and the blockchain might be that big 12 months from now.  You can get yourself a crazy $10,000 server that is expandable up to 192 GB of RAM... but even many older consumer PCs now can hook up a HDD through (e)SATA and get 2 TB for less than $100.

Right, but on a laptop increasing either is often hard.

Sure, the RAM reduce work you're doing is appreciated - of course I'd rather have the blockchain on disk than in RAM.  But not everyone has 1TB of anything.  Fortunately 128GB will, I hope, be enough in the short term.

roy

[Roy Badami]

I'd also point out that Armory 0.87 still seems to work on my little old Macbook Air (OS X 10.6; 4GB RAM; 128B SSD).  Not sure it will much longer, and tbh I'm surprised it does now.  Kinda cool that it does though

roy

[etotheipi]

I'd also point out that Armory 0.87 still seems to work on my little old Macbook Air (OS X 10.6; 4GB RAM; 128B SSD).  Not sure it will much longer, and tbh I'm surprised it does now.  Kinda cool that it does though

I believe it still works on Linux and OSX systems with 4GB if you are running nothing else.  Windows has significantly higher resource usage for some reason.  It's most likely that Armory uses the same amount of RAM, but Windows likes to eat up 1 GB of RAM just to use the OS, whereas Linux uses like 100-300 MB. 

Meh, it won't matter shortly

[Roy Badami]

Is that true?  Pretty sure it used to be the case that Gnome was way more of a resource hog than Windows.

Maybe Linux desktops have got better (and Windows worse) in this respect...

[128bpm]

Right, but on a laptop increasing either is often hard.

Sure, the RAM reduce work you're doing is appreciated - of course I'd rather have the blockchain on disk than in RAM.  But not everyone has 1TB of anything.  Fortunately 128GB will, I hope, be enough in the short term.

roy

When the blockchain starts getting into the hundreds of GBs, I suspect that most normal bitcoin users will be using Armory in "lite mode" that doesn't require the blockchain locally at all. 

Fortunately, I think that date is long enough into the future that Alan and co. should have this "lite mode" working well by then.

[chrisrico]

It really isn't about what you can afford, it's about what you can do.  If you're on a laptop with an SSD, you have 128GTB, maybe 256GB if it's a new one.

Sure that's enough for now.... but what's this 2TB you're going to plug in to your laptop and how is it going to work?

You can't really compare the size of the blockchain in many years to hard drive sizes today...

Anyway, with Armory storing its data on disk, you can run Bitcoin on another computer, so you're back to just 1x block data on your laptop.

[goatpig]

I've just successfully compiled my own take at LevelDB on MSVC. I need to bench it, test the x64 build and write a guide so that you can upgrade the project with newer versions as they come.

It's late now and I won't be around on Saturday nor for the best part of Sunday. Expect delivery at best on Sunday night.

[Kluge]

Right, but on a laptop increasing either is often hard.

Sure, the RAM reduce work you're doing is appreciated - of course I'd rather have the blockchain on disk than in RAM.  But not everyone has 1TB of anything.  Fortunately 128GB will, I hope, be enough in the short term.

roy

I don't think that argument holds up with modern laptops. Most of them have "hotswap" ports, where you just slide a HDD in. You can alternately simply connect an external HDD either via USB, or maybe even wirelessly via BT (not sure if those exist, yet).

[Roy Badami]

Right, but on a laptop increasing either is often hard.

Sure, the RAM reduce work you're doing is appreciated - of course I'd rather have the blockchain on disk than in RAM.  But not everyone has 1TB of anything.  Fortunately 128GB will, I hope, be enough in the short term.

roy

I don't think that argument holds up with modern laptops. Most of them have "hotswap" ports, where you just slide a HDD in. You can alternately simply connect an external HDD either via USB, or maybe even wirelessly via BT (not sure if those exist, yet).

Oh, sure I can use an external drive.... but having to lug one around seriously impacts the portability of my computer.  Anyway, none of this is intended to be a criticism of Armory - I'm very appreciative of the work that etotheipi has done and continues to do.  I'm just pushing back against this idea that resources are somehow infinite because you can always upgrade your machine.  That's clearly not true (and if it were there'd be no need for the ram reduction etotheipi is working on).

roy

EDIT: Fortunately I don't think anyone is seriously saying the cost of reducing the RAM usage below 4GB will be to increase the disk usage above 1TB - so this discussion is somewhat academic

[dserrano5]

When the blockchain starts getting into the hundreds of GBs, I suspect that most normal bitcoin users will be using Armory in "lite mode" that doesn't require the blockchain locally at all.

I just bought a 1 Tb disk for 0.5 btc. When the blockchain gets into the hundreds of Gb, we'll be able to buy 1 Pb disks for the same amount of money and I will happily get a disk to store the blockchain and support the network with a full node.

[KS]

Anything new on the message-signing front? Is it still un-encrypt/encrypt again? I thought jackjack was working on sth but couldn't find an update (or missed it).

@Alan: Congrats! Now that you're incorporated and funded, I can't wait for the LTC version of Armory!