Bitcoin Forum
April 24, 2017, 09:27:37 PM *
News: If the forum does not load normally for you, please send me a traceroute.
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 [126] 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 ... 232 »
  Print  
Author Topic: Armory - Discussion Thread  (Read 496597 times)
payb.tc
Hero Member
*****
Offline Offline

Activity: 812



View Profile
September 02, 2013, 12:19:19 PM
 #2501

*before* i attempt this, just wondering if anyone has ever tried it yet:

armory within vmware

virtual machine set to '12gb RAM' but host pc only physically has 2gb


...of course in the end the experiment/discussion will become moot if newer versions of armory use less ram
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1493069257
Hero Member
*
Offline Offline

Posts: 1493069257

View Profile Personal Message (Offline)

Ignore
1493069257
Reply with quote  #2

1493069257
Report to moderator
Carlton Banks
Legendary
*
Offline Offline

Activity: 1610



View Profile
September 02, 2013, 02:38:49 PM
 #2502

*before* i attempt this, just wondering if anyone has ever tried it yet:

armory within vmware

virtual machine set to '12gb RAM' but host pc only physically has 2gb


...of course in the end the experiment/discussion will become moot if newer versions of armory use less ram


Don't do that. VMWare will become unstable with those settings whether you try to use Armory or not. Armory will run worse than if you ran in it in the host OS, if it even verifys a quarter of the recent block updates. Then repeat this mantra: " VMWare does not turn my PC into the computing equivalent of Doctor Who's more-capacious-on-the-inside space ship telephone booth "

Vires in numeris
bezzeb
Member
**
Offline Offline

Activity: 103



View Profile
September 05, 2013, 06:49:13 AM
 #2503

I'm fine with the storage requirements, especially if it means being able to watch any address easily. Storage is cheap and easy to add.

Yup.  And at least you get something useful out of all that extra storage space:  you can get the entire history for any address (including multisig TxOuts) nearly instantaneously.  This will make armoryd.py quite a bit more powerful than bitcoind, since it will ultimately implement everything bitcoind does plus all the arbitrary addr/script lookups.  It also means that you can import addresses and wallets without having to rescan.

And with that, it may make sense to implement a basic python shell into the expert interface, that allows users to issue commands/requests directly from a "debug" window.  For instance, you'd run "Server Version" GUI (not armoryd.py), and open the "Armory CLI" window, and type in "getAddressHistory('<addr>')", "importAddress('<addr>', '<wallet>')", "findAllMultisig('<addr>')", etc.  I could make the same interface available through RPC, but that will take considerably longer because of how security-sensitive that is.

This super-node version also enables a very specific functionality that is not otherwise possible:  encrypted watching-only wallets.  Without a full address index, and without rescanning the blockchain every load, there's no way to hide the addresses in your watching-only wallet.  My new wallet design (which was side-tracked by this RAM-reduction effort) actually has a way to encrypt the public keys in a WO wallet, but it's not very useful if your database is storing only the transactions related to your wallets.  There's things you could do to improve the situation, but it won't be perfect unless you do a rescan every time you unlock your wallet and hold everything in RAM.  Or save the whole tx history as this super-node version is doing.  

So if you want absolute security and privacy, you can run this "server version" which will use a lot of HDD space, but it will run smoothly once the DB is built and you can hide the public keys in your wallet.  A lot of people have requested this functionality, because of concerns that a stolen laptop could lead to worst things than losing money.  For instance: the thief may not get your $100k worth of Bitcoin since he only has your WO wallet, but he can see that you have $100k worth of Bitcoin and can probably find your address somewhere else on the computer.   Many of my more paranoid/rich users would like to avoid that.

Profiling Update: my initial DB build is up to 221,000 blocks after 1.5 hours, and using 295 MB RAM (15 MB more than at 180k blocks).  So it looks like it may take about 3-4 hours to rebuild the whole DB and about 300 MB RAM.  Those are some solid, usable numbers!

Not only that, but it's very likely that I can build the initial database while Bitcoin-Qt is downloading the initial transaction history, meaning that Armory's DB building can be "hidden" and the whole new-user-startup may not be any worse than running vanilla Bitcoin-Qt.  I will not complicate it with that parallelization right now, but those numbers are very encouraging for a future upgrade.

Okay, enough rambling, more coding Smiley ...

Your progress sounds damn cool!!!!!  Better honestly than i was expecting!!  Ive been fretting on other threads about this in the past, but the world urgently needs a solution that allows local blockchain browsing a-la blockchain.info.  So if you can also deliver an engine that efficiently does this without a horrible rescan every time, then Armory "server" will have a killer app that others will lust for!  :-)   This is totally above and beyond the existing core coolness from your current feature set!

Woo hoo, ill tip you again fat style when the day comes that i can do an arbitrary blockchain query without waiting ages for a chain scan, and without telling everyone who is spying on web traffic the addresses i'm interested in by visiting blockchain.info.  They have a great service but frankly it's dangerous.  A query there is enough to publicly link your real world identity to those addresses that you are interested in.  Tor has its limits in terms of performance and market penetration.

No reason my isp or random law agencies deserve this info, its enough that the spies of the world could filter and log my transactions if they wanted to.  They don't deserve to also know all the other addresses i might peek into.
SimonBelmond
Full Member
***
Offline Offline

Activity: 201



View Profile
September 05, 2013, 12:54:02 PM
 #2504

Armory - Fort Knox of Bitcoin
etotheipi
Legendary
*
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
September 06, 2013, 03:56:08 PM
 #2505

Just another update, because I know people are anxious, and I'm making substantial progress.  I'll probably also need people to help with some testing this weekend Smiley

Quote from: GoogleTest
...

[----------] 8 tests from BlockUtilsTest
[ RUN      ] BlockUtilsTest.HeadersOnly
[       OK ] BlockUtilsTest.HeadersOnly (256 ms)
[ RUN      ] BlockUtilsTest.HeadersOnly_Reorg
[       OK ] BlockUtilsTest.HeadersOnly_Reorg (263 ms)
[ RUN      ] BlockUtilsTest.Load5Blocks
[       OK ] BlockUtilsTest.Load5Blocks (886 ms)
[ RUN      ] BlockUtilsTest.Load4BlocksPlus1
[       OK ] BlockUtilsTest.Load4BlocksPlus1 (665 ms)
[ RUN      ] BlockUtilsTest.Load5Blocks_Plus2NoReorg
[       OK ] BlockUtilsTest.Load5Blocks_Plus2NoReorg (595 ms)
[ RUN      ] BlockUtilsTest.Load5Blocks_FullReorg
[       OK ] BlockUtilsTest.Load5Blocks_FullReorg (589 ms)
[ RUN      ] BlockUtilsTest.RestartDBAfterBuild
[       OK ] BlockUtilsTest.RestartDBAfterBuild (857 ms)
[ RUN      ] BlockUtilsTest.RestartDBAfterBuild_withReplay
[       OK ] BlockUtilsTest.RestartDBAfterBuild_withReplay (871 ms)
[----------] 8 tests from BlockUtilsTest (4982 ms total)

[----------] 2 tests from BlockUtilsWithWalletBalance
[ RUN      ] BlockUtilsWithWalletBalance.PreRegisterScrAddrs
[       OK ] BlockUtilsWithWalletBalance.PreRegisterScrAddrs (595 ms)
[ RUN      ] BlockUtilsWithWalletBalance.PostRegisterScrAddr
[       OK ] BlockUtilsWithWalletBalance.PostRegisterScrAddr (636 ms)
[----------] 2 tests from BlockUtilsWithWalletBalance (1231 ms total)

[----------] Global test environment tear-down
[==========] 129 tests from 11 test cases ran. (9800 ms total)
[  PASSED  ] 129 tests.

I don't have all the reliability features in there.  Meaning, this should probably work 98% of the time, but it really needs to be more like 99.999%.  Things like DB/Armory crashes mid-build or while on a block that is eventually orphaned, etc, are not well-tested.  But I gotta start somewhere...

P.S. - Did I mention how much I love GoogleTest?  it's made my life sooooo much easier...

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
chrisrico
Hero Member
*****
Offline Offline

Activity: 496


View Profile
September 06, 2013, 04:57:46 PM
 #2506

I'll probably also need people to help with some testing this weekend Smiley

I'd be happy to help test. Is there anything in particular you need, or do you want people to just start using the new version?
SimonBelmond
Full Member
***
Offline Offline

Activity: 201



View Profile
September 06, 2013, 05:05:34 PM
 #2507

I'll probably also need people to help with some testing this weekend Smiley

I'd be happy to help test. Is there anything in particular you need, or do you want people to just start using the new version?

Same here!
etotheipi
Legendary
*
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
September 06, 2013, 05:07:40 PM
 #2508

I'll probably also need people to help with some testing this weekend Smiley

I'd be happy to help test. Is there anything in particular you need, or do you want people to just start using the new version?

Well, it's not done yet.  But those last two tests were the most exciting, because they demonstrate bootstrapping the old wallet system with the database data.  i.e. I actually left most of the thoroughly-tested, thoroughly-complex wallet infrastructure in place, and simply seed it with data from the database and it works smoothly.  That means that most of the GUI will work without modification (except for changing some function names).

I'm going to start trying to run the GUI with the new DB stuff and see what's still missing or what needs to be tweaked.  Before I'm totally done with the full upgrade, I might ask people to run it on mainnet but with a different home dir, to let me know how the initial DB build goes.  I've had mixed results with it -- the last test I did took 6 hours to build, but I've had a lot that took 3 hours.  I could use some extra data points on that one (by the way, I have one more optimization to put in place that should cut down that time even more, but at the moment I consider it "usable").

I'll let you know when I have something.  




Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
etotheipi
Legendary
*
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
September 06, 2013, 05:17:32 PM
 #2509

P.S. - ACK, I totally forgot something!  This only works in Linux!  I've taken some small stabs at getting it to work in Windows, but leveldb is a complete b**** in Windows.  Luckily, the core devs have done a lot of LevelDB stuff in Windows already, which I/we may be able to cannibalize, but there's no question it's going to be painful.

So when I talk about "testing this weekend", I'm most definitely talking about Linux testing.  If you're not on Linux, I won't have anything for you yet.

I will offer a 2 BTC bounty to anyone who can get this new stuff integrated and compiling in MSVS 2008 (it must be 2008, for now).  If it's too difficult, it may be possible to do it mingw, though Armory project itself would have to be ported to mingw and I'm not too keen on that.  But at the moment, any solution is a sufficient one.  It is probably possible to compile the leveldb & boost stuff into a .dll and access it from the existing MSVS project with a lot less pain (I recommend that because the core devs got it working using mingw but I really need to run the Armory C++ utilities in MSVS for debugging).

If you want to try this, you must follow the directions on the Building Armory from Source page, and get it working with the master branch.  You should be able to compile the C++ utilities in MSVS 2008 and run Armory off the compiled product.  Then switch to the ramreduceleveldb branch and... have fun.  If you can compile BlockUtils.cpp on the ramreduceleveldb branch in MSVS, then I can figure out the rest from there.

PLEASE document your process if you try it.  I've gotta understand how to integrate leveldb into my project.  Knowing that the core devs had gotten it working in Windows was sufficient for me to know I can get it working in Windows, but I forgot I still have to go through that process!

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
Inaba
Legendary
*
Offline Offline

Activity: 1260



View Profile WWW
September 06, 2013, 05:40:05 PM
 #2510

One thing you might consider doing with the large DB (aka server version?) is to allow the DB to be shared among several clients on a local network.  Ideally, you could have one version of the server version running on a host machine and several clients on remote machines utilizing the DB on the remote, removing the large space requirement for the lightweight clients but still offering the full functionality. 

I have a couple of environments where this would be really handy and I can imagine a few other uses for this type of distributed model, while not going full on cloud model.

If you're searching these lines for a point, you've probably missed it.  There was never anything there in the first place.
etotheipi
Legendary
*
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
September 06, 2013, 05:47:09 PM
 #2511

One thing you might consider doing with the large DB (aka server version?) is to allow the DB to be shared among several clients on a local network.  Ideally, you could have one version of the server version running on a host machine and several clients on remote machines utilizing the DB on the remote, removing the large space requirement for the lightweight clients but still offering the full functionality.  

I have a couple of environments where this would be really handy and I can imagine a few other uses for this type of distributed model, while not going full on cloud model.


That is actually the plan.  Or rather, when I talk about "super-node-lite-node split" that's what I'm talking about.  The lite version will be the same thing as the super version, but its calls to the DB engine will instead go over a socket to request it from another process or system.

It's a pretty big jump from where I currently am, but it's already part of my long-term plans Smiley

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
Sc@rF@c3
Jr. Member
*
Offline Offline

Activity: 33


View Profile
September 06, 2013, 06:30:07 PM
 #2512

Armory 0.88.1, 64bit, Win7, 16GB ram, i7 1st gen.
I don't know if this has been reported
I recently deleted my wallet and restored it.

Now Armory hangs at 98% - 15 seconds every day.
A close and restart gets it working again.

I don't have debug info.

I ran the installer and repaired the installation, fingers crossed, it has not shown any errors since.

If you found my posts helpful...
Tips: 13GiJUzw7oG9XnKCyt9n4zYGTsaQUAwpb9
payb.tc
Hero Member
*****
Offline Offline

Activity: 812



View Profile
September 07, 2013, 01:43:10 AM
 #2513

Then repeat this mantra: " VMWare does not turn my PC into the computing equivalent of Doctor Who's more-capacious-on-the-inside space ship telephone booth "

haven't tried it, but when you're just talking RAM, a virtual pc should be able to have 64 GB of ram, with the vm actually having it as a file somewhere, no?

i'm aware that it would be incredibly slow
Carlton Banks
Legendary
*
Offline Offline

Activity: 1610



View Profile
September 07, 2013, 12:56:03 PM
 #2514

Then repeat this mantra: " VMWare does not turn my PC into the computing equivalent of Doctor Who's more-capacious-on-the-inside space ship telephone booth "

haven't tried it, but when you're just talking RAM, a virtual pc should be able to have 64 GB of ram, with the vm actually having it as a file somewhere, no?

i'm aware that it would be incredibly slow


I think this depends on your virtualisation software, certainly any version 8 or less VMWare software doesn't have a hypervisor swap-file scheme like this, but perhaps the newer releases do.

Vires in numeris
chrisrico
Hero Member
*****
Offline Offline

Activity: 496


View Profile
September 07, 2013, 07:46:05 PM
 #2515

Then repeat this mantra: " VMWare does not turn my PC into the computing equivalent of Doctor Who's more-capacious-on-the-inside space ship telephone booth "

haven't tried it, but when you're just talking RAM, a virtual pc should be able to have 64 GB of ram, with the vm actually having it as a file somewhere, no?

i'm aware that it would be incredibly slow


When your computer needs more memory than it actually has, Windows uses a file on your hard drive as virtual memory (Unix uses a swap partition). As you state, it's incredibly slow. So no, using a VM doesn't benefit you at all.
etotheipi
Legendary
*
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
September 07, 2013, 07:52:43 PM
 #2516

Then repeat this mantra: " VMWare does not turn my PC into the computing equivalent of Doctor Who's more-capacious-on-the-inside space ship telephone booth "

haven't tried it, but when you're just talking RAM, a virtual pc should be able to have 64 GB of ram, with the vm actually having it as a file somewhere, no?

i'm aware that it would be incredibly slow


When your computer needs more memory than it actually has, Windows uses a file on your hard drive as virtual memory (Unix uses a swap partition). As you state, it's incredibly slow. So no, using a VM doesn't benefit you at all.

I'm not 100% convinced of that.  The issue is not slowness (well, it is an issue), but that these systems may be running out of RAM.  For instance, it just can't be run on 32-bit because of the lack of address space.  But if you virtualize a system that runs 64-bit OS with 16 GB of RAM (using 14 GB of disk), it may be possible to actually run it even though it takes 3 hours to sync. 

I'd be interested to see someone try it, though I would bet 2:1 that it still doesn't work at all.  But I can see why it might work.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
chrisrico
Hero Member
*****
Offline Offline

Activity: 496


View Profile
September 07, 2013, 09:41:42 PM
 #2517

I'm not 100% convinced of that.  The issue is not slowness (well, it is an issue), but that these systems may be running out of RAM.  For instance, it just can't be run on 32-bit because of the lack of address space.  But if you virtualize a system that runs 64-bit OS with 16 GB of RAM (using 14 GB of disk), it may be possible to actually run it even though it takes 3 hours to sync. 

I'd be interested to see someone try it, though I would bet 2:1 that it still doesn't work at all.  But I can see why it might work.

Can you run a 64 bit VM on 32 bit hardware? That doesn't seem right.

Anyway, the original question was whether setting up a VM would help alleviate the problem of only have 2GB of memory, as the VM could "have" 64GB. The answer is of course no. Smiley
etotheipi
Legendary
*
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
September 07, 2013, 10:02:43 PM
 #2518

I'm not 100% convinced of that.  The issue is not slowness (well, it is an issue), but that these systems may be running out of RAM.  For instance, it just can't be run on 32-bit because of the lack of address space.  But if you virtualize a system that runs 64-bit OS with 16 GB of RAM (using 14 GB of disk), it may be possible to actually run it even though it takes 3 hours to sync. 

I'd be interested to see someone try it, though I would bet 2:1 that it still doesn't work at all.  But I can see why it might work.

Can you run a 64 bit VM on 32 bit hardware? That doesn't seem right.

Anyway, the original question was whether setting up a VM would help alleviate the problem of only have 2GB of memory, as the VM could "have" 64GB. The answer is of course no. Smiley

From stackoverflow:

Quote
You can't run a 64-bit VM session on a 32-bit processor. However, you can run a 64-bit VM session if you have a 64-bit processor but have installed a 32-bit host OS and your processor supports the right extensions.

It's my understanding that every desktop/laptop CPU under the sun is 64-bit by now, even though many people still run 32-bit OS.  So it is most likely possible to do this. 

Yes, slow as dirt.  Painfully slow.  But it might actually, eventually work, instead of seg-faulting when it hits 94%.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
cypherdoc
Legendary
*
Offline Offline

Activity: 1764



View Profile
September 08, 2013, 02:11:09 AM
 #2519

Bug:

Alienware laptop, 32 bit OS, Ubuntu 12.10, 16 GB ram, I7 1st gen.

Armory  0.88.1 launches fine but self crashes after about 5 minutes.
Roy Badami
Hero Member
*****
Offline Offline

Activity: 562


View Profile
September 08, 2013, 12:07:23 PM
 #2520

I'm not 100% convinced of that.  The issue is not slowness (well, it is an issue), but that these systems may be running out of RAM.  For instance, it just can't be run on 32-bit because of the lack of address space.  But if you virtualize a system that runs 64-bit OS with 16 GB of RAM (using 14 GB of disk), it may be possible to actually run it even though it takes 3 hours to sync. 

I'd be interested to see someone try it, though I would bet 2:1 that it still doesn't work at all.  But I can see why it might work.

Can you run a 64 bit VM on 32 bit hardware? That doesn't seem right.

Anyway, the original question was whether setting up a VM would help alleviate the problem of only have 2GB of memory, as the VM could "have" 64GB. The answer is of course no. Smiley

From stackoverflow:

Quote
You can't run a 64-bit VM session on a 32-bit processor. However, you can run a 64-bit VM session if you have a 64-bit processor but have installed a 32-bit host OS and your processor supports the right extensions.

It's my understanding that every desktop/laptop CPU under the sun is 64-bit by now, even though many people still run 32-bit OS.  So it is most likely possible to do this. 

Yes, slow as dirt.  Painfully slow.  But it might actually, eventually work, instead of seg-faulting when it hits 94%.

There's a bit more to it than that.  To run 64-bit VMs your CPU also needs to support Intel VT (Virtualization Technology) or the AMD equivalent, and VT has to be enabled.  Most CPUs do support VT, but I believe Intel doesn't include it on some of their low-end CPUs.

But having a VT-capable CPU isn't enough - VT needs to be enabled, and this is normally done by the BIOS.  Often VT isn't enabled by default, but there is a BIOS setting to enable it.  But if your computer manufacturer chose not to provide a means of enabling VT then you may be out of luck, even if your CPU has the requisite capabilities.

roy
Pages: « 1 ... 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 [126] 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 ... 232 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!