Armory 0.93 testing release! (with 0.05 BTC bug bounty)

[zombieslayer9099]

...

Mine goes slightly slower than yours, and it fluctuates the exact same way.
My Bitcoin QT also disconnects, then reconnects roughly every 1:45 hours.

I noticed that read/write ops are running at the same speed that random ones do. Perhaps Armory or its new DB format isn't writing correctly?

I'd still like to know why building databases is taking forever.
I'm running on OSX 10.10 with 16 GB of RAM and an SSHD with approx 85 MB/s read/write

I am planning on buying an SSD in the near future

I'm having the same problems/questions (I'm using --supernode, haven't tried without it): slow database building, about 1 MB/sec disk access during database building and 3 MB/sec during transaction scanning instead of max (about 30 MB/sec on my HDD), and now that the database is built, even slower transaction scanning (estimate ranging from 3 days to 2 weeks, usually about 6 days). What is the bottleneck here, since it's obviously not CPU (24 mins of CPU time used in 17 hours of running - usually 0-1% use) or hard drive access?

Wait, so scanning transactions takes 6 days on average, you're saying?
If it really takes that long, I'm waiting for 0.93 stable release, I'll take my chances with 0.92.3(0.92.3 worked fine with 0.10).

[1714635171]

1714635171

[TimS]

Wait, so scanning transactions takes 6 days on average, you're saying?
If it really takes that long, I'm waiting for 0.93 stable release, I'll take my chances with 0.92.3(0.92.3 worked fine with 0.10).

On my machine, according to current estimates, yes, 6-7 days appears to be realistic to scan transactions for a supernode. I'm not surprised that the times are higher than for a standard node in the old version, but not being able to see the bottleneck does have me concerned. (if it were obviously pegging the HDD access, memory access, or one core of CPU, for instance, I would understand it)

[CHAOSiTEC]

...

Mine goes slightly slower than yours, and it fluctuates the exact same way.
My Bitcoin QT also disconnects, then reconnects roughly every 1:45 hours.

I noticed that read/write ops are running at the same speed that random ones do. Perhaps Armory or its new DB format isn't writing correctly?

I'd still like to know why building databases is taking forever.
I'm running on OSX 10.10 with 16 GB of RAM and an SSHD with approx 85 MB/s read/write

I am planning on buying an SSD in the near future

I'm having the same problems/questions (I'm using --supernode, haven't tried without it): slow database building, about 1 MB/sec disk access during database building and 1-3 MB/sec during transaction scanning instead of max (about 30 MB/sec on my HDD), and now that the database is built, even slower transaction scanning (estimate ranging from 3 days to 2 weeks, usually about 6 days). What is the bottleneck here, since it's obviously not CPU (24 mins of CPU time used in 17 hours of running - usually 0-1% use) or hard drive access?

it took 18 hours here to build the database running on ssd drive

[zombieslayer9099]

I just noticed one more thing in the logs.

-INFO  - 1420984504: (BlockUtils.cpp:1119) Starting at block file 216 offset 124278660
-INFO  - 1420984504: (BlockUtils.cpp:1121) Block height 338437
-DEBUG - 1420984504: (Blockchain.cpp:208) Organizing chain w/ rebuild
-INFO  - 1420984816: (BlockUtils.cpp:1236) Loading block data... file 21 offset 45157597

-INFO  - 1421068874: (BlockUtils.cpp:1119) Starting at block file 217 offset 39342749
-INFO  - 1421068874: (BlockUtils.cpp:1121) Block height 338585
-DEBUG - 1421068875: (Blockchain.cpp:208) Organizing chain w/ rebuild
-INFO  - 1421069155: (BlockUtils.cpp:1236) Loading block data... file 48 offset 62952398

[jammers]

Finally finished building the database while I'm out, now scanning transaction history. 9% done, 15 hours to go.

Is there any way armory could use a GPU as well as a CPU to help speed these up?

[doug_armory]

I'd still like to know why building databases is taking forever.
I'm running on OSX 10.10 with 16 GB of RAM and an SSHD with approx 85 MB/s read/write

I am planning on buying an SSD in the near future

This must be a subtle bug of some sort. I'm running OSX 10.9.5 (I'll upgrade to 10.10 soon) with an SSD, and I've never had issues with DB building. My 10.10 VM works fine too. Perhaps you're running a bit low on space? I'm not aware of any other potential causes right now, as this seems to happen to some people on every OS.

Thanks.

[TimS]

Perhaps you're running a bit low on space?

This isn't the issue in my case: my 2TB HDD with the Armory DB and Bitcoin block chain has at least 745GB free when doing this. My OS hard drive is a smaller SSD with 36.7 GB (out of 232) free.

[Carlton Banks]

My OS hard drive is a smaller SSD with 36.7 GB (out of 232) free.

That can cause issues, whether it's the source of this problem or not. It's not a good idea to leave your system disk with anything less than about 20% free space, more so with SSD's.

Still, with plenty room on the system disk and the Armory datadir on a separate disk to that, it took me 6 hours to build the 0.93.x database, and about 5 hours for the full scan.

[jammers]

Just back in now, says it's scanning transaction history and done 26% so far, reporting 18 hours to go. So looks like it'll take from lunchtime Sunday, till tomorrow morning (Tuesday) to have it installed, downloaded the bootstrap torrent, synced from there, build the database and scan the transaction history. 48 hours is a LONG time. I know it has to be done once, but is it just me on a normal hard drive having this?

[goatpig]

Hi,

I'm responsible for the DB code. For those who care, it takes me 3h30 to build supernode, which approximates 90GB with the current blockchain size.

If you have less than 16GB of RAM and a HDD, do not expect supernode to build under a week. Your machine can't handle it. This DB type tracks ALL transactions in the blockchain, it's meant for heavy duty servers. You should at least try and see how long it will take your system to build fullnode before thinking of going with supernode.

The obvious bottleneck is I/O. Moreover, LMDB maps its whole dataset in RAM, so a machine with little RAM and a swapping file on a HDD will get crushed.

[zombieslayer9099]

snip

The obvious bottleneck is I/O. Moreover, LMDB maps its whole dataset in RAM, so a machine with little RAM and a swapping file on a HDD will get crushed.

I have 16 GB of RAM and a 1 GB swap file on my SSHD(85 MB/s read/write), it still goes extremely slowly, and I see Armory using ≤41 MB of the swap file and ≤200 MB of RAM. Armory only uses about 1 MB/s of IO right now.

[goatpig]

snip

The obvious bottleneck is I/O. Moreover, LMDB maps its whole dataset in RAM, so a machine with little RAM and a swapping file on a HDD will get crushed.

I have 16 GB of RAM and a 1 GB swap file on my SSHD(85 MB/s read/write), it still goes extremely slowly, and I see Armory using ≤41 MB of the swap file and ≤200 MB of RAM. Armory only uses about 1 MB/s of IO right now.

You are still building the DB (copying block data over), not scanning (actually parsing the transaction data). When scanning supernode, you will be using over 2GB RAM at least.

[zombieslayer9099]

You are still building the DB (copying block data over), not scanning (actually parsing the transaction data). When scanning supernode, you will be using over 2GB RAM at least.

But why does it use such little resources? That's what I don't understand.
About 40% of RAM is free on my system at any given time, why doesn't Armory take it? Same with IO.

[TimS]

snip

The obvious bottleneck is I/O. Moreover, LMDB maps its whole dataset in RAM, so a machine with little RAM and a swapping file on a HDD will get crushed.

I have 16 GB of RAM and a 1 GB swap file on my SSHD(85 MB/s read/write), it still goes extremely slowly, and I see Armory using ≤41 MB of the swap file and ≤200 MB of RAM. Armory only uses about 1 MB/s of IO right now.

You are still building the DB (copying block data over), not scanning (actually parsing the transaction data). When scanning supernode, you will be using over 2GB RAM at least.

I have 16 GB of RAM as well, and my swap file is on my SSD. However, I have another task using just under 9GB. Armory uses about 900MB in the scanning stage. Is Armory limiting itself because it sees that there isn't a lot of RAM to spare (still over 2GB free that Armory isn't touching), or is something else the cause of this 900MB vs +2GB thing?

[goatpig]

You are still building the DB (copying block data over), not scanning (actually parsing the transaction data). When scanning supernode, you will be using over 2GB RAM at least.

But why does it use such little resources? That's what I don't understand.
About 40% of RAM is free on my system at any given time, why doesn't Armory take it? Same with IO.

The DB creation works in 2 phases:

1) Building, which is simply copying the blockchain as a whole over to the DB. This is the phase you are in, which takes very little resources. This code is single threaded and was barely modified in this release (only to support headers first Core). It could be made faster but this isn't a priority in any way.

2) Scanning the transaction history: this part requires heavy IO and processing, and has been largely modified. It will eat all the RAM on your system and I/O capacity of your drives, take several hours/days, and runs on 2 to 4 threads (possible more to come).

[goatpig]

I have 16 GB of RAM as well, and my swap file is on my SSD. However, I have another task using just under 9GB. Armory uses about 900MB in the scanning stage. Is Armory limiting itself because it sees that there isn't a lot of RAM to spare, or is something else the cause of this 900MB vs +2GB thing?

As I said LMDB is a mapped DB. File mapping is entirely controlled by the OS. Depending on how much resources your other active processes require, the OS will throttle down the amount of physical RAM it makes available to hold the mapped file. This in return will crush the write speed (which needs to resolve the page on which the key is or will be before it can write in the data).

My scans take usually an extra 30-40% speed if I play old steam games in the meantime.

To give you an idea, I use a laptop with an i7 haswell, 32GB DDR3 and 3 micro SSDs in RAID0 and no swapping file. It takes me about 1h30 to copy the blocks over and 3h30 to scan supernode (last time I did it was around block 335k). About 45 minutes to scan fullnode, but that's cause the processing is performed in a single thread. Fullnode scans could be lightning fast if I multithreaded that part, as they require very little writing past the block copying. However, that again was not a priority.

I can't wait for DDR4 in laptops so that I can move to 128GB RAM

[jammers]

16gb ram here and quad core i7, but not ssd

now up to 34% into scanning transaction history. my cpu  is low, but disk and ram usage nearly 100%

I have a large swapfile though, so I wonder if that's what's causing the delay in my case?

[goatpig]

16gb ram here and quad core i7, but not ssd

now up to 34% into scanning transaction history. my cpu  is low, but disk and ram usage nearly 100%

I have a large swapfile though, so I wonder if that's what's causing the delay in my case?

It's not helping that the OS is constantly moving stuff in and out of the swap, it eats disk I/O for no reason. You can try without.

You can halt the scan process at any time, even by killing the process. The DB is resilient to segfaults.

[jammers]

I stopped it while it was at 41% scanning transactions, going to get rid of the swapfile, reboot, and then try again. fingers crossed! It's initialising the bitcoin engine, so hopefully it'll get through initialising the bitcoin engine, building the db's and picking up from scanning the transaction history.

It's just synced with network now, and flying through loading database headers, took about 4 minutes. Lets see how long it takes to build the database. Not very long at all, went straight from 0% to 99% straight away. Went out for a few minutes and when I came back, it's back to scanning transaction history at 41%. There isn't any noticeable difference at all whether pagefile is on or off, still taking forever. I'll leave it running overnight and hopefully my wallets will be up to date in the morning.

But

You can halt the scan process at any time, even by killing the process. The DB is resilient to segfaults.

+1

Just another minor bug, I noticed noticed that the link on the welcome message isn't working

Links to https://bitcoinarmory.com/faqs (which shows page not found), should be https://bitcoinarmory.com/faq/



Btw, a quick way to fix all of these broken links is to install something like https://wordpress.org/plugins/page-links-to/ , and then you can set up the pages to link to the right one.

[goatpig]

Links have been down for a while now. We're aware of the snafu but have been lazy about it I guess =(