Nothing more on the dashboard. If you were running this on Linux, you'd see a report in the terminal for each 2500 block milestone. We'll adjust the progress bar granularity eventually.

HDD is getting murdered I see. I'll test building on a HDD and optimize fullnode for that. Supernode is hopeless on a HDD to begin with.

What's your RAM and CPU usage at again?

There is a wealth of solutions and yours certainly is an acceptable one. Thank you for your concern. However, the issue is about segregation of duties, not implementation. This simply is not a part of the codebase I deal with right now, and every time I ran across this bug in the past few months, I failed to pass it on to the right person.

CircusPeanut is keeping track of everything being reported in this thread and assigning tasks to the proper personnel. This may sound selfish, but I deal with the DB issues, not the rest. Maybe 90% of the DB code was changed this time around and I am solely focusing on getting that part solid. Also I don't usually answer unless the issue is dire or I have at least some sort of a fix worked in already. However I do read all the posts in this thread.

I'm already working on a couple bugs I found on my own a few days ago, although I don't expect any of you will run into those. I'm also paying attention to DB building speed with the different machines you guys run on. From that I will decide if there is need to make anything faster. Keep in mind that I expect most private users will be running fullnode, not supernode, so give that DB type some love too =)

Links have been down for a while now. We're aware of the snafu but have been lazy about it I guess =(

It's not helping that the OS is constantly moving stuff in and out of the swap, it eats disk I/O for no reason. You can try without.

You can halt the scan process at any time, even by killing the process. The DB is resilient to segfaults.

As I said LMDB is a mapped DB. File mapping is entirely controlled by the OS. Depending on how much resources your other active processes require, the OS will throttle down the amount of physical RAM it makes available to hold the mapped file. This in return will crush the write speed (which needs to resolve the page on which the key is or will be before it can write in the data).

My scans take usually an extra 30-40% speed if I play old steam games in the meantime.

To give you an idea, I use a laptop with an i7 haswell, 32GB DDR3 and 3 micro SSDs in RAID0 and no swapping file. It takes me about 1h30 to copy the blocks over and 3h30 to scan supernode (last time I did it was around block 335k). About 45 minutes to scan fullnode, but that's cause the processing is performed in a single thread. Fullnode scans could be lightning fast if I multithreaded that part, as they require very little writing past the block copying. However, that again was not a priority.

I can't wait for DDR4 in laptops so that I can move to 128GB RAM

The DB creation works in 2 phases:

1) Building, which is simply copying the blockchain as a whole over to the DB. This is the phase you are in, which takes very little resources. This code is single threaded and was barely modified in this release (only to support headers first Core). It could be made faster but this isn't a priority in any way.

2) Scanning the transaction history: this part requires heavy IO and processing, and has been largely modified. It will eat all the RAM on your system and I/O capacity of your drives, take several hours/days, and runs on 2 to 4 threads (possible more to come).

You are still building the DB (copying block data over), not scanning (actually parsing the transaction data). When scanning supernode, you will be using over 2GB RAM at least.

Hi,

I'm responsible for the DB code. For those who care, it takes me 3h30 to build supernode, which approximates 90GB with the current blockchain size.

If you have less than 16GB of RAM and a HDD, do not expect supernode to build under a week. Your machine can't handle it. This DB type tracks ALL transactions in the blockchain, it's meant for heavy duty servers. You should at least try and see how long it will take your system to build fullnode before thinking of going with supernode.

The obvious bottleneck is I/O. Moreover, LMDB maps its whole dataset in RAM, so a machine with little RAM and a swapping file on a HDD will get crushed.

Yeah it's fully implemented and set by default. We still use cryptopp only


Translation tags (tr()) have introduced some bugs, we'll get on top of it

I meant that I haven't heard of another change to the raw block files since the announcement of headers first downloading (which involves out of order block data, indeed). We are ready for that part with the upcoming release, but 0.92.3 and older aren't.

Glad it went well. I hope users that read this thread resort to the ticket system instead of banging their heads on their keyboards instead.

It will scan, but fail to see new blocks while running if the headers appear out of order, maybe even seg fault. A restart would probably display the new blocks.

0.92.3 will choke on out of order headers but dev (upcoming 0.93) is ready for headers first. As for the DB, if you mean Core's underlying database, Armory doesn't use that, it only relies on the raw blocks. I haven't heard of a change to that format since headers first.

Just wait for Matt to take care of the ticket. If it's something complicated (I doubt it is), he'll escalate and I'll take care of it.

Same weird guy asking for your logs -_-". And I'm only delirious on certain days of the week.

Need both armorylog.txt and armorycpplog.txt

Not sure about the confirmation email, we changed ticket engine a short time ago, and I'm not familiar with the new one yet.

I can tell from your screenshots that Armory is not online. It's in an inconsistent state (STATE != COUNTRY), or situation if you prefer, or status, or condition. Anyways, it's not online. Add your logs to the ticket and wait for instructions on that end.

At least someone got a laugh out of this.

You used the right address. Bitcoin addresses come with a checksum so you couldn't just have messed up the address and sent your coins to blackhole. As bc.info displays, this address only has one transaction. If the coins would have been stolen, there would 2 transactions, yours funding the address, and one for the thief looting you.

Yet Armory shows neither 1 or 2 transactions, or any number for that matter. It shows 0, so obviously it isn't online.

Now, may it be mother Russia or not, dev tells you what to do, so make that ticket!

Our solution is to side step USB entirely, but for the sake of the argument:

The industry standard is to sell programmable MCUs (with NVRAM), where the USB firmware shares the same address space with whatever extra code the 2nd stage manufacturer will purpose it for. For the USB storage class, there is very little custom code to add, however the implication is that the MCU is mounted on a PCB that is wired to allow updating the NVRAM, usually through ICSP.

Bottom line, you are often sold USB devices that could be entirely re-purposed through an easy firmware update, and this is where the vulnerability lies.

There are 3 solutions to this dilemma:

1) Use MCUs with cooked-in logic: there is a class of MOS (i think) to which you can burn in the firmware permanently the first time you program it in. This would create firmwares that can't be messed with. A firmware attack would have to limit itself to the device's RAM, and that won't survive a power cycle, which would limit the infection to the one machine.

2) Destroy the "commute to write" pin. To set the NVRAM to write mode you need to power a dedicated pin. However, destroying it might be complicated, those QFP style packages have some very small pin spacing, chances are you could damage the entire MCU or the board trying to take down a single pin. If you manage to, you got yourself into the same situation as 1)

3) Side step USB entirely: it's a "managed" port, that needs firmware on both host and client side, too many vulnerabilities.

Keep in mind that even a sane USB device (on which no firmware attack will survive a power cycle) is not safe vector per say. You could simply attack the data written on it rather than the firmware, like say, tamper the change address (Armory will warn you of that however).

I stress this point because in an adversary situation (when an attacker can run arbitrary code on your machine), it will be entirely easier to modify the data written to the USB key than attack its firmware. Granted, it may not yield as large a reward as the USB attack.

Make a ticket, attach your log files, our Mac maintainer will look at it.

https://support.bitcoinarmory.com/home