At the moment, none of the upgrades are critical. You should/need to be upgrading Bitcoin-Qt/bitcoind, because you risk forking yourself off the main chain after May 15. But the Armory upgrades are usability and feature upgrades, not security upgrades.
You should receive notifications about new versions through the app, when you restart. It will show you what has changed, and you can decide whether or not you want to upgrade. If the upgrade is important, I'll make it the first item in the list of changes: i.e. "Security-Critical Update. Please upgrade immediately!". So far, I've never had to do that...
The next major update, will be the one that dramatically reduces RAM usage and startup time. That is an important upgrade for many users. But it's still just a usability thing... For now, you should be fine with any version past 0.86.3, which included some upgraded code for dealing with new blockchain format in the latest Bitcoin-Qt.
Thanks a bunch!
