You should be fine with that scenario. You can also add the offline package to your live usb. Make a second partition on the USB key and keep all your extra files over there. Then mount it and use.
So I have Ubuntu on a USB stick with a persistent file system. I installed Armory and encrypted a test wallet. My biggest concerns are that I can't seem to get the live OS to require login, and I can't figure out a way to do whole disk encryption and keep USB bootable. It shouldn't be an issue because my passphrases for my offline Armory wallets are complex and long enough, but it'd be nice to force somebody to have to jump through at least one more layer of encryption.