It's not ideal, but it's not terrible either, providing you don't keep it on a computer and save it offline. Create it on a safe and offline computer and keep it on a USB and/or CD-R etc.
If you boot from a ubuntu live disk and do something like back up your wallet or copy your keys down you really will be fine.
Another thing is that if you're using system without full disk encryption it's possible to find encryption keys used with Office 2013 document within swap file or hibernate image and get access to protected document. If somebody got access to your computer. Actually it's simply possible to undelete the file which was extracted from protected archive/document to temporary file. Also during editing process Office can save temporary files anywhere it wants, no guarantee at all.
Okay so, here's an idea. Let me know your opinions.
I create a live disk of Windows on an offline/virus-free computer using WinPE:
http://en.wikipedia.org/wiki/Windows_Preinstallation_Environmenthttp://en.wikipedia.org/wiki/List_of_live_CDs#Microsoft_Windows-basedI boot up the live disk (without connecting to the internet) and install Office on it, paste in the private key, add a password, save the file. Then move the encrypted word file onto a flash drive
Since everything is done in Random-access instead of Read-only, there are no traces left after I power off the computer