I believe it's the dumbest thing ever to use a VPS for keeping anything really valuable.
Here's why:
1. management console - while given to you it can allow doing weird things that require repairs via console later it's certainly a potential security breach.
2. Host system/hypervisor hack from the guest OS - while it doesn't happen often, it still can happen.
3. "someone else's error" - I've used lot's of VPS's and a couple of times some sort of "forced upgrade" did happen, with my settings being silently replaced with some defaults, and so on, once I even got a "fresh new VPS" instead of the one I was using. You may say backup & such, but I don't have a guarantee that someone else didn't get mine original VPS, with all the data inside.
4. Encryption - heh, you really believe it's a problem to get into your data for the VPS hosting company if they need (e.g. are forced) to?
As for dedicated setups, security should be a concern as well - e.g. your hard drive starts doing weird things/gets couple of bads - you request it to be changed. what hapens next? Yes, you get the "new" drive but the old one - it can easily slip into someone else's server. with all of your data, sometimes completely intact. The same happens when you cancel service - so to fish something it's enough to order second, third, and so on drive into the server
.
Remember to always wipe your drives before the billing ends and cancellation request is filed.
