in my opinion, when talking about safety, the order should be a little bit like this (from unsafe to safe)
- online wallets (unsafest, since a thirth party has your private key, hacking,...)
- wallet software on your client connected to the internet (bitcoin core, electrum, multibit)
- cold storage (paper wallet, usb stick,...)
- hardware wallet
I guess there is some discussion about the last 2 options, but generally, i think there will be consensus that online wallets are the worst when it comes to security.
I would stick to this order and pick the level of security you need need OP.
The biggest pro of online wallets is that they are no maintenance, fast to access and easy to spend. If you pick a good password, your coins are safe as long as the online wallet service remains.
As for cold storage/ usb, I would only use this as a back-up of your wallet or to make some rare transaction from you cold storage to hot wallet. If you plug it in everywhere, it is just as unsafe as your wallet software. Nice thing is ofcourse that you have easy access to it if you carry it around.