Myetherwallet is a ETH wallet which support the ERC20 tokens, many ICO's now a days using ERC token to send you and only some wallets support ERC tokens. No exchange wallet can be used to get ERC tokens. There is only one disadvantage of myetherwallet is that if you lost your private key, you lost all your coins in the wallet. Save your private key in safe place otherwise you cannot retrieve your coins.
I also agree with you and yes, the only key you need to access your funds in the wallet is your private key and when this key is lost, you will be unable to access you funds. That is why it is advisable to always keep this key very private and avoid disclosing it to a third party.