Thanks krishnapramod for you explanation. Did not realize effectively coinbase was keeping my coin. What if they got hacked?
Looks like once I invest large enough amount I need to move the coin to my own private wallet. Given the current blockchain transaction cost it's not economical to do so for small bitcoin amount.
Yes, please do this if you want to add more securities to your bitcoin as you have planned to invest in bitcoin for a long-term, the best wallet in the market id Ledger Nano S (
https://www.ledgerwallet.com/products/ledger-nano-s) as you can keep bitcoin out of the online network, it's extremely secure, just keep it in the safe place. If you want a free wallet, you can install electrum wallet on your desktop or download it to your smartphone.
Because once an exchange gets hacked than everyone who has coins in the exchange are in loss.