So if I want to hold is better to keep my Bitcoin in a wallet or in an exchange ? What's the difference ?
If you want to hold the answer is clearly wallet.
The cheapest option is paper wallets you can build and print yourself, just make sure you do all that offline and in a safe environment.
Of course, hardware wallets are a common option. Air gaped computer is another very good option, just it may be a bit expensive (PC) or technical (raspberry PI).
The difference: an exchange only tells you "that's your money". If they get bankrupt, if they get hacked and so on, they'll tell you "sorry, no more money for you".
If you have a wallet you are the owner of the private key for the addresses the wallet contains. As long as the seed that generates those keys or the keys themselves are safe and backed up and as long as the computer you use to spend those money is safe, your funds are safe.
I said about paper wallet because after you've generated that you can actually keep the private key on paper for real. And you just check on the internet the money that came in, with publicly available info, without jeopardizing the safety of your funds.