A cold wallet is not connected to the internet, which makes it virtually immune to hacking, malware, and phishing attacks.
You mentioned hardware wallets. Ledger and Trezor had vulnerability issues in the past.
Also phishing emails are often sent to Trezor and Ledger users from the leaked emails. Anyone that do not know about how to avoid phishing attack might fall for it.
People need to be careful of clipboard malware.
Common cold wallets:
Hardware wallets like Ledger, Trezor, Paper wallets like printed copy or a hand written copy ,Air-gapped computers never connected to the internet
With Ledger, you can send your seed phrase to third party online. It is divided into 3 parts and sent to three different companies online. Should we say that is a cold wallet? Its secure element is close source. Not valuing users privacy with how they let hackers breached the emails and home addresses of their users that bought the wallet directly from their site.
I will also advice you not to see wallets that are not airgapped as cold storages.
You Control Your Private Keys
Cold wallets give full ownership and control over your private keys it's always not your keys and not your coins if you leave your crypto on an exchange (a hot wallet), you're trusting someone else with your funds
With hot/online wallet, if it is noncustodial, you still also have full control over your private keys. This is not how you should do comparison. Even if the exchange says it is using cold wallet, know that it is still not your key and not your coin so far it is an exchange.
Yes, cold wallets are safe and secure than hot wallets.
Peace of Mind
Knowing your funds are stored in a device or medium that’s disconnected from online threats brings a high level of reassurance this will help you to be at rest and not be too concerned about the safety of your investment especially from scammers .
To backup your seed phrase appropriately is also important. I use passphrase to extend my seed phrase now.
