This vulnerability alone is not the reason why people should stop using Ledger. First of all, Ledger is partially open source but its core firmware tied to secure chip remains closed source and here is the most important part, Ledger has been lying to us that keys never leave Secure Chip. Later, they introduced Ledger Recover service, which made it clear that keys leave Secure Chip. So, basically, the most important part of their code is closed-source and they also lie to us. Basically, we don't know the code and they don't tell the truth, so it's a big no.
Ledger also said the seed phrase sent to the recovery services can be given to the government if the government demanded for it. That means the coins people have on ledger, they do not have full control over it. People will always be people, they supposed to all stop using Ledger after doing all these.
When talking about privacy, Ledger is the worst that you can use. Scammers and hackers may know that you are using Ledger. But this is becoming common to other wallets.