2FA I feel is not secure because if someone gets ahold of my phone, then that's not good. And if I lose my phone number, again not good.
What would be the most secure way where my phone or phone number being compromised would not result in my wallet being compromised?
If you're using SMS 2fa, it's time to stop. It's insecure. Easier to intercept, large amount of ppl have been a victim of sim swapping and someone around you can easily take it out of your phone.
There are different types of malware to fall into and it's possible hackers may have already gotten everything they need before you could do anything. I suggest focusing on preventing falling into malwares/scams first OP, it is better than cure
Security a lot of times is interconnected. You have to take care of everything that could be a weakness like your devices, email, browsing habits, how you keep safe sensitive information, what you download, etc.. Try to improve your overall cybersecurity, see:
personal-security-checklist but only take it as a reference, don't go blind on anything. Make sure you fully know what you're about to do cause some are not beginner friendly and might backfire if done wrong.