Privacy can also be another main issue when all your Bitcoin is stored in one wallet, it becomes easier for anyone who has knowledge of one of your addresses to easily analyze your transactions and estimate your total holdings which reduces your financial privacy on the block chain.
You are right, meanwhile you should also have it in mind that modern wallet allows you to make multiple addresses,
So in that case if someone manages to get one address they will not be able to see everything,
If you have 2 address and they get to know one, they will only be able to see transactions you have made in one address, without the knowledge that you have more under that wallet, so to an extent that is also another way of getting privcy with having to create another wallet.
Having to create multiple wallets to invest isn't a bad Idea, but to make things easier you can create more address and even more seed phrase, with that you are doing yourself a favour by protecting your bitcoinn.