You actually came up with a very good point. In fact, the great thing about Bitcoin is that Bitcoin doesn't need all people to accept it for it to be successful. As long as they have access to the network and their private keys, anyone can still hold Bitcoin for Long term and also transfer their Bitcoin.
It does not but it still need people to grow. If you look at the Bitcoin market cap, it's sitting around 1.5 trillion equivalent price of $80k. Do you think the available money can make it go as high as 32T and above to compete with Gold and other world class assets with the largest market cap? Even the 1.5T it's traded wasn't done by few people, the money came from different people that bought, they are the reason why the price is here trading this high.
Unlike the normal financial system, in which you need to depend on the banks, payment companies or others who are between you and your cash. That's why Bitcoin is so special compared to any other investment, as you see, everyone has the freedom of ownership.
The regular financial market is controlled by the government and they control how the money flows. If the government want an asset to be valuable, it's small thing for them. If they want Bitcoin to be near up high like Gold, they can do it but they will not because they don't contol the supply. The government plays a vital role in stability of things, even if they don't play part in the price, they can play a major role in adoption and price.