Bitcoin's biggest problem is that people still trust the current financial, currency and banking system.
I think that bitcoins biggest problem's are the following:
1. People just don't understand btc. The average person doesn't understand the whole concept of what bitcoins actually are.
2. No charge backs while this is one of the biggest advantages it is also a downfall for those who fall victim to scammers.
3. The price is always changing therefore people may be weary of actually investing in btc when they can potentially loose money.
If Bitcoin were somewhat mainstream tomorrow, there will still be a
large group of people who
want charge backs, because they want protection, and I agree with picolo that people trust the current system to hold their hands and protect them.
They will argue that there are too many scam sites, exchanges can't be trusted with cryptocurrency (whether they've been hacked or inside jobs they're not very trustworthy as a whole right now), computers and online wallets are too easy to be hacked, and there are plenty phishing sites and emails.
While many of those can be said about fiat also, fiat is insured and chargebacks make people feel safe.