Bitcoin's cryptographic trust model seems to place too much trust in cryptography and too little acceptance of reality. Network and user security is a virtual wild west and likely forever will be. The moment a device is connected to a network, it is vulnerable to intrusion. That in itself is where the breakdown in trust starts. Black and white hat hackers have been playing leapfrog for state-of-the-art since the beginning. As we continue to pour more of our lives into the ether this will go on.
The problem we keep seeing may not be with the protocol itself, but the implementations. Bad code interfacing with the protocol (ie. at an exchange), lax security update procedures, known OS holes, theft/'insolvency'... these all are real world barriers to the widespread acceptance of bitcoin.
A major drawback to the decentralized nature of bitcoin is that some very critical tasks (ie. security) are largely left up to the end user. Today this includes trolling these boards to stay on top of security issues, which exchanges/services/merchants are legit/solvent, keeping wallets up to date, etc. Mainstream Anytown, Anywhere doesn't have time for this. It's just money, we're used to taking these things for granted (and paying for it).
Will bitcoin ever get to the point where I can trust my grandma with her pension?