I understand the OP his vision on this.
Why should you trust people that claim you should trust in the math it's build on.
I mean the only way you can confirm this is to take a look at the source code.
But what if you cannot read C++?
Well, then you need to trust the people that claim it's alright.
You're right to a degree; you do need a certain amount of trust in
someone. The beauty of bitcoin is that the Satoshi client is not all there is; plus there are customised versions of the client; plus there are those that can read C++ and do look. You have to believe in a truly massive conspiracy to think that there is a maliciously inserted back door in Bitcoin. Every single person involved in the conspiracy would need to have an incentive from it; and since that would include miners, and new miners can always joins, that is not a fixed pool. That means the conspirators would have to be constantly recruiting, and recruiting in secret.
You're right then that we need to trust that the above isn't happening. It's not an easy leap to make though.
Even given all that though; there is nothing to stop you sitting with the protocol specification, and a hex dump of whatever transactions you thought were dodgy and verifying them by hand. I'm not suggesting you do; but it's not outside the realms of possibility. That possibility adds to the unlikelihood of the conspiracy.
But those people will also probably have Bitcoin and won't say Bitcoin is a shitty currency since it's at there own intrest to make Bitcoin look good.
If they're holding bitcoin they have an incentive to ensure there is no back door; and that what they're holding really is secure. If they discovered it was not; then they could sell their holdings
then reveal the insecurity. There would be no advantage to keeping it secret, since someone else might discover it as well and invalidate their holdings before they can sell them.
In short: bitcoin holder's biggest incentive is not to try to prop up a bad currency; but to ensure that what they do hold really is secure.