The public are afraid that when cryptocurrency comes to stay or when cryptocurrency is fully implemented that most companies will not have work again and some countries may go into recession. That's the public perception. This is not true, why? Because whether there is cryptocurrency or not people are going to participate in the activities of financial transactions. To me, the true of the matter is that some persons have failed to understand the use cases of blockchain and which area they should focus on and this is the reason for all these fear.
For me, it is not the public that is afraid of Bitcoin, it's the government. And the government controls publicity and it influences the mass population. If they say that digital currencies are not good and is a threat to economic safety, majority will believe them. The lack of central authority is what makes bitcoin itself, and it's what the governments fear for.
Control. Control matters for the government, they need to watch over the monetary transactions. They dictate banks to produce or destroy money out of thin air. They control the regulation of fiat money to control the amount of money spent on investments, savings, etc.
If bitcoin were to take over the monetary system, banks will be irrelevant, however this will have a downside too.
