What someone with a lot of money could do, (and it would probably cost them dearly), is to pump and dump, over and over. They could jack the price up to sky-high amounts, and then dump all the bitcoins they bought, making the price incredibly unstable, and scaring everyone away.
Yep, this is a risk with anything which has a relatively small market cap. You don't need to buy up everything in order to be able to manipulate the market.