If everyone stops mining I will run a CPU miner at work and get all of the mined Bitcoins.
But then...not everyone would have stopped mining since I would be mining.
So basically, mining will never stop.
But you could have oscillations in hash rate, much like alternate chains (e.g., Namecoin). 75% of hashing power fires up, mines 2 weeks worth of coins in 3 days, hash rate quadruples. 75% shut down for six weeks. Mine for 3 days, shut down for a six weeks. This would be a steady state where mining power quadruples and quarters, alternating.
End result: everyone but the patsies/free power guys mining at artificially inflated difficulty get their coins at a quarter of the price (but 75% get half the possible coins and it takes everyone much longer overall). Still, it's downward pressure on price as a double whammy of a reduced utility and security of the network (~24 minute confirms 93% of the time) as well as cheaper to produce coins.
It could turn into a feedback loop where mining 7% of the time for a 75% lower cost is the only way to be profitable for the majority. Heck, were that to happen now you could even get CPU miners and Nvidia guys joining the fun during profitable periods. "Unprofitable", artificial difficulty would keep creeping up until even the free power guys are looking at a higher cost to mine vs buy in equipment failure/depreciation and decide to drop out during the bad times. That would require around 99% of the mining power to flee, so I don't think it's possible.