Well I can see another problem: where would mining go?
As I mentioned, it isn't incompatible with (merged) mining. Mining still can be used for initial coin distribution. It just isn't an ultimate authority.
I.e. when central block chain disagrees disagrees with miner's block chain we just stop processing transaction and wait for a manual resolution. (Unlike bitcoin which automatically picks longest block chain.)
Bitcoin isn't yet mature enough, it would be quite less attractive without the money from mining...
Sure, mining is a significant part of appeal. Or 'was', since CPU mining just makes no sense now and even GPU-based mining isn't that profitable.
...but I wouldn't underestimate the psycological effect of being able to say that (in principle) the system is completely decentralized. That what really sets us apart, after all.
'Decentralized' is just a buzzword which can mean many things. Ranging from democratic control to robustness to 'barrier to entry'.
If you remember those threads which discuss blockchain size scaling, people were arguing that at some point (many transactions per second) only large mining operations will ever touch blockchain directly since it would be of humongous size. So getting into bitcoin business at a tier1 player would cost a lot of money to get hardware, but even then, what if largest current operators will collude to ignore blocks you mine? Is it still decentralized?
On the other hand, in scheme I proposed above democratic control is much more straightforward as it comes directly from users, not from mining operations. 'Barrier to entry' is just different -- now bitcoin obviously wins, but in case of miner collusion it would take a lot of $$$ to override it, but in case with centralized timestamping you 'just' need to convince majority of users to switch to your server. Arguably, it might be easier. Centralized timestamping absolutely loses in terms of robustness, though, but that's another story.
If you remember, fate of P2SH was decided mostly among a small number of people (Gavin, Tycho, Luke-Jr and slush), so collusion isn't too far fetched to consider.
(but, there would be the risk of someone invading the network with large amounts of clients, maybe tiny virtual machines on a server... that's why proof-of-work was chosen originally, after all. But maybe it would be possible to stop this behaviour, I don't know)
Yeah, that's the Sybil attack I was talking about. This is why timestamping server should be selected manually and consciously rather than via an automated algorithm, without proof-of-work there is no other way, I think.
If you worry about initial coin distribution, traditional mining can be used for that.
If you worry about choice of timestamping server after a failure, consider this scenario: you want to buy some item from an AwesomePantsStore which accepts weirdcoins. You have no other choice than to accept same timestamping server as used by that AwesomePantsStore, otherwise merchant wouldn't recognize your transaction as valid. There is absolutely no risk for you: if AwesomePantsStore picks a wrong timestamping server, it just risks to lose money due to double spending. So risk always lies on a merchant, buyer can always choose timestamping server suggested by an entity he sends coins to.