I finally got a few hours to finish up some work on block finding probabilities and how that affects any attempts to estimate the instantaneous hash rates. The results surprised me as they show that we're very likely to underestimate or overestimate hash rates over any fairly short period of time (e.g. a few hours or even a day). Even when we get out to looking at a 14 day period, statistical randomness can end up playing a much bigger impact on individual difficulty changes than we might expect. It's actually quite likely that individual difficulty changes could be "wrong" by several percent and it's equally possible that it could take quite a number of subsequent changes to correct for it.

The full analysis is over at: http://hashingit.com/16-hash-rate-headaches

Here's the probability curve for a 14 day period in which the hash rate actually stays constant:

It's surprisingly likely that randomness will lead us to being above the 90th centile or below the 10th (roughly once every 5 difficulty changes) so there's far more noise in the difficulty numbers than we might hope. Long term trends are pretty good, but short term changes have far more random volatility in them. As an example even if the hash rate was totally static we have a very high probability that once every 2 years there will be either a +5%/-5% difficulty change and that every 5 difficulty changes we would expect to either see a +2.8%/-2.8% difficulty change