Hi Steve,
I believe that the restrictions actually originate with cloudflare, a service used to protect websites by regulating traffic and preventing DDoS attacks which serves as a 'middle-man' between a website (ie, data.mtgox.com) and you. I don't think there is actually a definite limit of 10 requests per 10 seconds, I've ran an app before at roughly 20 per 10s period and it's tended to function fine, the limits are more recommendations - levels at which you run no risk of being blocked.
Cloudflare also introduces a lot of caching, which seems to be giving you problems. The strategy I would recommend is a mix of the websocket and http api. The websocket shouldn't suffer from any caching (at least from cloudflare, there is still going to be some internal caching for the ticker, but that should be much more frequent), however I have heard of trouble and uncertainties in performing trades. I suggest that you therefore use sockets for all requests which don't actually do any actions, those which just get information such as ticker price, order info, account info, etc. All critical requests, such as placing orders, cancelling them, etc, should be run through the http api.
Addressing some of your points more specifically: There seems to be a lot of variance in caching for people, generally my own requests seem to be very current, with ticker price updating at least every 1s, sometimes even better. Some however experience much less frequent updates. This is probably due to cloudflare identifying certain regions, ISPs and IP ranges as more active than others, and so their different CDN servers probably cache at different rates, depending on this factor. Hence why for these information requests it's better to use the socket api.
I believe the socket API also, by default, pushes you trades as they happen on the "trade.BTC" channel which you are autosubscribed to, so this should hopefully fill in your trade data, if I understand you correctly. There are also ticker and depth channels which regularly push you up-to-date info. Please note that when using HTTP methods over the socket API you need to use the v1 methods (info on implementation
here).
If you need historical trade data though, please avoid the API and have a look at
the new (soon to be official) method and
my tool for more info. These are still in development and the data is not current yet, but in the near future the mechanism and the tool should be ready.
I hope this answers all your questions, let me know if you're still unsure about anything.
Thanks,
nitrous
