Data source url change (ticker, depth, history)

[MagicalTux]

Hi, You did not mention socket.io api url changes. Will it change too?

And if yes do you know about current socket.io behaviour? Client connects, connect event invoked, and then no any subscribtion message appear. Just silence.

Can you comment on this effect? What can be done?

No change on the socketio url, however we are testing a new system that'll be much better to handle the load. Tests are currently being done.

[snoleo]

In the last 24 hours, data.mtgox.com has served 143513 requests for a total of 25.9 GB bandwidth.

That's just a fraction of what's being served on mtgox.com, but that helps already quite a lot.

Hi Tux,

I think the json data returned by data.mtgox.com could be compressed or reduced. Let's compare the returned json data of regular DepthTable of http V0 version and V2 version:

V0 version:

https://data.mtgox.com/api/0/data/getDepth.php



v2 version

https://data.mtgox.com/api/2/BTCUSD/money/depth



json data size compare:




From above , I suggest that Http v2 version json of depth use the format of http v0 which could reduce more than 90% data size. Or you could place a "switch" like

https://data.mtgox.com/api/2/BTCUSD/money/depth/reduced
or
https://data.mtgox.com/api/2/BTCUSD/money/depth?format=reduced

to let people get the proper data they want.

The website like bitcoin.clarkmoody.com  or info.btc123.com (btc123 caches json data itself for 10 seconds) is designed to maintain a realtime data of ticker , most recent trades and current depth. There could be a very frequent data getting requirement.

Also for ticker and trades:

ticker v0 version:


ticker v2 version:


trade v0 version:


trade v2 version: (The great is that v2 version add trades property "market" "limit" to each transaction.)


From above I think there is a lot of information could be removed from json data, maybe only the yellow background data is essential. And I suggest that the ticker and trades data format could take v0 version depth for reference, or add a switch (or say an option) to provide a version for those who just want to get realtime market information (like info.btc123.com) to get the reduced or compressed json data.

Many Thanks!

[lucif]

Ah there it is!

https://bitcointalk.org/index.php?topic=156833

Well, you suck. Why I should dig forums and newspapers what the fuck is going on with my apps?

Could you please inform about futher similar epic changes like that at least on YOUR FRONT PAGE NEWS? And at most in email newsletter?

[zhalox]

MagicalTux, please read the following posts on this thread, and learn from this highly unprofessional failure at Mt. Gox by improving your business practices.  An official apology from Mt. Gox regarding this serious failure over the past few days would help customers to regain confidence.  Hopefully you learn from this and make better business decisions in the future to avoid embarrassing situations like this for Mt. Gox: https://bitcointalk.org/index.php?topic=156833

I agree, they can't just post in some obscure forum about these sorts of things, they need to make an announcement on their website via email.  Communication is VERY important.  They should create an email list where you can subscribe to updates like this (press releases, announcements, etc.).  However, their failure to update their own products' APIs or at least provide a fail-safe (Mt. Gox mobile Android app, http://classic.mtgox.com) is COMPLETELY unacceptable.

[franky1]

anyone wanting the easy PHP code to get a ticker running as trying to find an easy copy and paste on the wiki takes many pages of searching

Code:

<?
$json = file_get_contents('http://data.mtgox.com/api/2/BTCGBP/money/ticker');
$jdec = json_decode($json);

$check=$jdec->{'result'};

if ($check<>"success") 
 {
     echo "ticker is offline";
 }
else 
 {
     $rate = $jdec->{'data'}->{'high'}->{'value'};
     echo "£" . $rate;
 }
?>

all you have to do is change the BTCGBP in the url to your desired currency and ofcourse the £ in the echo code needs changing if your not looking for english pounds

[bzzard]

MagicalTux - please at least respond to snoleo post - APIv0 data format is actually great

I've talked also with MtGox support on this topic but they ignore this question.


Can you say in numbers - how much cache time is changed?

No change on the socketio url, however we are testing a new system that'll be much better to handle the load. Tests are currently being done.

Sounds great. So whats the name of new system? Or it's still socket.io?

[snoleo]

Hi bzzard,

Are you the author of btccharts.com ?

It is a very great flash real time depth chart and I like it. 

[snoleo]

And I also find that maybe MtGox has cached the json data, since whether I use php curl or Chrome browser directly , I cannot get the trades executed in last 10-15 seconds . Instead , I always get the trades executed 1-3 minutes ago. And users watching the realtime trades data complain about the delays. 2-4 minutes is just too long.


So I guess maybe MtGox has cached the trades data for 2-3 minutes. Tux , would you please give some specification on that?

[MagicalTux]

So I guess maybe MtGox has cached the trades data for 2-3 minutes. Tux , would you please give some specification on that?

Hi,

The v0 getTrades is cached ~120 seconds, even with ?since=. The v1/v2 api is cached 15 seconds only if ?since= is passed.

However if you're looking for realtime activity, you should be using the realtime feeds.

[genuise]

Hi, You did not mention socket.io api url changes. Will it change too?

And if yes do you know about current socket.io behaviour? Client connects, connect event invoked, and then no any subscribtion message appear. Just silence.

Can you comment on this effect? What can be done?

No change on the socketio url, however we are testing a new system that'll be much better to handle the load. Tests are currently being done.

Hi, can you be so kind to tell more details about newsystem. In another thread they mentioned socketio-beta.mtgox.com I tried to connect there with my nodejs socket.io-client lib and got only multiple errors.

while the same lib seems compatible with the old socketio.mtgox.com but it is highly highly unstable. Very often  client connects (get connect event) and then nothing more just silence no any subscribtion mesages. Also I connect with such url https://socketio.mtgox.com/mtgox?Currency=USD,EUR,GBP,CNY,RUB,CAD,PLN,AUD which contains 8 currencies. Could this be an issue in my case?

please advice.

[beckspace]

https://classic.mtgox.com/trade/history not showing info anymore! 

[snoleo]

The v0 getTrades is cached ~120 seconds, even with ?since=. The v1/v2 api is cached 15 seconds only if ?since= is passed.

However if you're looking for realtime activity, you should be using the realtime feeds.

Hi Tux,

1. About the cache time, does getDepth and ticker cached the same time as getTrades in v0?
2. What is the cache time of depth and trades in v1 and v2?
3. Would you please provide a v1/v2 api returned reduced data to save the bandwidth?
4. And if there is an api to provide the most recent 100 or 60 trades?

Sorry for so many questions. Please kindly give some specifications, thanks!

[cilphex]

Hi Tux, I'm new to the API.  Can you tell me what the since= param means?  I imagine it means "trades since this time" or "trades since this far back", but I don't know what unit it's measuring in.  Minutes?  Hours?  Days?

[snoleo]

Hi Tux, I'm new to the API.  Can you tell me what the since= param means?  I imagine it means "trades since this time" or "trades since this far back", but I don't know what unit it's measuring in.  Minutes?  Hours?  Days?

since=N

It will get trades which were execuated after that trades ID N (excludes ID N)

PS: Tux seems too busy to answer our questions , so we have to try to help each other. But who will be kind to come answer my question?  

[2112]

But who will be kind to come answer my question?  

The answer is that they are trying to provide support for the FIX protocol http://en.wikipedia.org/wiki/Financial_Information_eXchange .
Please learn it or be left it the dustbin. There is no way forward that continues to use the old RPC polling paradigm.

Yep, see the top of the thread. The APIs will stay the same at launch. We will probably layer on FIX support for CoinLab customers.

[phelix]

as snoleo pointed out already:

why did you remove the old ticker and bloat it even more? it was too much already.


and the depth... oh my.

[phelix]

Because of the hickups (price jumping randomly +-30%) in the old ticker I switched to the new one today.

Only now do I realize how messed up it really is.


For some reason the standard python json decoder decodes the numbers as unicode strings and not as numbers....

Probably these encoded chars are the problem:
vol":{"value":"72063.77361237","value_int":"7206377361237","display":"72,063.77\u00a0BTC","display_short":"72,063.77\u00a0BTC"

Took me a while to get aware of that, watch out.


edit: Now that was BS, the reason is the quotes around the numbers of course. why did you add quotes to the values?

[cilphex]

2112, thanks for your help with that.

Anybody know if there's a way to specify that you want data since a certain date?  Like if I wanted the past 24 hours of trades, is there a way to do that? 

[shsmith]

2112, thanks for your help with that.

Anybody know if there's a way to specify that you want data since a certain date?  Like if I wanted the past 24 hours of trades, is there a way to do that? 

since is in time units*1000000
to start x hours ago you can calculate
   since = int(time() - x*60*60)*1000000
   url = "http://data.mtgox.com/api/1/BTCUSD/trades?since=" + str(since)

You will get batches of 100 trades, so you need to keep calling
with since set to highest tid in previous batch

[cilphex]

2112, thanks for your help with that.

Anybody know if there's a way to specify that you want data since a certain date?  Like if I wanted the past 24 hours of trades, is there a way to do that? 

since is in time units*1000000
to start x hours ago you can calculate
   since = int(time() - x*60*60)*1000000
   url = "http://data.mtgox.com/api/1/BTCUSD/trades?since=" + str(since)

You will get batches of 100 trades, so you need to keep calling
with since set to highest tid in previous batch

But it was stated earlier that the "since" parameter is for a trade ID, not a timestamp, so I don't think that would work.