Bitcoin Forum

Bitcoin => Bitcoin Discussion => Topic started by: MagicalTux on March 09, 2013, 01:45:33 AM



Title: Data source url change (ticker, depth, history)
Post by: MagicalTux on March 09, 2013, 01:45:33 AM
Many people around here use our ticker to get the latest Bitcoin price, or other of our public data API (depth, etc).

Because of this, the strain on the system has been growing day after day, and has reached a level where we need absolutely to do something. We now have the ticker, depth and trade history available at a new url.

Because not everyone can afford to change their software to support our api v1/v2 ticker, we made this in a way that just replacing the hostname should work (mtgox.com => data.mtgox.com). Other API calls (ie. any api where you authenticate) should still be done as usual on mtgox.com. Please post on this thread or contact MtGox's support (info@mtgox.com) if you have any issue with this change.

Caching is slightly more aggressive for now, but we will reduce it once we confirm everything works the way we expect it to work. Please note that we added a "now" value to the ticker that will allow you to know how old the ticker is.

Ticker

Code:
http://data.mtgox.com/api/2/BTCUSD/money/ticker
http://data.mtgox.com/api/1/BTCUSD/ticker
http://data.mtgox.com/api/0/data/ticker.php?Currency=USD
http://data.mtgox.com/code/data/ticker.php?Currency=USD

Depth

Regular depth data

Code:
http://data.mtgox.com/api/2/BTCUSD/money/depth/fetch
http://data.mtgox.com/api/1/BTCUSD/depth/fetch
http://data.mtgox.com/api/0/data/getDepth.php
http://data.mtgox.com/code/data/getDepth.php?Currency=USD

Full depth data

Code:
http://data.mtgox.com/api/2/BTCUSD/money/depth/full
http://data.mtgox.com/api/1/BTCUSD/depth/full

"Large" depth data (deprecated)

Code:
http://data.mtgox.com/api/0/data/getDepth.php?mode=large
http://data.mtgox.com/code/data/getDepth.php?mode=large&Currency=USD

Trades

Code:
http://data.mtgox.com/api/2/BTCUSD/money/trades/fetch
http://data.mtgox.com/api/1/BTCUSD/trades/fetch
http://data.mtgox.com/api/0/data/getTrades.php?Currency=USD

Code:
http://data.mtgox.com/api/2/BTCUSD/money/trades/fetch?since=1
http://data.mtgox.com/api/1/BTCUSD/trades/fetch?since=1
http://data.mtgox.com/api/0/data/getTrades.php?Currency=USD&since=1

We will eventually replace the URLs on MtGox.com with Location redirects, however since this will most likely break many applications (follow location is not enabled by default on curl for example) and have a negative impact on performance of said applications, we start with a public announcement.

Please update your applications/software/etc by April 1st 2013.

http://data.mtgox.com/code/png/24hours.png?Currency=USD


Title: Re: Data source url change (ticker, depth, history)
Post by: franky1 on March 09, 2013, 02:12:41 AM
thank you magicaltux. with the ramp and dump the other day i suspect your bandwidth got stretched to breaking point with EVERYONE looking at the charts/tickers.

so im glad to hear your moving the tickers to separate area so that it helps to keep live data flowing and most probably helps ease the ability to add more servers later with higher demand over the next few years, without affecting the other parts of mtgox.

it might be worth adding that it is best to use these tickers for actual 'live day traders' and other people EG just for 'curious/browsing audience' to use charting/data servers such as clark moody/bitcoinity.org. as they are only a few milliseconds different. but it help offload/reduce the strain on mtgox themselves.


Title: Re: Data source url change (ticker, depth, history)
Post by: Donatus on March 09, 2013, 01:41:03 PM
Can you make the change more compatible with the existing api please?  For example, the fulldepth url is documented in the wiki as being...

https://mtgox.com/api/1/BTCUSD/fulldepth

but I don't see that in the OP?

--D


Title: Re: Data source url change (ticker, depth, history)
Post by: klaus on March 09, 2013, 01:52:01 PM
@Donatus

with http://data.mtgox.com they can separate it (even physically) from http://mtgox.com. Thats the intention and its good.


Title: Re: Data source url change (ticker, depth, history)
Post by: jl2012 on March 09, 2013, 03:52:58 PM
Can you make the change more compatible with the existing api please?  For example, the fulldepth url is documented in the wiki as being...

https://mtgox.com/api/1/BTCUSD/fulldepth

but I don't see that in the OP?

--D


You should try before asking for that

https://data.mtgox.com/api/1/BTCUSD/fulldepth (https://data.mtgox.com/api/1/BTCUSD/fulldepth)


Title: Re: Data source url change (ticker, depth, history)
Post by: prof7bit on March 09, 2013, 04:01:50 PM
The websocket server does not respond anymore since yesterday. It connects instantly but then its totally silent, not even sending headers.

(Socketio-websocket still "works" but with a delay of 2+ Minutes on top of the usual order-lag, it won't even ack my calls to give me an oid for my pending orders in less than 2 minutes even when order-lag says "idle", this is almost unusable for trading)

Is websocket.mtgox.com affected by the change? Is it a known problem, Is somebody already working on the problem and will it be fixed?


Title: Re: Data source url change (ticker, depth, history)
Post by: MagicalTux on March 09, 2013, 10:37:16 PM
Can you make the change more compatible with the existing api please?  For example, the fulldepth url is documented in the wiki as being...

https://mtgox.com/api/1/BTCUSD/fulldepth

but I don't see that in the OP?

--D

https://mtgox.com/api/1/BTCUSD/fulldepth is actually an alias to https://mtgox.com/api/1/BTCUSD/depth/full


Title: Re: Data source url change (ticker, depth, history)
Post by: playtin on March 10, 2013, 04:07:21 AM
IIRC, the "old" mtgox.com (non-socket) API had some "query it 10 times and get blocked for 24h" policy.
Are those limits lifted for the new data.mtgox.com API?


Title: Re: Data source url change (ticker, depth, history)
Post by: Bitcoinpro on March 10, 2013, 05:03:27 AM
thanks for the info


Title: Re: Data source url change (ticker, depth, history)
Post by: MagicalTux on March 10, 2013, 07:01:49 AM
IIRC, the "old" mtgox.com (non-socket) API had some "query it 10 times and get blocked for 24h" policy.
Are those limits lifted for the new data.mtgox.com API?

The limits are lifted, however spamming will still end in your IP getting blocked (not by us, but by CloudFlare, using their own method to choose to block you).

Either way getting the same data again and again will not result in new data being returned, remember this.


Title: Re: Data source url change (ticker, depth, history)
Post by: prof7bit on March 10, 2013, 09:03:00 AM
Either way getting the same data again and again will not result in new data being returned, remember this.

There are other reasons for requesting the same API more often than only 5 times per hour, for example when developing client software and testing it. Your API should just be able to take such usage without a problem, after all its mostly only reading and not writing, so it should not put any noticeable stress on your servers.

A few suggestions: If you ever decide to make an API V2 then please consider the following:

 * remove all the redundant different representations of the same number (value, display, display_short), only use value_int. There is a lot of totally redundant data in these structures, if you remove it all it would probably cut down bandwidth by 2/3 and your server would not need to waste time with string formatting.
 * gzip all data
 * allow orders to carry a user defined ID number, so I can generate my own order-id *before* submitting it to mtgox. Then when requesting "private/orders" or inside the private:user_order message each order would have two IDs, my own and the official mtgox oid. Then I could use my own ID for all my internal bookkeeping in my own database during the entire lifetime of the order and use the mtgx id only for commands that I send to mtgox.
 * please provide historical OHLCV data of various time frames, they don't all need to have the entire history, just 100 bars each would be enough already, this would remove the need for many people (including me) to request huge amounts trading history if one just quickly want to plot a graph where for example each 15 minutes or 4 hours or an entire week of trading is consolidated into only 5 numbers.

And finally: Is there a thread or a section on this forum specifically for mtgox feature requests/problems/questions/etc?

PS: the websocket server is working again, did you fix it or does it go off and on on its own every Friday? Maybe its the cleaning lady who needs a wall socket for the vacuum cleaner every Friday, maybe you should just lock the door to that room to avoid these problems?


Title: Re: Data source url change (ticker, depth, history)
Post by: MagicalTux on March 11, 2013, 12:23:45 AM
In the last 24 hours, data.mtgox.com has served 143513 requests for a total of 25.9 GB bandwidth.

That's just a fraction of what's being served on mtgox.com, but that helps already quite a lot.

There are other reasons for requesting the same API more often than only 5 times per hour, for example when developing client software and testing it. Your API should just be able to take such usage without a problem, after all its mostly only reading and not writing, so it should not put any noticeable stress on your servers.

If that's for uses such as development, then it's fine. Some websites actually hit our ticker backend once for every page load on their side.


And finally: Is there a thread or a section on this forum specifically for mtgox feature requests/problems/questions/etc?

None, but your suggestions are welcome on info@mtgox.com. I've read what you posted here and will use it. I'm not sure about user-defined order ids, as it's going to be using a lot of storage on our side, and all data is already gzipped if supported by the user agent. We'll be checking your other suggestions.


Title: Re: Data source url change (ticker, depth, history)
Post by: prof7bit on March 11, 2013, 10:50:39 AM
I'm not sure about user-defined order ids, as it's going to be using a lot of storage on our side

It would really be a great help. Imagine a bot that needs to place orders at certain prices if certain conditions are met but under no circumstances may place the same order twice. Ideally it would look up in its own order list whether its orders are there already (sent but not yet acked or acked and pending or open) or whether they are still missing.

Currently this is not easy: One possibility at the moment would be for my bot to generate an ID and send it in the id field of the signed call and then wait for the op:result that returns this id *and* the official mtgox oid and then update my own database to link these two ids together. But unfortunately the op:result is the only message that will ever refer to this user-generated ID, its the only link between my ID and mtgox ID, if its ever lost for some reasons (temporary disconnect between order/add and op:result during busy times, etc) I cannot reconstruct it. did the order/add goo through? is it pending? is it missing? If it suddenly appears 20 minutes later how do I know where it came from? If I query the private/orders it will not contain these user-generated IDs, I cannot know which of the existing orders belong to which of my bots or belong to manual trading.

Ideally I would want to be able implement a send_order() function in my trading client that is non-blocking and immediately returns an (internal) ID that I can rely on that will survive disconnects, restarts, all kinds of errors, etc. and my cancel_order() function and other order functions would always be able to use this internal ID to refer to the order.

Currently the only reliable way to do it is to encode such additional information in the least significant bits of the order volume: Instead of buying 1 BTC my bot would buy 1.00000042 or <whatever_amount> + <some_small_ID_number>, so each of my bots can recognize their own orders and restore this important part of their state from a simple private/orders query. But I consider this a dirty hack. Its only a workaround.

Such an ID field would not have to be large, people would not need to be able to store their entire autobiography in this field, 32 bit unsigned int would be plenty enough for this purpose.


Title: Re: Data source url change (ticker, depth, history)
Post by: Seal on March 12, 2013, 12:39:46 AM
In the last 24 hours, data.mtgox.com has served 143513 requests for a total of 25.9 GB bandwidth.

That's just a fraction of what's being served on mtgox.com, but that helps already quite a lot.

Thanks MagicalTux. I've switched one of my feeds over. Is data.mtgox.com based on another server? What is the latency between data.mtgox.com and mtgox.com?

I also have a suggestion for cutting your bandwidth usage in half... A typical ticker request looks like this:

Code:
{"result":"success","data":{"high":{"value":"48.46900","value_int":"4846900","display":"$48.47","display_short":"$48.47","currency":"USD"},"low":{"value":"45.54000","value_int":"4554000","display":"$45.54","display_short":"$45.54","currency":"USD"},"avg":{"value":"47.57262","value_int":"4757262","display":"$47.57","display_short":"$47.57","currency":"USD"},"vwap":{"value":"47.63996","value_int":"4763996","display":"$47.64","display_short":"$47.64","currency":"USD"},"vol":{"value":"41608.55637760","value_int":"4160855637760","display":"41,608.56\u00a0BTC","display_short":"41,608.56\u00a0BTC","currency":"BTC"},"last_local":{"value":"48.36900","value_int":"4836900","display":"$48.37","display_short":"$48.37","currency":"USD"},"last":{"value":"48.36900","value_int":"4836900","display":"$48.37","display_short":"$48.37","currency":"USD"},"last_orig":{"value":"48.36900","value_int":"4836900","display":"$48.37","display_short":"$48.37","currency":"USD"},"last_all":{"value":"48.36900","value_int":"4836900","display":"$48.37","display_short":"$48.37","currency":"USD"},"buy":{"value":"48.03003","value_int":"4803003","display":"$48.03","display_short":"$48.03","currency":"USD"},"sell":{"value":"48.37000","value_int":"4837000","display":"$48.37","display_short":"$48.37","currency":"USD"},"now":"1363048242058242"}}

The most common request must be for an up to date ticker, however I'm willing to bet that the majority of apps and bots will only ever require Buy, Sell and last price.

high, low, avg, vwap, vol, last_orig and last_all will be redundant to a lot of sites.

Following on from prof7bit's suggestions about cutting the data responses down for redundant data, can you produce a second ticker feed with a more condensed dataset? This will speed up response times for sites requesting data too.

Alternatively as a suggestion is there a way to code it so url parameters decide what should be returned? Eg something like http://data.mtgox.com/api/2/BTCUSD/money/ticker/bid/ask/last


Title: Re: Data source url change (ticker, depth, history)
Post by: Puppet on March 20, 2013, 07:25:35 AM
Is this preferred over websockets? Since the ws server is down more often than its up, Im falling back to "hammering" your system with ticker requests over http. This seems contraproductive ?

Also Im confused about the depth api. How can I request depth changes since a given change or txid?
What depth messages does http://data.mtgox.com/api/2/BTCUSD/money/depth/fetch provide ?


Never mind, figured it out. I missed the  "filter_min_price" and  "filter_max_price" records, that explains. How often can we call this api?



Title: Re: Data source url change (ticker, depth, history)
Post by: genuise on March 20, 2013, 09:21:17 AM
In the last 24 hours, data.mtgox.com has served 143513 requests for a total of 25.9 GB bandwidth.

That's just a fraction of what's being served on mtgox.com, but that helps already quite a lot.

There are other reasons for requesting the same API more often than only 5 times per hour, for example when developing client software and testing it. Your API should just be able to take such usage without a problem, after all its mostly only reading and not writing, so it should not put any noticeable stress on your servers.

If that's for uses such as development, then it's fine. Some websites actually hit our ticker backend once for every page load on their side.


And finally: Is there a thread or a section on this forum specifically for mtgox feature requests/problems/questions/etc?

None, but your suggestions are welcome on info@mtgox.com. I've read what you posted here and will use it. I'm not sure about user-defined order ids, as it's going to be using a lot of storage on our side, and all data is already gzipped if supported by the user agent. We'll be checking your other suggestions.


Requesting the same API more than 5 times per hour can be in production mode too - in case when my engine supports more than 5 currencies orderbooks at the same time - when socket reconnects I will need to synchronize all supported orderbooks in the same moment. Then I request full orderbooks for every currency i support. And first test show the first 5 http requests work and the remaining requests get timout response. Thanks to the quite complext logic of managing timeouts for each currency orderbook my engine did not get blocked yet. But it would be good that you give your understanding of this situation and maybe suggest some possible solution in this case?


Title: Re: Data source url change (ticker, depth, history)
Post by: myself on March 21, 2013, 07:12:08 AM
http://data.mtgox.com/api/1/BTCUSD/ticker

Quote
Database access error, please retry later
:(


Title: Re: Data source url change (ticker, depth, history)
Post by: Kris on March 21, 2013, 04:25:36 PM
Don't know how I missed this thread.

Everything updated.

Will the ticker always stay at HTTP to prevent any overhead of HTTPS (SSL) ?

Thanks.


Title: Re: Data source url change (ticker, depth, history)
Post by: myself on March 21, 2013, 10:14:49 PM
http://data.mtgox.com/api/1/BTCUSD/ticker

Quote
Database access error, please retry later
:(

That's what we are trying to avoid by having people use data.mtgox.com. Guess there was a long enough ticker lag to get even data.mtgox.com to fail.
Dear god.


Title: Re: Data source url change (ticker, depth, history)
Post by: genuise on March 21, 2013, 10:22:16 PM
http://data.mtgox.com/api/1/BTCUSD/ticker

Quote
Database access error, please retry later
:(

That's what we are trying to avoid by having people use data.mtgox.com. Guess there was a long enough ticker lag to get even data.mtgox.com to fail.

Hi, You did not mention socket.io api url changes. Will it change too?

And if yes do you know about current socket.io behaviour? Client connects, connect event invoked, and then no any subscribtion message appear. Just silence.

Can you comment on this effect? What can be done?



Title: Re: Data source url change (ticker, depth, history)
Post by: MagicalTux on March 22, 2013, 11:05:37 PM
Hi, You did not mention socket.io api url changes. Will it change too?

And if yes do you know about current socket.io behaviour? Client connects, connect event invoked, and then no any subscribtion message appear. Just silence.

Can you comment on this effect? What can be done?

No change on the socketio url, however we are testing a new system that'll be much better to handle the load. Tests are currently being done.


Title: Re: Data source url change (ticker, depth, history)
Post by: snoleo on March 23, 2013, 06:00:37 AM
In the last 24 hours, data.mtgox.com has served 143513 requests for a total of 25.9 GB bandwidth.

That's just a fraction of what's being served on mtgox.com, but that helps already quite a lot.


Hi Tux,

I think the json data returned by data.mtgox.com could be compressed or reduced. Let's compare the returned json data of regular DepthTable of http V0 version and V2 version:

V0 version:

https://data.mtgox.com/api/0/data/getDepth.php

http://www.btc123.com/TmP/mtapi/v0_depth.png

v2 version

https://data.mtgox.com/api/2/BTCUSD/money/depth

http://www.btc123.com/TmP/mtapi/v2_depth.png

json data size compare:

http://www.btc123.com/TmP/mtapi/wget.png


From above , I suggest that Http v2 version json of depth use the format of http v0 which could reduce more than 90% data size. Or you could place a "switch" like

https://data.mtgox.com/api/2/BTCUSD/money/depth/reduced
or
https://data.mtgox.com/api/2/BTCUSD/money/depth?format=reduced

to let people get the proper data they want.

The website like bitcoin.clarkmoody.com  or info.btc123.com (btc123 caches json data itself for 10 seconds) is designed to maintain a realtime data of ticker , most recent trades and current depth. There could be a very frequent data getting requirement.

Also for ticker and trades:

ticker v0 version:
http://www.btc123.com/TmP/mtapi/v0_ticker.png

ticker v2 version:
http://www.btc123.com/TmP/mtapi/v2_ticker.png

trade v0 version:
http://www.btc123.com/TmP/mtapi/v0_trades.png

trade v2 version: (The great is that v2 version add trades property "market" "limit" to each transaction.)
http://www.btc123.com/TmP/mtapi/v2_trades.png

From above I think there is a lot of information could be removed from json data, maybe only the yellow background data is essential. And I suggest that the ticker and trades data format could take v0 version depth for reference, or add a switch (or say an option) to provide a version for those who just want to get realtime market information (like info.btc123.com) to get the reduced or compressed json data.

Many Thanks!


Title: Re: Data source url change (ticker, depth, history)
Post by: lucif on March 23, 2013, 09:16:00 PM
Ah there it is!

https://bitcointalk.org/index.php?topic=156833

Well, you suck. Why I should dig forums and newspapers what the fuck is going on with my apps?

Could you please inform about futher similar epic changes like that at least on YOUR FRONT PAGE NEWS? And at most in email newsletter?


Title: Re: Data source url change (ticker, depth, history)
Post by: zhalox on March 23, 2013, 11:54:43 PM
MagicalTux, please read the following posts on this thread, and learn from this highly unprofessional failure at Mt. Gox by improving your business practices.  An official apology from Mt. Gox regarding this serious failure over the past few days would help customers to regain confidence.  Hopefully you learn from this and make better business decisions in the future to avoid embarrassing situations like this for Mt. Gox: https://bitcointalk.org/index.php?topic=156833

Quote
I agree, they can't just post in some obscure forum about these sorts of things, they need to make an announcement on their website via email.  Communication is VERY important.  They should create an email list where you can subscribe to updates like this (press releases, announcements, etc.).  However, their failure to update their own products' APIs or at least provide a fail-safe (Mt. Gox mobile Android app, http://classic.mtgox.com) is COMPLETELY unacceptable.


Title: Re: Data source url change (ticker, depth, history)
Post by: franky1 on March 24, 2013, 08:28:35 AM
anyone wanting the easy PHP code to get a ticker running as trying to find an easy copy and paste on the wiki takes many pages of searching

Code:
<?
$json = file_get_contents('http://data.mtgox.com/api/2/BTCGBP/money/ticker');
$jdec = json_decode($json);

$check=$jdec->{'result'};

if ($check<>"success")
 {
     echo "ticker is offline";
 }
else
 {
     $rate = $jdec->{'data'}->{'high'}->{'value'};
     echo "£" . $rate;
 }
?>

all you have to do is change the BTCGBP in the url to your desired currency and ofcourse the £ in the echo code needs changing if your not looking for english pounds


Title: Re: Data source url change (ticker, depth, history)
Post by: bzzard on March 24, 2013, 01:04:11 PM
MagicalTux - please at least respond to snoleo post - APIv0 data format is actually great

I've talked also with MtGox support on this topic but they ignore this question.


Can you say in numbers - how much cache time is changed?


No change on the socketio url, however we are testing a new system that'll be much better to handle the load. Tests are currently being done.
Sounds great. So whats the name of new system? Or it's still socket.io?


Title: Re: Data source url change (ticker, depth, history)
Post by: snoleo on March 25, 2013, 02:22:44 AM
Hi bzzard,

Are you the author of btccharts.com ?

It is a very great flash real time depth chart and I like it.  :)


Title: Re: Data source url change (ticker, depth, history)
Post by: snoleo on March 25, 2013, 05:54:22 AM
And I also find that maybe MtGox has cached the json data, since whether I use php curl or Chrome browser directly , I cannot get the trades executed in last 10-15 seconds . Instead , I always get the trades executed 1-3 minutes ago. And users watching the realtime trades data complain about the delays. 2-4 minutes is just too long.


So I guess maybe MtGox has cached the trades data for 2-3 minutes. Tux , would you please give some specification on that?


Title: Re: Data source url change (ticker, depth, history)
Post by: MagicalTux on March 25, 2013, 09:23:26 AM
So I guess maybe MtGox has cached the trades data for 2-3 minutes. Tux , would you please give some specification on that?

Hi,

The v0 getTrades is cached ~120 seconds, even with ?since=. The v1/v2 api is cached 15 seconds only if ?since= is passed.

However if you're looking for realtime activity, you should be using the realtime feeds.


Title: Re: Data source url change (ticker, depth, history)
Post by: genuise on March 25, 2013, 09:07:20 PM
Hi, You did not mention socket.io api url changes. Will it change too?

And if yes do you know about current socket.io behaviour? Client connects, connect event invoked, and then no any subscribtion message appear. Just silence.

Can you comment on this effect? What can be done?

No change on the socketio url, however we are testing a new system that'll be much better to handle the load. Tests are currently being done.

Hi, can you be so kind to tell more details about newsystem. In another thread they mentioned socketio-beta.mtgox.com I tried to connect there with my nodejs socket.io-client lib and got only multiple errors.

while the same lib seems compatible with the old socketio.mtgox.com but it is highly highly unstable. Very often  client connects (get connect event) and then nothing more just silence no any subscribtion mesages. Also I connect with such url https://socketio.mtgox.com/mtgox?Currency=USD,EUR,GBP,CNY,RUB,CAD,PLN,AUD which contains 8 currencies. Could this be an issue in my case?

please advice.


Title: Re: Data source url change (ticker, depth, history)
Post by: beckspace on March 25, 2013, 09:22:25 PM
https://classic.mtgox.com/trade/history not showing info anymore!  :-\


Title: Re: Data source url change (ticker, depth, history)
Post by: snoleo on March 26, 2013, 01:42:02 AM

The v0 getTrades is cached ~120 seconds, even with ?since=. The v1/v2 api is cached 15 seconds only if ?since= is passed.

However if you're looking for realtime activity, you should be using the realtime feeds.

Hi Tux,

1. About the cache time, does getDepth and ticker cached the same time as getTrades in v0?
2. What is the cache time of depth and trades in v1 and v2?
3. Would you please provide a v1/v2 api returned reduced data to save the bandwidth?
4. And if there is an api to provide the most recent 100 or 60 trades?

Sorry for so many questions. Please kindly give some specifications, thanks!


Title: Re: Data source url change (ticker, depth, history)
Post by: cilphex on March 27, 2013, 07:10:11 AM
Hi Tux, I'm new to the API.  Can you tell me what the since= param means?  I imagine it means "trades since this time" or "trades since this far back", but I don't know what unit it's measuring in.  Minutes?  Hours?  Days?


Title: Re: Data source url change (ticker, depth, history)
Post by: snoleo on March 27, 2013, 11:49:24 AM
Hi Tux, I'm new to the API.  Can you tell me what the since= param means?  I imagine it means "trades since this time" or "trades since this far back", but I don't know what unit it's measuring in.  Minutes?  Hours?  Days?

since=N

It will get trades which were execuated after that trades ID N (excludes ID N)

PS: Tux seems too busy to answer our questions , so we have to try to help each other. But who will be kind to come answer my question?  ???


Title: Re: Data source url change (ticker, depth, history)
Post by: 2112 on March 27, 2013, 12:51:31 PM
But who will be kind to come answer my question?  ???
The answer is that they are trying to provide support for the FIX protocol http://en.wikipedia.org/wiki/Financial_Information_eXchange .
Please learn it or be left it the dustbin. There is no way forward that continues to use the old RPC polling paradigm.

Yep, see the top of the thread. The APIs will stay the same at launch. We will probably layer on FIX support for CoinLab customers.


Title: Re: Data source url change (ticker, depth, history)
Post by: phelix on March 27, 2013, 05:41:59 PM
as snoleo pointed out already:

why did you remove the old ticker and bloat it even more? it was too much already.


and the depth... oh my.


Title: Re: Data source url change (ticker, depth, history)
Post by: phelix on March 27, 2013, 07:04:11 PM
Because of the hickups (price jumping randomly +-30%) in the old ticker I switched to the new one today.

Only now do I realize how messed up it really is.


For some reason the standard python json decoder decodes the numbers as unicode strings and not as numbers....

Probably these encoded chars are the problem:
vol":{"value":"72063.77361237","value_int":"7206377361237","display":"72,063.77\u00a0BTC","display_short":"72,063.77\u00a0BTC"

Took me a while to get aware of that, watch out.


edit: Now that was BS, the reason is the quotes around the numbers of course. why did you add quotes to the values?



Title: Re: Data source url change (ticker, depth, history)
Post by: cilphex on March 27, 2013, 11:37:13 PM
2112, thanks for your help with that.

Anybody know if there's a way to specify that you want data since a certain date?  Like if I wanted the past 24 hours of trades, is there a way to do that? 


Title: Re: Data source url change (ticker, depth, history)
Post by: shsmith on March 28, 2013, 02:41:21 AM
2112, thanks for your help with that.

Anybody know if there's a way to specify that you want data since a certain date?  Like if I wanted the past 24 hours of trades, is there a way to do that? 
since is in time units*1000000
to start x hours ago you can calculate
   since = int(time() - x*60*60)*1000000
   url = "http://data.mtgox.com/api/1/BTCUSD/trades?since=" + str(since)

You will get batches of 100 trades, so you need to keep calling
with since set to highest tid in previous batch



Title: Re: Data source url change (ticker, depth, history)
Post by: cilphex on March 28, 2013, 04:36:20 AM
2112, thanks for your help with that.

Anybody know if there's a way to specify that you want data since a certain date?  Like if I wanted the past 24 hours of trades, is there a way to do that? 
since is in time units*1000000
to start x hours ago you can calculate
   since = int(time() - x*60*60)*1000000
   url = "http://data.mtgox.com/api/1/BTCUSD/trades?since=" + str(since)

You will get batches of 100 trades, so you need to keep calling
with since set to highest tid in previous batch

But it was stated earlier that the "since" parameter is for a trade ID, not a timestamp, so I don't think that would work.


Title: Re: Data source url change (ticker, depth, history)
Post by: shsmith on March 28, 2013, 01:07:25 PM
But it was stated earlier that the "since" parameter is for a trade ID, not a timestamp, so I don't think that would work.
It does work, at least for me.  It appears they calculate trade id based on time stamp.  In every case I have examined, the trade id was the same as trade time, but with 6 extra digits.


Title: Re: Data source url change (ticker, depth, history)
Post by: phelix on March 28, 2013, 03:39:29 PM
Quote
MtGox: 314.5 USD/BTC

The much I like that number...  is the ticker jumping for me only or for everybody?


Quote
{"saveTime": 1364484370.9484749, "warning": "THIS API IS DEPRECATED. Please upgrade to newer API such as https://data.mtgox.com/api/2/BTCUSD/money/ticker", "ticker": {"sell": 317, "buy": 314.5, "last": 314.5, "vol": 979, "last_local": 314.5, "last_all": 310.49079, "vwap": 299.89697478400001, "high": 316.91019, "low": 279.89999999999998, "avg": 301.46424582200001}}
???


Title: Re: Data source url change (ticker, depth, history)
Post by: snoleo on March 30, 2013, 01:28:14 PM

The v0 getTrades is cached ~120 seconds, even with ?since=. The v1/v2 api is cached 15 seconds only if ?since= is passed.


Hi Tux,

What about the cache time of v1/v2 api of Ticker and Depth?


Title: Re: Data source url change (ticker, depth, history)
Post by: myself on March 30, 2013, 02:24:04 PM
Magicaltux yesterday feeds from mtgox.com were timeouting since bitfinex.com is trading quite allot on your books can plz fix your stuff or provide a dedicated API


Title: Re: Data source url change (ticker, depth, history)
Post by: myself on April 25, 2013, 10:08:11 PM
@MagicalTux can you explain whit your ticker send a last price at 112 when the best bid was 140 ??