[BETA] MTGox websocket API, testers wanted

[daybyter]

Java? I wrote some code for the HTTP API. Maybe we could collaborate for the socket stuff?

[molecular]

- Consider making your SocketIO classes more generic and creating an MTGox sub-class which inherits from it, passing in event handlers (on_connect, on_msg, etc). This will make things easier if and when other exchanges add streaming APIs.

Consider this considered. However, when thinking about integrating other exchanges, there is a lot more work on other parts of my code to be done. I've been wanting to do this for a while now but decided if I would do it, I'd reimplement in another language that better suits my experience.

What languages do you recommend for best integration with SocketIO and streaming data?

I've been looking into scripting pages with PHP but it seems a little tough to integrate as its a language which generates pages on call.

Also I'd like to write a server-side script to run at home which will accept a socketio feed and place trades automatically. I currently do this again with the json calls API however because its written in PHP, it requires a cron call every x minutes. I'm looking for a better solution.

java, node.js, write timer loop yourself in php or use some php scheduling stuff? I don't know. For the kind of stuff we're looking at here, there's good APIs in every language, so it really boils down to personal language preference, I guess. I'd probably use java because I'm most comfortable with it. Unfortunately I cannot recommend my "traidor" code, because the orderbook still gets out of sync.

One remark: I don't know about the stability of the socket.io feed nowadays, but maybe you should implement a fallback to http api?

[Seal]

java, node.js, write timer loop yourself in php or use some php scheduling stuff? I don't know. For the kind of stuff we're looking at here, there's good APIs in every language, so it really boils down to personal language preference, I guess. I'd probably use java because I'm most comfortable with it. Unfortunately I cannot recommend my "traidor" code, because the orderbook still gets out of sync.

One remark: I don't know about the stability of the socket.io feed nowadays, but maybe you should implement a fallback to http api?

molecular - I currently use a scheduler to execute a PHP call every minute using the http api, however I'm interested in having a live feed for the best prices as they are added to the orderbook. To my knowledge, PHP isnt brilliant at handling streaming feeds and I'm yet to find a way of doing this server-side.

I could stand corrected though, I'm just waiting for a script-kiddy to tell me otherwise.

Whats java like at handling streaming data? How would you run a script like that on a server?

[daybyter]

I would simply run it as a daemon on the server. It would poll every x seconds/minutes . I'll drop you a mail with more details and some code (cannot attach anything here).

[molecular]

java, node.js, write timer loop yourself in php or use some php scheduling stuff? I don't know. For the kind of stuff we're looking at here, there's good APIs in every language, so it really boils down to personal language preference, I guess. I'd probably use java because I'm most comfortable with it. Unfortunately I cannot recommend my "traidor" code, because the orderbook still gets out of sync.

One remark: I don't know about the stability of the socket.io feed nowadays, but maybe you should implement a fallback to http api?

molecular - I currently use a scheduler to execute a PHP call every minute using the http api, however I'm interested in having a live feed for the best prices as they are added to the orderbook. To my knowledge, PHP isnt brilliant at handling streaming feeds and I'm yet to find a way of doing this server-side.

I could stand corrected though, I'm just waiting for a script-kiddy to tell me otherwise.

Whats java like at handling streaming data?

Are you talking about streaming from your server to some clients or about receiving mtgox socket.io/websocket stream?

If the latter: http://stackoverflow.com/questions/5783086/java-socket-io-client
If the former: http://code.google.com/p/socketio-java/

I never tried any of these, so I don't know how well they handle things.

How would you run a script like that on a server?

If it's also serving web-pages I'd probably run it in the context of a servlet container like tomcat or jetty.

If not, I'd probably write an init-script that just runs "java -jar myjar.jar" and do scheduling within the app. To be honest, I might just start it in a screen (gnu screen) session if I'm the only one depending on it

[daybyter]

Perfect. We are thinking exactly in the same lines.

I started to write the Java app in a way, that I could run in client or server mode. In client mode, a GUI comes up. In server mode, there's no GUI, so it could be started in an init.d script.

Drop me a PM, if you want to discuss more details.

[molecular]

Perfect. We are thinking exactly in the same lines.

I started to write the Java app in a way, that I could run in client or server mode. In client mode, a GUI comes up. In server mode, there's no GUI, so it could be started in an init.d script.

Drop me a PM, if you want to discuss more details.

In case you're talking to me: maybe better talk to Seal, I was merely giving ideas to Seal and have currently nothing of the sort planned (except maybe (but unlikely (unless someone coughs up coins)) at some point trading/bot infrastructure and exchange abstraction layer in java)

[daybyter]

Scary....maybe you are my long lost twin brother, or so...

I already send a PM to Seal and waiting for some reply with an email, so I can send him some code...

Part of my code is a TradeSite interface (just started), that I already implement for MtGox and BtcE. So you can loop over the sites, combine all trades, compare fees etc.

[Seal]

I'm here! Yes, I'll reply to your PM now daybyter.

Thanks for the advice molecular. I've been developing trading scripts for a while, mostly in PHP, however I've got to admit, socket.io just confuses me.

It most likely won't be serving client pages. However the script I'd like to write will be based on a server and receiving the mtgox stream, whereby it could then decide on trades itself.

If not, I'd probably write an init-script that just runs "java -jar myjar.jar" and do scheduling within the app. To be honest, I might just start it in a screen (gnu screen) session if I'm the only one depending on it

I dont think my current host offers terminal sessions so ideally it'd run in a standard LAMP build environment.

... either that or I'll have to purchase some sort of virtual box.

[molecular]

I dont think my current host offers terminal sessions so ideally it'd run in a standard LAMP build environment.

... either that or I'll have to purchase some sort of virtual box.

oh, ok. hmm. I'm very content with one of these which I pay in BTC: http://www.cinfu.com/vps/

[molecular]

Scary....maybe you are my long lost twin brother, or so...

I already send a PM to Seal and waiting for some reply with an email, so I can send him some code...

Part of my code is a TradeSite interface (just started), that I already implement for MtGox and BtcE. So you can loop over the sites, combine all trades, compare fees etc.

be sure to tell this thread (or me) when you have some shareable code. I'm interested

[runeks]

Is the websocket API still working for everyone else? It worked for me fine until today. Now this is the first message I receive after connecting:

Code:

{'suggest': 'retry', 'error': 'command failed', 'op': 'error'}

This is when I connect to https://socketio.mtgox.com/socket.io as I've always been doing. When I connect to https://socketio.mtgox.com/mtgox the mtgox_io script fails because it can't read the heartbeat interval from the first piece of data received when connecting to the server.
The data received in mtgox_io's connect() function at the line "r = response.read().split(':')" from https://socketio.mtgox.com/socket.io looks like this:

Code:

['9861138011216185358', '60', '60', 'websocket,htmlfile,xhr-polling,jsonp-polling']

While, when connecting to https://socketio.mtgox.com/mtgox, it's:

Code:

['Welcome to Tibanne Websocket']

A welcoming the script doesn't seem to appreciate.

Is it not working for anyone else, or is it just my corner of the internet that's broken?

[Clark]

Is it not working for anyone else, or is it just my corner of the internet that's broken?

Take a glance at the console on the bottom of bitcoin.clarkmoody.com, and you will see the same thing. Streaming data has been disabled (by Gox) for most of the afternoon on my website.

So you're not alone.

[runeks]

I'm seeing:

Code:

01:21:39 > Got 3864 asks and 5085 bids.

01:21:38 > Synchronizing the order book.

01:21:16 > Subscribed to depth

01:21:16 > Subscribed to ticker

01:21:16 > Subscribed to trades

01:21:16 > MtGox: Connected.

01:21:16 > Attempting connection over websocket

Looks to me like it's working. What am I missing?

Has Mt. Gox gone public about why it's closed down, as you say?

EDIT: It's working for your site because it seems to be up now: works for me too! Yay

Oh live feed, how I've missed you.

[1455]

I tried for a few hours and crawled the forum for a solution but I'm still not able to fetch websocket depth for EUR.

Does anyone know if it is possible to receive market depth updates for other currencies (EUR) in the socket.io/websocket API?
I found this list of channels: https://en.bitcoin.it/wiki/Talk:MtGox/API but couldn't subscribe using the API.
The API only takes a "type" argument (Possible values: "trades, ticker, depth") e.g. {"op":"mtgox.subscribe","type":"trades"}
but there is no documented value for "depth.EUR".

You can subscribe with ?Channel=EUR added to the subscribe URL. I have a working EUR order book, chart, and time & sales on my website. Simply choose EUR from the drop down menu at the top: bitcoin.clarkmoody.com

It works fine for USD with (Ruby-code):

Code:

 s = SocketIO.connect("https://socketio.mtgox.com/mtgox", {:sync => true, :reconnect => :true}) do
  before_start do
    puts "here we are"
    on_json_message do |m|
      m = JSON.parse m
      q << m["depth"] if m["op"] == "private" and m["private"] == "depth"
    end
    on_connect do
      send_message({"op" => "mtgox.subscribe", "type" => "depth"}.to_json)
    end
  end
end

but I wasn't able to get EUR depth with the mentioned solutions.

These are not working:

Code:

s = SocketIO.connect("https://socketio.mtgox.com/mtgox?Currency=EUR", {:sync => true, :reconnect => :true})

Code:

s = SocketIO.connect("https://socketio.mtgox.com/socket.io/1?Currency=EUR", {:sync => true, :reconnect => :true})

Could someone please help me with that? Thanks alot!

[whitslack]

Is it just me, or have the event feeds come completely unglued today?

EDIT: Appears to be working again.

[genuise]

Hi,

my task is to use one socket-io client and subscribe to multiple currencies depth channels.

When I connect with url https://socketio.mtgox.com/mtgox
then I automatically is subscribed to depth, tickers and trades channels.

Depth channel is only for USD messages.

When I connect with https://socketio.mtgox.com/mtgox?Currency=EUR, then automatic subscribtion is done for EUR depth channel

Code:

057bdc6b-9f9c-44e4-bc1a-363e4443ce87

then at the same time I managed also to subscribe mannualy to USD channel using command

Code:

{
	"op":"mtgox.subscribe", 
	"type":"depth"
}

so currently I have both EUR and USD messages comming at the same time.

The question is how can I subscribe to more than two currency depth channels?

My current solution does look more like a hack and I cannot understand how to subscriibe for third currecny GBP and fourth CNY
in order to have mdepth messages for 4 currencies.

please advice

[alpet]

Hi!

I'm programmer with expirience making trading software for russian market (RTS/FORTS), and want to offer a few improvements to current MtGox API.
1. Optional filtering for depth messages, by price range and minimal volume, aggregating depth messages by price rounding to 0.001 BTC  (this help to significant reduce traffic).
2. Broadcating with depth and ticker messages some additional fields:  ticker.last*.timestamp, depth.timestamp  as double in milliseconds from midnight GMT.

Now I adopts a program TradeStudio to work with the MtGox:

[genuise]

Hi,

my task is to use one socket-io client and subscribe to multiple currencies depth channels.

When I connect with url https://socketio.mtgox.com/mtgox
then I automatically is subscribed to depth, tickers and trades channels.

Depth channel is only for USD messages.

When I connect with https://socketio.mtgox.com/mtgox?Currency=EUR, then automatic subscribtion is done for EUR depth channel

Code:

057bdc6b-9f9c-44e4-bc1a-363e4443ce87

then at the same time I managed also to subscribe mannualy to USD channel using command

Code:

{
	"op":"mtgox.subscribe", 
	"type":"depth"
}

so currently I have both EUR and USD messages comming at the same time.

The question is how can I subscribe to more than two currency depth channels?

My current solution does look more like a hack and I cannot understand how to subscriibe for third currecny GBP and fourth CNY
in order to have mdepth messages for 4 currencies.

please advice

for those who are interested there is  a possibility to pass several currencies in the url parameter to subscribe for multiple currencies channels

Code:

https://socketio.mtgox.com/mtgox?Currency=EUR,USD,CNY,EUR

[prof7bit]

Problems I have found:

* sometimes a signed call like for example "private/info" or "private/orders" will fail for no apparent reason. At the beginning after connection succeeded and I subscribed to all channels (depth, trade, ticker) I send 3 signed calls ("private/idkey", "private/orders" and "private/info"), most of the time this works perfectly but sometimes one or two of them will be answered with "invalid call". In an attempt to work around this I just send them again when this error occurs, as often a needed until it finally succeeds. I don't know if I am missing something here (the API is poorly documented), I suspect a bug on the server side. "invalid call" just is not the correct answer when the server is busy or whatever might be causing this.

* I have not found any way to request fulldepth through the websocket API, again there is documentation missing, is this possible at all?

* I have managed to send trading commands like order/add and order/cancel through the streaming API, most of the time it works but sometimes they just go missing, maybe similar to the other calls mentioned above. This shoud not happen, there should be a queue on the server and a valid order should always go through. The HTTP API seems more reliable for trading but it would be nice if I could use the streaming API for that also.

* It would be nice if there were a way to request OHLCV data of certain timeframes from the server. I can request the trade history and build them myself but this is not practical if I want for example a daily chart of the last 365 days, this wold be a mega-download, possibly hitting the rate limit and banning me. Currently I have no idea where to get OHLCV data of arbitrary timeframes from, please implement this, it will save you a lot of bandwidth.

* The connection is lost very often, sometimes its even impossible to reconnect for periods of many minutes, please fix this. As a first step to remove load from your servers I would suggest completely abandoning this totally unnecessary Socket.io monstrosity that serves no other purpose than demonstrating that some crazy programmer somewhere was able to build the Frankenstein-Monster equivalent of network-protocols. Please drop this nonsense and revert back to good old plain websockets.

* Please improve the documentation. There are a lot of things that can only be found out by experimenting, I would have edited the wiki myself and added lots of useful information that I collected through experimenting and reverse engineering other snippets of code but the Bitcoin-Wiki authorities don't give me write access and I am NOT willing to pay money for being allowed to work on the wiki, the admins seem to have totally lost the connection to reality.

Maybe a multi-million dollar company like MtGox should be able to publish an *official* API documentation on the *official* MtGox website and not (ab)use some 3rd party wiki for it. Also I would expect a multi-million dollar company like MtGox to be able to afford servers that won't crash or freeze every 10 Minutes.


***

For those who are looking for yet another example: Here is my python client implementation: https://github.com/prof7bit/goxtool  it implements both protocols: plain old websocket and the bizarre socket.io-monstrosity that is always lagging behind the websocket by at least 5 seconds (you can configure which one to use)