[BETA] MTGox websocket API, testers wanted

[prof7bit]

2. My websocket connection keeps dropping, for no reason that I can see. This thread's history has some talk about having to send heartbeat messages,

websocket should never disconnect at all, I have it running here 24/7 and it disconnects only maybe once or twice a day, some days it doesn't disconnect at all (and probably my few disconnects here are only due to my very poor wifi connection).

[Floomi]

Well, it seems that the diagnosis for at least one of these two issues is "I'm an idiot" I've edited my post above with an update.

[prof7bit]

1. Once the connection is established I subscribe to the USD trades channel:

Code:

TRADES = "dbf1dee9-4f2e-4a08-8cb7-748919a71b21"
def subscribe( channel ):
	output = { 'op': 'mtgox.subscribe', 'channel': channel }
	return json.dumps( output )

ws.send(subscribe(TRADES))

...but I get the remark spat back "Unknown channel requested or not a public channel".

Try "mtgox.subscribe" instead of "subscribe" and/or also try "type" instead of "channel" and/or also try using the channel name instead of the GUID.

The documentation seems incomplete or outdated, you need to experiment a bit.

For example this works for me: {"op":"mtgox.subscribe", "type":"trades"}
And also: {"op":"mtgox.subscribe", "channel":"ticker.BTCUSD"}

And for subscribing to the private channel (after receiving my idkey) I'm using the following:
{"op":"mtgox.subscribe", "key":idkey}

[Floomi]

That works for me, thanks! It's really helpful that the documentation is full of lies 

[Floomi]

Thread necro! Is there a limit on the number of times you can call private/idkey? My bot asks for it at startup, which is many times a day during development, and towards the end of the day (perhaps past midnight, if that has anything to do with it?), I start getting "invalid call" back instead of the key.

It's trivial enough for me to store the idkey and the time it was requested, but I wanted to know if anyone else has seen this behaviour, or if it's just Broken On My Machine?

Ta.

[Floomi]

And another one (after this I'll just start shouting Bueller?): I don't seem to be able to subscribe to both depth and trade messages. If I do, I get depth but not trade. I can repro this in <50 lines of python; is this normal behaviour?

[yucca]

And another one (after this I'll just start shouting Bueller?): I don't seem to be able to subscribe to both depth and trade messages. If I do, I get depth but not trade. I can repro this in <50 lines of python; is this normal behaviour?

Normal for us two maybe...

Im trying to get websocket stream working (coded on top of vanilla C socket) and I seem to have same problem?

I am getting lag, depth and ticker but no trades.

One error message comes back, related to the failing trade subscription.

Code:

!!!THANKS TO PROF7BIT FOR SUBCRIPTION INFO IN EARLIER POST!!!

04:25:30 MtGox_Socket::SocketCreate: CONNECTED TO: websocket.mtgox.com:80
04:25:31 MtGox_Socket::DoHandshake_WebSocket: GOOD: NowListening
04:25:31 TX_STR_FRAMED -> {"op": "mtgox.subscribe", "channel": "trade.lag"}
04:25:31 TX_STR_FRAMED -> {"op": "mtgox.subscribe", "channel": "ticker.BTCUSD"}
04:25:31 TX_STR_FRAMED -> {"op": "mtgox.subscribe", "channel": "depth.BTCUSD"}
04:25:31 TX_STR_FRAMED -> {"op": "mtgox.subscribe", "channel": "trades.BTCUSD"}

04:25:33 
{
    "message":"Unknown channel requested or not a public channel",
    "op":"remark",
    "success":false
}
.....
Now lag, depth and ticker JSON comes streaming in
.....

[Clark]

Subscribe to trade.BTC 

c.f. http://data.mtgox.com/api/2/stream/list_public

[genuise]

wait a second, now you have to subscribe on your own?
They really changed the default behavior when on connect you were subscribed automatically?

[yucca]

Subscribe to trade.BTC 

c.f. http://data.mtgox.com/api/2/stream/list_public

Awesomeo, thx Clark!

[yucca]

Well Socket.io and Websocket have been down for a while, for all of us it seems.

Except for this guy, older API version or something? Or maybe just lots of HTTP requests?:

http://bitcointicker.co/

EDIT: curiosity got the better of me, wireshark caught this:

GET /socket.io/1/websocket/qRpvu5AQFZvju4nIao6m HTTP/1.1

[yucca]

wait a second, now you have to subscribe on your own?
They really changed the default behavior when on connect you were subscribed automatically?

Yes, it changed for websocket some time ago, or maybe websocket has always been like that I don't know.

And I just noticed that socket.io now also serves a dead stream to start with, you need to subscribe (i think/hope).

For socket.io I was passing all needed currencies as args in initial GET, that doesn't work now.

[yucca]

OK, I got websocket working in my client again.

I'm implementing the websocket protocol myself in C++ on a raw socket so this is probably not relevant to those using javascript in browsers.

It turns out that the MtGox websocket now requires properly framed messages as per RFC6455.

A while ago only the Socket.IO required the RFC6455 framing.

Before I was getting away with just this minimal framing for websocket: [0x00] ....RAW STRING DATA... [0xFF]

Now you need the RFC6455 framing function, I posted C code earlier in this thread.

Now I need to get my socket.io implementation working also

WebSocket + JSON == WASTED_BANDWIDTH

[Fireball]

WebSocket + JSON == WASTED_BANDWIDTH

I plan to investigate the option of using WebSocket + BJSON to prevent bandwidth wasting in ICBIT. We are fine as it is so far, but it's always good to plan for future.

[Clark]

WebSocket + JSON == WASTED_BANDWIDTH

I plan to investigate the option of using WebSocket + BJSON to prevent bandwidth wasting in ICBIT. We are fine as it is so far, but it's always good to plan for future.

Have you had a look at MessagePack? http://msgpack.org/

[savis]

I was using modified C# http://socketio4net.codeplex.com/ library to consume depths updates from https://socketio.mtgox.com/mtgox. It was working fine till the biggining of this month. Now it may work for several hours, then just drop and stop accepting connections (Error: The connection was closed unexpectedly.).

So I tried to switch to https://github.com/sta/websocket-sharp and connect to wss://websocket.mtgox.com/mtgox. The library seems to be recommended on https://en.bitcoin.it/wiki/MtGox/API/Streaming. First I spent several hours before I found I need to specify Origin. Now it works fine expect that server sends close frame every 5-60 seconds.

Any idea what could be the reason? Thanks.

Code:

01:49:02.874|DEBUG|26 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"132.13605","type":2,"type_str":"bid","volume":"-0.10083051","price_int":"13213605","volume_int":"-10083051","item":"BTC","currency":"USD","now":"1379713750305094","total_volume_int":"0"}}
01:49:02.960|DEBUG|31 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"123.90414","type":2,"type_str":"bid","volume":"0.60778582","price_int":"12390414","volume_int":"60778582","item":"BTC","currency":"USD","now":"1379713750414105","total_volume_int":"60778582"}}
01:49:04.186|DEBUG|31 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"139.39536","type":1,"type_str":"ask","volume":"1.103871","price_int":"13939536","volume_int":"110387100","item":"BTC","currency":"USD","now":"1379713751538095","total_volume_int":"110387100"}}
01:49:07.415|DEBUG|8  |MtGoxApi: Close Event raised
01:49:08.622|DEBUG|31 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"135.986","type":1,"type_str":"ask","volume":"5.196","price_int":"13598600","volume_int":"519600000","item":"BTC","currency":"USD","now":"1379713756075743","total_volume_int":"519600000"}}
01:49:08.661|DEBUG|30 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"133.85221","type":2,"type_str":"bid","volume":"-3.855","price_int":"13385221","volume_int":"-385500000","item":"BTC","currency":"USD","now":"1379713756090538","total_volume_int":"0"}}
01:49:08.676|DEBUG|31 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"134.00225","type":2,"type_str":"bid","volume":"-10.8","price_int":"13400225","volume_int":"-1080000000","item":"BTC","currency":"USD","now":"1379713756105880","total_volume_int":"540000000"}}
01:49:09.511|DEBUG|30 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"134.00225","type":2,"type_str":"bid","volume":"-5.4","price_int":"13400225","volume_int":"-540000000","item":"BTC","currency":"USD","now":"1379713756934842","total_volume_int":"0"}}
01:49:09.571|DEBUG|31 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"131.41101","type":2,"type_str":"bid","volume":"-5.4","price_int":"13141101","volume_int":"-540000000","item":"BTC","currency":"USD","now":"1379713756950113","total_volume_int":"1080000000"}}
01:49:09.607|DEBUG|30 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"133.27742","type":2,"type_str":"bid","volume":"5.4","price_int":"13327742","volume_int":"540000000","item":"BTC","currency":"USD","now":"1379713757041934","total_volume_int":"1620000000"}}
01:49:13.229|DEBUG|8  |MtGoxApi: Close Event raised
01:49:14.963|DEBUG|30 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"129.6864","type":2,"type_str":"bid","volume":"0.25","price_int":"12968640","volume_int":"25000000","item":"BTC","currency":"USD","now":"1379713762412059","total_volume_int":"125000000"}}
01:49:15.040|DEBUG|31 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"134.00226","type":2,"type_str":"bid","volume":"-10.34395506","price_int":"13400226","volume_int":"-1034395506","item":"BTC","currency":"USD","now":"1379713762427277","total_volume_int":"0"}}
01:49:15.101|DEBUG|30 |MtGoxApi: {"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","channel_name":"depth.BTCUSD","op":"private","origin":"broadcast","private":"depth","depth":{"price":"134.00228","type":2,"type_str":"bid","volume":"10.34331649","price_int":"13400228","volume_int":"1034331649","item":"BTC","currency":"USD","now":"1379713762550896","total_volume_int":"1034331649"}}

[alikim]

Does anyone know how to subscribe to "own" private channel?

I start with this message:

{"call":"idkey","id":".....","nonce":...}

Successfully get the key:

{"id":"....","op":"result","result":"GhXJXwx..."}

Then subscribe:

{"op":"mtgox.subscribe","key":"GhXJXwx..."}

... and I don't get any response to the subscribe message and after sending orders or their execution I don't get any messages in the private "own" channel.

What am I doing wrong?

Thank you!

[Clark]

Then subscribe:

{"op":"mtgox.subscribe","key":"GhXJXwx..."}

... and I don't get any response to the subscribe message and after sending orders or their execution I don't get any messages in the private "own" channel.

What am I doing wrong?

Thank you!

Mt. Gox has been sending a message that my key is not valid. Are you seeing a "remark" message coming in after you attempt to subscribe?

My guess: Either the API has changed, or the private-channel server is down.

[alikim]

Then subscribe:

{"op":"mtgox.subscribe","key":"GhXJXwx..."}

... and I don't get any response to the subscribe message and after sending orders or their execution I don't get any messages in the private "own" channel.

What am I doing wrong?

Thank you!

Mt. Gox has been sending a message that my key is not valid. Are you seeing a "remark" message coming in after you attempt to subscribe?

My guess: Either the API has changed, or the private-channel server is down.

Yes, if I artificially add another symbol at the end of the key in subscribe call, I get that message that the key is not valid.

If I subscribe with the key as it is, I get no message, no error neither success.

Do I understand it right that you do not sign the subscribe message?

[Clark]

Yes, if I artificially add another symbol at the end of the key in subscribe call, I get that message that the key is not valid.

If I subscribe with the key as it is, I get no message, no error neither success.

Do I understand it right that you do not sign the subscribe message?

Interesting. I am sending the key as-is and getting an 'invalid key' message.

I am not signing the message, simply sending under mtgox.subscribe.