[BETA] MTGox websocket API, testers wanted

[slush]

Same status here, lack of any documentation is pretty irritating.

[1714009686]

1714009686

[1714009686]

1714009686

[1714009686]

1714009686

[1714009686]

1714009686

[molecular]

Same status here, lack of any documentation is pretty irritating.

magicaltux has some php sample code that was helpfull for me to code socket.io handshake in python.

(it's still got some debugging output and heartbeat lost detection is not implemented, so on connection drop it doesn't auto-reconnect). oh, and it only does websocket, no other transport.

from threading import *
import urllib2, urllib
import simplejson as json
import ssl, socket
import time
from websocket import WebSocket


class SocketIO:
  def __init__(S, url, callback):
    S.url = url
    S.callback = callback
    
  def connect(S):
    data = urllib.urlencode({})
    req = urllib2.Request('https://' + S.url + "/1", data)
    response = urllib2.urlopen(req)
    r = response.read().split(':')
    S.heartbeat_interval = int(r[1])
    print 'heartbeat: ', S.heartbeat_interval
    if 'websocket' in r[3].split(','):
      print "good: transport 'websocket' supported by socket.io server ", S.url
      S.id = r[0]
      print "id: ", S.id

    S.thread = Thread(target = S.thread_func)
    S.thread.setDaemon(True)
    S.thread.start()

  def stop(S):
    S.run = False
    S.thread.join(timeout=1)
    S.keepalive_thread.join(timeout=1)


  def thread_func(S):
    print 'SocketIO: websocket thread started'
    
    my_url = 'wss://' + S.url + "/1/websocket/" + S.id
    
    S.ws = WebSocket(my_url, version=0)
    S.run = True
    S.ws.send('1::/mtgox')

    # start keepalive thread
    S.keepalive_thread = Thread(target = S.keepalive_func)
    S.keepalive_thread.setDaemon(True)
    S.keepalive_thread.start()
    
    msg = S.ws.recv()
    while msg is not None and S.run:
      #print 'SocketIO msg: ', msg
      if msg[:10] == "4::/mtgox:":
        S.callback(msg[10:])
      #elif msg[:3] == "2::":
      #  True
      #else:
      #  print "SocketIO: dont know how to handle msg: ", msg
      msg = S.ws.recv()
      
  def keepalive_func(S):
    while S.run:
      time.sleep(S.heartbeat_interval)
      S.ws.send('2::');
      
def test_callback(msg):
  print 'msg: ', msg

# testcase
if True:
  sio = SocketIO('socketio.mtgox.com/socket.io', test_callback)
  sio.connect()
  time.sleep(100)

I got that websocket implementation from giel (on irc) ("from websocket import WebSocket"), here it is (put in file websocket/__init__.py):

  http://pastebin.com/jrxachSF

You can start using

  #> python socketio.py

it should dump the messages to stdout

[slush]

    S.ws.send('1::/mtgox')

*headdesk*

* slush bashing all developers writing incomplete documentations and expecting that some things are so obvious they don't need to be in doc.

molecular, you're my savior. What's your donation address?

[molecular]

   S.ws.send('1::/mtgox')

*headdesk*

* slush bashing all developers writing incomplete documentations and expecting that some things are so obvious they don't need to be in doc.

molecular, you're my savior. What's your donation address?

cool to have been of help, especiall to you, slush.

almost didn't post, but so many complained so I did... instant karma

1JX3YkScnu2RBHjUBZTLB6qBgmGeUWxANL

EDIT: slush, are you coming to Prague? I'm there on saturday and maybe sunday, maybe I recognize you

[slush]

No, I'm not coming to Prague...
(...dramatic drums...)
...because I already am in Prague!.

Would be nice to meet you, however I'm myself curious how I'll be able to communicate with my "fluent" english :-).

[molecular]

No, I'm not coming to Prague...
(...dramatic drums...)
...because I already am in Prague!.

Would be nice to meet you, however I'm myself curious how I'll be able to communicate with my "fluent" english :-).

Nice. Hoping to meet you. Thanks for the generous token of appreciation you sent me

[jothan]

   S.ws.send('1::/mtgox')

almost didn't post, but so many complained so I did... instant karma

1JX3YkScnu2RBHjUBZTLB6qBgmGeUWxANL

EDIT: slush, are you coming to Prague? I'm there on saturday and maybe sunday, maybe I recognize you

+1 BTC, Thanks !

[zelyony]

I created project for .NET socket.io + WebSockets at codeplex.com
http://socketiowebsockets.codeplex.com/
for now it can hold ssl-connection with mtgox (no parsing json and etc)

my old ideas (in this topic) died and raised again

[Ente]

..I try to get a simple python client running for days now. When I finally managed to connect via websockets, the first thing I read was "deprecated, use socket.io instead".

Instead of using socket.io (which I cant get to work) I could as well just connect to the underlying websocket just as before, not? I will not have fallback to longpoll etc in case the websocket dies and the socket.io still works. But besides that, my ticker should happily work with just websockets for the future, not?

Is there a new websocket daemon running now, for socket.io, with a different adress?

TL;DR:
Will ws://websocket.mtgox.com/mtgox continue to work?

Ente

[piotr_n]

..I try to get a simple python client running for days now. When I finally managed to connect via websockets, the first thing I read was "deprecated, use socket.io instead".

Even though it's "deprecated" it's still much more reliable than the socket.io interface.
So please don't turn it off!

The only problem is that it doesn't work with the latest Chrome and other late WebSocket libs... and nobody wants to fix it.
That's probably why they labeled it "deprecated".

[molecular]

..I try to get a simple python client running for days now. When I finally managed to connect via websockets, the first thing I read was "deprecated, use socket.io instead".

Instead of using socket.io (which I cant get to work) I could as well just connect to the underlying websocket just as before, not? I will not have fallback to longpoll etc in case the websocket dies and the socket.io still works. But besides that, my ticker should happily work with just websockets for the future, not?

Is there a new websocket daemon running now, for socket.io, with a different adress?

TL;DR:
Will ws://websocket.mtgox.com/mtgox continue to work?

Ente

the way it works: you connect to socket.io server and it will give you a unique ID which will be part of the url you can subsequently use to connect to websocket. socket.io is just a "negotiation protocol".

look at my python code here: https://bitcointalk.org/index.php?topic=5855.msg629600#msg629600

[molecular]

..I try to get a simple python client running for days now. When I finally managed to connect via websockets, the first thing I read was "deprecated, use socket.io instead".

Even though it's "deprecated" it's still much more reliable than the socket.io interface.
So please don't turn it off!

The only problem is that it doesn't work with the latest Chrome and other late WebSocket libs... and nobody wants to fix it.
That's probably why they labeled it "deprecated".

"direct connection" to old websocket url is deprecated. you should now use socket.io to get an ID which will be part of a websocket url you can use.

check out sample implementation: https://bitcointalk.org/index.php?topic=5855.msg629600#msg629600

[piotr_n]

"direct connection" to old websocket url is deprecated. you should now use socket.io to get an ID which will be part of a websocket url you can use.

So you see the new "indirect connection" working reliably?

[molecular]

"direct connection" to old websocket url is deprecated. you should now use socket.io to get an ID which will be part of a websocket url you can use.

So you see the new "indirect connection" working reliably?

no, not reliable since roughly a week, lots of outages on the websocket part, pretty stable many weeks before that. havent seen the socket.io part fail yet.

[bzzard]

havent seen the socket.io part fail yet.

How come you say such thing when such sites as:
http://www.btccharts.com/
http://bitcoin.clarkmoody.com/

have a connection but there's no data received, any explanation for this?

[molecular]

havent seen the socket.io part fail yet.

How come you say such thing when such sites as:
http://www.btccharts.com/
http://bitcoin.clarkmoody.com/

have a connection but there's no data received, any explanation for this?

socket.io is just a tool to negotiate a transport layer and keepalive. one of the possible transports is websocket. this means you usually use socket.io PLUS websocket (at least that's how I do it).

I'm guessing a connected websocket with no data flowing means some problem in magicaltux' backend.

any insights appreciated.

[zelyony]

request to "https://socketio.mtgox.com/socket.io/1" doesnt respond (net::ERR_CONNECTION_REFUSED)
Is URL valid?
something happened.. but WHAT happened?..

UPD1:
P.S.
I wrote to Mt.Gox support issue: open some technical twitter channel with info: what? when? how long?
IRC-channel is just flame - no info, no logs, no history, no search..

UPD2 (2:28 AM):
its working now

[molecular]

request to "https://socketio.mtgox.com/socket.io/1" doesnt respond (net::ERR_CONNECTION_REFUSED)
Is URL valid?
something happened.. but WHAT happened?..

UPD1:
P.S.
I wrote to Mt.Gox support issue: open some technical twitter channel with info: what? when? how long?
IRC-channel is just flame - no info, no logs, no history, no search..

UPD2 (2:28 AM):
its working now

yeah, I see this, too from time to time. also a lot of times the connection setup works fine, but there's just no data flow. mtgox datastreaming is on and off on most levels as far as I can tell.

supposedly socket.io is some typical bay-area shit and there's a big rewrite pull request pending for a while now that could fix the issue mtgox is suffering from.

my latest "kind-of-official" info is that the socket.io server is now restarted every 2 hours to mitigate effects of this (and potentially other) issue(s).

I for one am preparing to survive using api requests (switched form api/0 to api/1)

[zelyony]

I see data

{"channel":"24e67e0d-1cad-4cc0-9e7a-f8523ef460fe","op":"private","origin":"broadcast","private":"depth","depth":{"price":"5.80025","type":2,"type_str":"bid","volume":"0","price_int":"580025","volume_int":"0","item":"BTC","currency":"USD","now":"1326266472912980","total_volume_int":"19811387"}}    

why exists messages with volume=0 ?
is this rounding errors in MtGox's guts or users posts such orders to exchange ?

+++ existence of "total_volume_int" - is wonderful

[rwcp97]

havent seen the socket.io part fail yet.

How come you say such thing when such sites as:
http://www.btccharts.com/
http://bitcoin.clarkmoody.com/

have a connection but there's no data received, any explanation for this?

I have the same problem using the websocket from my C++/Qt-bot - sometimes, usually after some hours working correctly, there is no more data coming in through the socket (but the connection isn't closed by the remote host).
So I check for incoming data and if there isn't anything for 3 minutes, I close the connection and reconnect. Afterwards it's working without problems again. Strange anyway