Bitcoin Forum
December 09, 2016, 09:50:20 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 [15] 16 17 18 19 20 21 »  All
  Print  
Author Topic: [BETA] MTGox websocket API, testers wanted  (Read 72257 times)
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
March 21, 2013, 09:33:53 PM
 #281

Clarks's client shows the same. Connects and then silence.

Does anybody see the same?

Total MtGox tease. Usually happens during high-traffic times.

Thank you. Just wanted to be sure that it is not my local problem.

By the way I wanted to thank you for your hint about orderbook state synchronization.

full book stamp and 60 secs depth channel buffer + ticker fixing made my mtgox stream api client work very well Smiley
Thank you once more Smiley

1481277020
Hero Member
*
Offline Offline

Posts: 1481277020

View Profile Personal Message (Offline)

Ignore
1481277020
Reply with quote  #2

1481277020
Report to moderator
1481277020
Hero Member
*
Offline Offline

Posts: 1481277020

View Profile Personal Message (Offline)

Ignore
1481277020
Reply with quote  #2

1481277020
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481277020
Hero Member
*
Offline Offline

Posts: 1481277020

View Profile Personal Message (Offline)

Ignore
1481277020
Reply with quote  #2

1481277020
Report to moderator
Puppet
Legendary
*
Offline Offline

Activity: 966


View Profile
March 22, 2013, 07:46:06 PM
 #282

i seem to be able to connect to websocket occasionally; but socket.io seems completely non responsive now?
prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
March 23, 2013, 03:57:55 PM
 #283

i seem to be able to connect to websocket occasionally; but socket.io seems completely non responsive now?

Yesterday in the #mtgox channel on freenode MT has finally announced the hostname of the new beta server: socketio-beta.mtgox.com

This new server seems much improved, I have not yet seen it disconnecting or acting strange a single time since yesterday (maybe partially also because only few are using it yet).

goxtool now also supports this new server: pull the latest release and then:

Code:
./goxtool.py --protocol=socketio-beta

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
March 23, 2013, 04:04:48 PM
 #284

so the only thing that have to be changed is url? Any changes to socket.io lib version necessary?


are you sure you placed correct url? I tried and getting multiple reconnects and errors from socket io client?

can you check?

prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
March 23, 2013, 04:33:25 PM
 #285

so the only thing that have to be changed is url? Any changes to socket.io lib version necessary?


are you sure you placed correct url? I tried and getting multiple reconnects and errors from socket io client?

can you check?

This also seems to use different server software and it at the moment it seems it is essential that the websocket connection is done on the same open socket that the initial http request came from. So the client has to fetch the session ID (with the header Connection: keep-alive) and then use that same socket to  upgrade the connection to websocket. Otherwise it will fail with reason "client not handshaken". MT said this is a bug and he intends to fix it. Meanwhile I have tweaked my own socketio client to be able to use the same socket for both requests, so for me it works. I don't know how the javascript implementation handles this.

Code:
<prof7bit> socketio-beta rocks. no disconnect no freeze, thats how it should have been all the time
<MagicalTux> heh
<MagicalTux> prof7bit: except connection will usually fail if you do not use the same socket for the initial request and the websocket
<prof7bit> that is fixed in my client now. it was actually simpler to fix than I expected
<MagicalTux> ya, but that's something /we/ should be actually fixing
<MagicalTux> we'll probably disable everything but websocket and flashsocket, and make socket.io server accept any session id

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
March 23, 2013, 05:08:20 PM
 #286

Does it mean that it is actually not original socket.io client have to be used but rather different client implementation?

Do you have any link to the new socketio-beta project?

or what I did not get?

yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
March 23, 2013, 07:53:50 PM
 #287

Using puTTY I tried:

=====================================
websocket.mtgox.com, port 80, RAW

Connection is established, I then type:

GET /mtgox HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: websocket.mtgox.com
Origin: null


Server replied with handshake ACK.

Then I was getting the stream displayed, until server seemed to drop connection. I tried this a few times with success and now cant repeat.

=====================================
socketio-beta.mtgox.com, port 80, RAW

Connection is established, I then type:

GET /mtgox HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: socketio-beta.mtgox.com
Origin: null


No reply, Server remains silent?
=====================================

Puppet
Legendary
*
Offline Offline

Activity: 966


View Profile
March 23, 2013, 08:38:45 PM
 #288

Im getting this error using the beta socketio server with the javascript library:


DEBUG: Error: Hostname/IP doesn't match certificate's altnames
    at SecurePair.<anonymous> (tls.js:1280:23)
    at SecurePair.EventEmitter.emit (events.js:92:17)
    at SecurePair.maybeInitFinished (tls.js:883:10)
    at CleartextStream.read [as _read] (tls.js:421:15)
    at CleartextStream.Readable.read (_stream_readable.js:293:10)
    at EncryptedStream.write [as _write] (tls.js:330:25)
    at doWrite (_stream_writable.js:211:10)
    at writeOrBuffer (_stream_writable.js:201:5)
    at EncryptedStream.Writable.write (_stream_writable.js:172:11)
    at write (_stream_readable.js:547:24)
yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
March 23, 2013, 08:47:32 PM
 #289

Im getting this error using the beta socketio server with the javascript library:


DEBUG: Error: Hostname/IP doesn't match certificate's altnames
    at SecurePair.<anonymous> (tls.js:1280:23)
    at SecurePair.EventEmitter.emit (events.js:92:17)
    at SecurePair.maybeInitFinished (tls.js:883:10)
    at CleartextStream.read [as _read] (tls.js:421:15)
    at CleartextStream.Readable.read (_stream_readable.js:293:10)
    at EncryptedStream.write [as _write] (tls.js:330:25)
    at doWrite (_stream_writable.js:211:10)
    at writeOrBuffer (_stream_writable.js:201:5)
    at EncryptedStream.Writable.write (_stream_writable.js:172:11)
    at write (_stream_readable.js:547:24)


maybe too much DNS? try using this as the URL:

socketio-lb-1655485032.us-west-1.elb.amazonaws.com

I still cant hook up thru telnet.

prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
March 23, 2013, 08:51:45 PM
 #290

Using puTTY I tried:

=====================================
socketio-beta.mtgox.com, port 80, RAW

Connection is established, I then type:

GET /mtgox HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
Host: socketio-beta.mtgox.com
Origin: null


No reply, Server remains silent?
=====================================

becaue its a socket.io server and not a websocket server:

telnet socketio-beta.mtgox.com 80
Trying 184.169.152.65...
Connected to socketio-lb-1655485032.us-west-1.elb.amazonaws.com.
Escape character is '^]'.
GET /socket.io/1 HTTP/1.1
Host: socketio-beta.mtgox.com:80
Connection: keep-alive

HTTP/1.1 200 OK
Content-Type: text/plain
Date: Sat, 23 Mar 2013 20:35:15 GMT
Connection: keep-alive
Transfer-Encoding: chunked

53
W_4my4iKDRSNXtJN89vh:60:60:websocket,flashsocket,htmlfile,xhr-polling,jsonp-polling
0

then you parse the second line of the response take the ID W_4my4iKDRSNXtJN89vh to form a new URL that look like /socket.io/1/websocket/W_4my4iKDRSNXtJN89vh and then you use the same still open socket (its still open because you sent keep-alive header) and initiate the websocket request:

GET /socket.io/1/websocket/W_4my4iKDRSNXtJN89vh HTTP/1.1
Upgrade: WebSocket
and so on...

If it does not work then make sure that you are sending websocket version 13 headers (see the websocket RFC I posted a few days ago) and not the old version you showed above (that were not version 13 headers), this also means you need to provide Sec-Websocket-Key header and all that. See how goxtool does it (its using https://pypi.python.org/pypi/websocket-client/ with an overridden connect() method to do the initial http for the ID before the actual websocket starts).

You either need to find a complete socket.io implementation for your programming language or a websocket V13 client that you can tweak the same way like I did it with the python client.

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
Fireball
Hero Member
*****
Offline Offline

Activity: 673


View Profile WWW
March 23, 2013, 08:53:09 PM
 #291

Im getting this error using the beta socketio server with the javascript library:

Yes, that's because this server uses cert issued for socketio.mtgox.com, which obviously does not match the different hostname.

Margin trading platform OrderBook.net (ICBIT): https://orderbook.net
Follow us in Twitter: https://twitter.com/orderbooknet
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
March 23, 2013, 09:04:10 PM
 #292

So this is a problem of certificates and not of socket.io-client version?

Or which version of socket.io-client should I use?

prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
March 23, 2013, 09:18:26 PM
 #293

here is a complete example as far into the handshake as it is possible with telnet (because after that it switches to binary WS protocol):

--($)-- telnet socketio-beta.mtgox.com 80
Trying 184.169.152.65...
Connected to socketio-lb-1655485032.us-west-1.elb.amazonaws.com.
Escape character is '^]'.
GET /socket.io/1 HTTP/1.1
Host: socketio-beta.mtgox.com:80
Connection: keep-alive


HTTP/1.1 200 OK
Content-Type: text/plain
Date: Sat, 23 Mar 2013 21:06:56 GMT
Connection: keep-alive
Transfer-Encoding: chunked

53
Qi-RuYva5rgLXtXuhsrE:60:60:websocket,flashsocket,htmlfile,xhr-polling,jsonp-polling
0


GET /socket.io/1/websocket/Qi-RuYva5rgLXtXuhsrE HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: socketio-beta.mtgox.com:80
Origin: socketio-beta.mtgox.com:80
Sec-WebSocket-Key: UWaphFPiSqq3f2gOmaD5Sg==
Sec-WebSocket-Version: 13


HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: Bzz8qKJPEMNNOgC4hOZd3iZNb5o=

�1::�2::


the �1:: and the �2:: are the first two websocket frames it sends (websocket is a binary protocol), now it expects "1::/mtgox" and for every "2::" it sends you must reply "2::", of course you need to properly mask and frame what you send as it is described in the Websocket RFC. This is how socket.io over websocket (or websocket over socket.io?) looks like. Unfortunately from here on its impossible to proceed with telnet because of the binary frames.

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
March 23, 2013, 09:20:36 PM
 #294

So this is a problem of certificates and not of socket.io-client version?

Or which version of socket.io-client should I use?

Maybe not use SSL, use unencrypted on port 80

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
March 23, 2013, 09:23:07 PM
 #295

So this is a problem of certificates and not of socket.io-client version?

Or which version of socket.io-client should I use?

Maybe not use SSL, use unencrypted on port 80


Tried that already - the same result - errors

yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
March 23, 2013, 11:05:03 PM
 #296

@prof7bit,

Thanks so much for your great reply!

I explored thru telnet first just to get my toes wet.

I am working ontop of WindowsMFC CAsyncSocket class and I got the websocket going OK (when server is up).

I'll set about trying to get CAsyncSocket to act as socket.io now and manage the binary frames as they arrive.

I'm not very experienced when it comes to streaming stuff, you just helped me bigtime! Smiley

alpet
Legendary
*
Offline Offline

Activity: 1535


View Profile WWW
March 25, 2013, 08:58:37 AM
 #297

My bot received trades with timestamps from future (~ 5 seconds). But my system clock it is precisely synchronized with atomic time (~0.5 ms) through several NTP servers.
Also now all calls (order/lag or private/info) always rejected with message "Invalid call". Can I retrieve more info for this error or API have limited verbosity?

Cryptsy, BFL and Bitsyncom - scammers, don't work with them!
Плавайте поездами Аэрофлота.
prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
March 25, 2013, 09:34:23 AM
 #298

My bot received trades with timestamps from future (~ 5 seconds). But my system clock it is precisely synchronized with atomic time (~0.5 ms) through several NTP servers.
Also now all calls (order/lag or private/info) always rejected with message "Invalid call". Can I retrieve more info for this error or API have limited verbosity?

Did you change your algorithm that creates your nonce? If you did then you might need a fresh API key. Nonce problems were the only problems I have witnessed causing sudden mysterious "Invalid call" problems.

Also if you send a lot of commands immediately after each other they will be processed in parallel on the server and this can mess up the chronological order (basically a race condition on their server) that will cause sporadic "Invalid call" (because the nonce not strictly incrementing anymore if the sequence is messed up). The only way around it is to answer each "Invalid call" by just sending the same command (with a new nonce of course) again... until it succeeds.

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
alpet
Legendary
*
Offline Offline

Activity: 1535


View Profile WWW
March 25, 2013, 10:25:12 AM
 #299

prof7bit
Thanks! I have not changed anything in the message signing function, my nonce is MD5 of timestamp. API key and secret are correct. Multiple attempts give same result "Invalid call". So bad that the server does not provide a detailed description of the error.

Cryptsy, BFL and Bitsyncom - scammers, don't work with them!
Плавайте поездами Аэрофлота.
yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
March 25, 2013, 01:25:42 PM
 #300

I've got a little further initiating the beta socketio stream, below is my TRACE.

############SOCKET OPENED BY CLIENT

############TX:
GET /socket.io/1 HTTP/1.1
Host: socketio-beta.mtgox.com:80
Connection: keep-alive

############RX:
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Mon, 25 Mar 2013 03:48:51 GMT
Connection: keep-alive
Transfer-Encoding: chunked


############TX:
GET /socket.io/1/websocket/ HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: socketio-beta.mtgox.com:80
Origin: socketio-beta.mtgox.com:80


############RX:
53
GEyZU7Bw-ivuvI6CZon3:60:60:websocket,flashsocket,htmlfile,xhr-polling,jsonp-polling
0


############TX:
GET /socket.io/1/websocket/GEyZU7Bw-ivuvI6CZon3 HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: socketio-beta.mtgox.com:80
Origin: socketio-beta.mtgox.com:80
Sec-WebSocket-Key: UWaphFPiSqq3f2gOmaD5Sg==
Sec-WebSocket-Version: 13


############SOCKET CLOSED BY HOST

The server closes the socket when I dont give it correct Sec-WebSocket-Key.

So I'm guessing in my last TX above I need to derive Sec-WebSocket-Key somehow.

Should i do it like mentioned in websocket wiki?:

Quote
The client sends a Sec-WebSocket-Key which is base64 encoded. To form a response, the magic string 258EAFA5-E914-47DA-95CA-C5AB0DC85B11 is appended to this (undecoded) key. The resulting string is then hashed with SHA-1, then base64 encoded. Finally, the resulting reply occurs in the header Sec-WebSocket-Accept.

Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 [15] 16 17 18 19 20 21 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!