Bitcoin Forum
December 03, 2016, 11:57:00 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [18] 19 20 21 »  All
  Print  
Author Topic: [BETA] MTGox websocket API, testers wanted  (Read 72197 times)
prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
March 31, 2013, 09:26:09 AM
 #341

are you plotting volume also? if so do you find the live numbers sum closely to the history?

Not yet. I'm only plotting a quick & dirty price chart of the last 24 hours so I can quickly see from 5 meters distance if it has gone up or down during the night. I was too lazy until now to also write code to plot volume bars (I also don't have enough room in the terminal window to plot too many details in ascii-art with ncurses)

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
1480809420
Hero Member
*
Offline Offline

Posts: 1480809420

View Profile Personal Message (Offline)

Ignore
1480809420
Reply with quote  #2

1480809420
Report to moderator
1480809420
Hero Member
*
Offline Offline

Posts: 1480809420

View Profile Personal Message (Offline)

Ignore
1480809420
Reply with quote  #2

1480809420
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
April 08, 2013, 01:02:58 AM
 #342

I think I remember reading in this thread that MagicalTux was going to change the order API so that people cant place more asks than BTC they have on account.

Watching the live depth stream I dont think that this is implemented yet.

On the GBP stream below we can see a bot went crazy and started placing many asks of the 2 same values resulting in 2 long horizontal lines.

This was clearly exceeding the holdings of its owner. The stream window below is 5 minutes.


prof7bit
Hero Member
*****
Offline Offline

Activity: 672



View Profile WWW
April 08, 2013, 10:30:30 AM
 #343

This was clearly exceeding the holdings of its owner. The stream window below is 5 minutes.

No, it cannot exceed the holdings of the owner, otherwise it would not show up in the order book. It would stay in "invalid" state until funds become available and only be visible to the owner himself but not for anyone else.

Old Bitcoins recycling and disposal: 1N7ZcHoe77JncRYAg27RVqJZbrjqUQ77WE
Blockchain.info -> Multibit converter (and much more) https://github.com/prof7bit/wallet-key-tool
yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
April 08, 2013, 02:47:23 PM
 #344

No, it cannot exceed the holdings of the owner, otherwise it would not show up in the order book. It would stay in "invalid" state until funds become available and only be visible to the owner himself but not for anyone else.
Wow well there are some large players then!

Well I now have stable multicurrency stream.io coming into my app. I still notice the odd hang during heavy times (maybe the ddos is not helping?).

I look for consecutive zero receive bytes or receive timout, if I see this I close the socket, destroy it, recreate and reconnect after a short delay.

One note to others (has been mentioned before), the channel UUIDs are not worth much from the client end. if you want multicurrency stream then you should filter on the human readable channel type and currency type. All UUIDs are not published in MtGox API docs, I suppose if you want to use them you would have to discover them yourself in the RX log.

Now how long this will stay good until I have to revisit this part of my code I dont know, hopefully for a long long time. Thankyou to all who helped me stumble to this point  Grin

genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
April 08, 2013, 03:24:51 PM
 #345

This is the list of channel ids I descovered till the moment

Quote
{ 'USD' : {
                              'trades' : 'dbf1dee9-4f2e-4a08-8cb7-748919a71b21',
                              'ticker' : 'd5f06780-30a8-4a48-a2f8-7ed181b4a13f',
                              'depth' : '24e67e0d-1cad-4cc0-9e7a-f8523ef460fe'
                           },
                           'EUR' : {
                              'trades' : 'dbf1dee9-4f2e-4a08-8cb7-748919a71b21',
                              'depth' : '057bdc6b-9f9c-44e4-bc1a-363e4443ce87',
                              'ticker' : '0bb6da8b-f6c6-4ecf-8f0d-a544ad948c15'
                           },
                           'GBP' : {
                              'trades' : 'dbf1dee9-4f2e-4a08-8cb7-748919a71b21',
                              'depth' : '60c3af1b-5d40-4d0e-b9fc-ccab433d2e9c',
                              'ticker': '7b842b7d-d1f9-46fa-a49c-c12f1ad5a533'
                           },
                           'CNY' : {
                              'trades' : 'dbf1dee9-4f2e-4a08-8cb7-748919a71b21',
                              'depth' : '0d1ecad8-e20f-459e-8bed-0bdcf927820f',
                              'ticker' : 'c251ec35-56f9-40ab-a4f6-13325c349de4'
                           },
                           'RUB' : {
                              'trades' : 'dbf1dee9-4f2e-4a08-8cb7-748919a71b21',
                              'depth' : 'bd04f720-3c70-4dce-ae71-2422ab862c65',
                              'ticker' : 'd6412ca0-b686-464c-891a-d1ba3943f3c6'
                           },
                           'CAD' : {
                              'trades' : 'dbf1dee9-4f2e-4a08-8cb7-748919a71b21',
                              'depth' : '10720792-084d-45ba-92e3-cf44d9477775',
                              'ticker' : '5b234cc3-a7c1-47ce-854f-27aee4cdbda5'
                           },
                           'PLN' : {
                              'trades' : 'dbf1dee9-4f2e-4a08-8cb7-748919a71b21',
                              'depth' : 'b4a02cb3-2e2d-4a88-aeea-3c66cb604d01',
                              'ticker' : 'e4ff055a-f8bf-407e-af76-676cad319a21'
                           },
                           'AUD' : {
                              'trades' : 'dbf1dee9-4f2e-4a08-8cb7-748919a71b21',
                              'depth' : 'eb6aaa11-99d0-4f64-9e8c-1140872a423d',
                              'ticker' : '296ee352-dd5d-46f3-9bea-5e39dede2005'
                           }

Clark
Hero Member
*****
Offline Offline

Activity: 540


So much code.


View Profile WWW
April 08, 2013, 04:40:00 PM
 #346

This is the list of channel ids I descovered till the moment

Well since we're playing that game  Smiley

Code:
            {
                'AUD': {'eb6aaa11-99d0-4f64-9e8c-1140872a423d': 'ticker',
                        '296ee352-dd5d-46f3-9bea-5e39dede2005': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'CAD': {'10720792-084d-45ba-92e3-cf44d9477775': 'ticker',
                        '5b234cc3-a7c1-47ce-854f-27aee4cdbda5': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'CHF': {'2644c164-3db7-4475-8b45-c7042efe3413': 'ticker',
                        '113fec5f-294d-4929-86eb-8ca4c3fd1bed': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'CNY': {'c251ec35-56f9-40ab-a4f6-13325c349de4': 'ticker',
                        '0d1ecad8-e20f-459e-8bed-0bdcf927820f': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'DKK': {'e5ce0604-574a-4059-9493-80af46c776b3': 'ticker',
                        '9219abb0-b50c-4007-b4d2-51d1711ab19c': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'EUR': {'0bb6da8b-f6c6-4ecf-8f0d-a544ad948c15': 'ticker',
                        '057bdc6b-9f9c-44e4-bc1a-363e4443ce87': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'GBP': {'7b842b7d-d1f9-46fa-a49c-c12f1ad5a533': 'ticker',
                        '60c3af1b-5d40-4d0e-b9fc-ccab433d2e9c': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'HKD': {'d3ae78dd-01dd-4074-88a7-b8aa03cd28dd': 'ticker',
                        '049f65dc-3af3-4ffd-85a5-aac102b2a579': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'JPY': {'a39ae532-6a3c-4835-af8c-dda54cb4874e': 'ticker',
                        '94483e07-d797-4dd4-bc72-dc98f1fd39e3': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'NOK': {'7532e866-3a03-4514-a4b1-6f86e3a8dc11': 'ticker',
                        '66da7fb4-6b0c-4a10-9cb7-e2944e046eb5': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'NZD': {'5ddd27ca-2466-4d1a-8961-615dedb68bf1': 'ticker',
                        'cedf8730-bce6-4278-b6fe-9bee42930e95': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'PLN': {'b4a02cb3-2e2d-4a88-aeea-3c66cb604d01': 'ticker',
                        'e4ff055a-f8bf-407e-af76-676cad319a21': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'RUB': {'bd04f720-3c70-4dce-ae71-2422ab862c65': 'ticker',
                        'd6412ca0-b686-464c-891a-d1ba3943f3c6': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'SEK': {'6caf1244-655b-460f-beaf-5c56d1f4bea7': 'ticker',
                        '8f1fefaa-7c55-4420-ada0-4de15c1c38f3': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'SGD': {'2cb73ed1-07f4-45e0-8918-bcbfda658912': 'ticker',
                        '41e5c243-3d44-4fad-b690-f39e1dbb86a8': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'THB': {'d58e3b69-9560-4b9e-8c58-b5c0f3fda5e1': 'ticker',
                        '67879668-532f-41f9-8eb0-55e7593a5ab8': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'},
                'USD': {'d5f06780-30a8-4a48-a2f8-7ed181b4a13f': 'ticker',
                        '24e67e0d-1cad-4cc0-9e7a-f8523ef460fe': 'depth',
                        'dbf1dee9-4f2e-4a08-8cb7-748919a71b21': 'trades'}
            }

PGP KEY | 1Bitcoin3Tg2KWyAq3wzivdqwYqGwKYaGd
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
April 08, 2013, 05:22:04 PM
 #347

Well Done !!! Clark Smiley

Clark
Hero Member
*****
Offline Offline

Activity: 540


So much code.


View Profile WWW
April 08, 2013, 06:00:58 PM
 #348

Pro tip: http://data.mtgox.com/api/1/generic/currency?currency=USD

PGP KEY | 1Bitcoin3Tg2KWyAq3wzivdqwYqGwKYaGd
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
April 08, 2013, 06:02:42 PM
 #349


wow! cool Smiley

yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
April 08, 2013, 06:48:18 PM
 #350


+1! very useful!

puax
Newbie
*
Offline Offline

Activity: 17


View Profile
April 13, 2013, 11:02:09 AM
 #351

Is it me or the websocket has been down for days? Even my most basic implementation is silent.
yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
April 13, 2013, 11:40:48 AM
 #352

Is it me or the websocket has been down for days? Even my most basic implementation is silent.

My socket.io implementation gets as far as upgrade and then I just receive zero-byte returns, I had previously coded zero-byte return to trigger connection reset, I took that trigger out and the receive function just finishes instantly with zero-bytes received.

It's been like this since MtGox went down, and has not recovered even after MtGox opened for trading via the user-account web interface again.

So I'm not sure what the problem is. Maybe we just wait? Sad


13:34:43
=============================================================
THR_MtGox::SocketCreate: CONNECTED TO: socketio.mtgox.com:80
=============================================================

13:34:43 TRANSMIT>>>>>>>>
GET /socket.io/1/?Currency=EUR HTTP/1.1
Host: socketio.mtgox.com/mtgox?Currency=EUR
Connection: keep-alive



13:34:44 RECEIVE>>>>>>>>>
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Sat, 13 Apr 2013 11:34:43 GMT
Connection: keep-alive
Transfer-Encoding: chunked

30
b-LJ3EKgBF4kkuvdLP1d:60:60:websocket,flashsocket
0


13:34:44 TRANSMIT>>>>>>>>
GET /socket.io/1/websocket/b-LJ3EKgBF4kkuvdLP1d?Currency=EUR HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: socketio.mtgox.com/mtgox?Currency=EUR
Origin: socketio.mtgox.com/mtgox?Currency=EUR
Sec-WebSocket-Key: UWaphFPiSqq3f2gOmaD5Sg==
Sec-WebSocket-Version: 13


13:34:45 RECEIVE>>>>>>>>>
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: Bzz8qKJPEMNNOgC4hOZd3iZNb5o=

1::
13:34:45 TRANSMIT>>>>>>>>
TX_STR_FRAMED -> "1::/mtgox"


...ZERO-BYTE RX FROM NOW ON...

genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
April 13, 2013, 11:44:38 AM
 #353

Since mtgox trading halt socket.io api does not work, or works sparadically.

I hade to upgrade our client to be able to work in hybrid mode basically request full orderbook via http without socket.io connect.

and additional timeout logic not to get blocked by mtgox blocker.

yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
April 13, 2013, 11:53:16 AM
 #354

Since mtgox trading halt socket.io api does not work, or works sparadically.

I hade to upgrade our client to be able to work in hybrid mode basically request full orderbook via http without socket.io connect.

and additional timeout logic not to get blocked by mtgox blocker.

So you try socket.io and if it fails you fall back to http get with extended connection timeout. Kudos for working around it!

What kind of delay do you have?... Approx how old is the latest book entry that you receive? Do you get depth and trades?

genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
April 13, 2013, 12:10:17 PM
 #355

Actually when socket.io works I start with http request full orderbook then when it is initalized socket sends update messages.

when socket.io reconnects the whole initializaton starts again with requesting full order book via http.

the problem is that you if you request full orderbook more then 5 times per hour as stated in mtgox stream api documentation.

So I had to add 5 times limitating logic.

and the whole system worked well until recently.

so now every 15 minutes I trigger connection check and if it is not connected request system to reinitialize orderbook.

this additional logic is implemented as upper layer over original client system.

currently I listen to depth and ticker for  quick fixes of broken orderbook without unnecessary reinitialization.

for trades we use bitcoincharts stream it contains not only mtgox trades.

We plan to advance the system to be able to get trades from mtgox channel too but we want to integrate it with bitcoincharts stream for robustness.

yucca
Full Member
***
Offline Offline

Activity: 217



View Profile WWW
April 13, 2013, 02:41:37 PM
 #356

Actually when socket.io works I start with http request full orderbook then when it is initalized socket sends update messages.

when socket.io reconnects the whole initializaton starts again with requesting full order book via http.

the problem is that you if you request full orderbook more then 5 times per hour as stated in mtgox stream api documentation.

So I had to add 5 times limitating logic.

and the whole system worked well until recently.

so now every 15 minutes I trigger connection check and if it is not connected request system to reinitialize orderbook.

this additional logic is implemented as upper layer over original client system.

currently I listen to depth and ticker for  quick fixes of broken orderbook without unnecessary reinitialization.

for trades we use bitcoincharts stream it contains not only mtgox trades.

We plan to advance the system to be able to get trades from mtgox channel too but we want to integrate it with bitcoincharts stream for robustness.


Thanks for sharing your insight, sounds like your going the extra mile to mantain as much order as possible on such a shaky foundation.

I'm not sure I have the drive to start wrtiting/debugging such a workaround, I would end up getting kicked off for >5 tries an hour, I'd reset my gateway and try again and probably end up being block banned.

At the moment I have no users reliant on my software so do not have the stressful job of trying to get a fix/workaround out. I will sit tight for a few days and work on some GUI stuff.

Perhaps when MtGox stream re-establishes properly it will run like a dream, let us hope so. ( until it breaks again:Smiley ).

Best of luck with your commendable efforts.


pescador
Newbie
*
Offline Offline

Activity: 14



View Profile
April 22, 2013, 05:51:04 PM
 #357

I'm experimenting with the MtGox websocket API. I'm using Python with the websocket module from https://pypi.python.org/pypi/websocket-client. See example code below

Code:
import threading
import websocket
import json

class mtgox( threading.Thread ):

    def run( self ):
        websocket.enableTrace( True )
        url = 'ws://websocket.mtgox.com/mtgox?Currency=USD'
        self.socket = websocket.WebSocketApp( url, on_open = self.on_open )
        self.socket.run_forever( )

    def subscribe( self, channel ):
        output = { 'op': 'mtgox.subscribe', 'type': channel }
        output = json.dumps( output )
        self.socket.send( output )

    def on_open( self, socket ):
        self.subscribe( 'depth' )
        self.subscribe( 'lag' )        
        self.subscribe( 'ticker' )
        self.subscribe( 'trades' )

if __name__ == '__main__':
    mtgox = mtgox( )
    mtgox.start( )

When I run this code I receive ticker and depth messages, but no lag or trade messages. Also I do not get any replies to my mtgox.subscribe commands, that the documentation seems to promise. However when I send a mtgox.subscribe command with a wrong type parameter, I get an error message "Unknown mtgox message type", so it seems my subscribe commands are received and accepted.

Could somebody please tell me why I'm not receiving trade and lag messages?
pescador
Newbie
*
Offline Offline

Activity: 14



View Profile
April 27, 2013, 08:52:40 AM
 #358

To answer my own question above. It seems to be working now. Now I also get trade and lag messages. Apparently it was something at MtGox's side.
pescador
Newbie
*
Offline Offline

Activity: 14



View Profile
April 28, 2013, 02:42:57 PM
 #359

I've downloaded the full bid / ask tables in USD with 'depth/full' and try to update them as depth and trade messages come in. From https://support.mtgox.com/entries/20800336-Multi-Currency-Trading I understand that the tables in USD also include the bids / asks in other currencies, that are converted to USD at the daily rate of the European Central Bank minus / plus a fee of 2.5 %.

How do I update my tables when a primary trade comes in with a price_currency other than USD?
genuise
Sr. Member
****
Offline Offline

Activity: 375


View Profile WWW
April 28, 2013, 08:00:40 PM
 #360

You should not update orderbooks using trades channel. You need to listen to depth channel to update orderbooks. And also it would be convinient to fix ordebooks with ticker channel messages if neccesary (in case not all depth messages were arrived and ordebook state is broken)

Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [18] 19 20 21 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!