Bitcoin Forum
December 02, 2016, 06:21:42 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 [3] 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 »  All
  Print  
Author Topic: [BETA] MTGox websocket API, testers wanted  (Read 72187 times)
Shrapnel
Newbie
*
Offline Offline

Activity: 9


View Profile
May 12, 2011, 10:19:27 AM
 #41

It seems, though, that I'm doing something wrong: I'm getting negative volumes in my depth data after a while.

Actually the negative values are cancelled orders.

Quote
Another point I'm unclear on: How can I possibly ensure that the timepoint of my request of the initial depth table and the starting timepoint of the websocket depth change messages match?
I'm also wondering the same thing. I think you can't unless the API send the complete data first.
I'm trying to do a realtime order book but it's currently very hard due to dark pool and sync issues.

My BTC OSX Dashboard Widget http://bitcointalk.org/index.php?topic=6690.0 (http://bitcointalk.org/index.php?topic=6690.0)
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480702902
Hero Member
*
Offline Offline

Posts: 1480702902

View Profile Personal Message (Offline)

Ignore
1480702902
Reply with quote  #2

1480702902
Report to moderator
1480702902
Hero Member
*
Offline Offline

Posts: 1480702902

View Profile Personal Message (Offline)

Ignore
1480702902
Reply with quote  #2

1480702902
Report to moderator
molecular
Donator
Legendary
*
Offline Offline

Activity: 2128



View Profile
May 12, 2011, 09:47:32 PM
 #42

It seems, though, that I'm doing something wrong: I'm getting negative volumes in my depth data after a while.

Actually the negative values are cancelled orders.

Of course. So to keep my copy of the "order book" synced, I add that negative value at the given price and should get 0, or (if some other order at the same price still exitsts) some positive value, but never a negative value, which happens to me.



PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
jed
Full Member
***
Offline Offline

Activity: 165

Jed McCaleb


View Profile WWW
May 12, 2011, 10:14:12 PM
 #43

I think it doesn't send every update to the order book. It seems like some random ones get dropped.

stellar.org   |    twitter
0x6763
Newbie
*
Offline Offline

Activity: 24


View Profile
May 13, 2011, 02:42:32 AM
 #44

It seems, though, that I'm doing something wrong: I'm getting negative volumes in my depth data after a while.

Actually the negative values are cancelled orders.

Of course. So to keep my copy of the "order book" synced, I add that negative value at the given price and should get 0, or (if some other order at the same price still exitsts) some positive value, but never a negative value, which happens to me.

Mtgox has some bugs.  The websocket often entirely skips sending some depth messages, sends some at a slightly different price than what they should be, and the APIs (or site in general) has rounding inconsistencies.  I've only partially worked around these issues by rounding all depth message prices to the 5th decimal place and volumes to the 3rd decimal place, and creating a button in my program that I can click to manually refresh the entire depth tables when I start noticing odd stuff.

An example is where a bot will download it's own open orders, see that one of it's orders is something like 6.253, but it wants it to be 6.252999, so it removes the order for 6.253 and creates a new one at 6.252999, and both of those come over the websocket just like that.  And the bot does this over and over again because the API for getting your own open orders rounds to the 4th decimal place, but the websocket will do at least 6 decimal places for prices and more than that for volume, so your table ends up removing the order at 6.253 over and over again, resulting in a negative number, while it keeps adding orders to 6.252999 over and over again, making some huge number that's not accurate.  The bugs and inconsistencies are pretty annoying.
Keefe
Hero Member
*****
Offline Offline

Activity: 681


View Profile
May 13, 2011, 03:48:53 AM
 #45

I was going to make a nice interface for the feed as well, but I'm not going to bother until the problems are resolved:
Rounding issues
No perfect way to start out in-sync
Alleged missing data and dark pool inconsistencies

molecular
Donator
Legendary
*
Offline Offline

Activity: 2128



View Profile
May 13, 2011, 11:32:29 PM
 #46

I was going to make a nice interface for the feed as well, but I'm not going to bother until the problems are resolved:
Rounding issues
No perfect way to start out in-sync
Alleged missing data and dark pool inconsistencies

MagicalTux, can you give us an estimate when you'll get around to implementing a solution to these problems or talk to us about a solution?

These problems are really not acceptable: we can't have the client side depth data out of sync or even worse stuff happening (missing data?)

I'm here to help with testing.

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
jaybny
Sr. Member
****
Offline Offline

Activity: 354


coder trader satoshi


View Profile
May 14, 2011, 12:49:05 AM
 #47

my program is listening to the WebSocket and publishing to twitter once an hour, on new highs or lows.

im using c# and am not  using JSON schema, just parsing directly... so if anything changes - I will break..

http://twitter.com/newberg_feed


Ycros
Newbie
*
Offline Offline

Activity: 1


View Profile
May 14, 2011, 08:10:11 AM
 #48

Herp, disregard what I posted before, I found my client library was sending an extraneous \r\n\r\n on the end there.

1BondRG1ZaHecQARxAtMcnT4g4uEH2pRrM
molecular
Donator
Legendary
*
Offline Offline

Activity: 2128



View Profile
May 14, 2011, 09:10:44 AM
 #49

I was going to make a nice interface for the feed as well, but I'm not going to bother until the problems are resolved:
Rounding issues
No perfect way to start out in-sync
Alleged missing data and dark pool inconsistencies

I'm seeing a slight problem for MagilTux to solve the "keep client depth tables in sync" problem, I hadn't thought about:

(Firstly, wether a client gets the initial depth data through the "classic API" or through an initial burst of depth-change messages doesn't make a difference for my thinking here.)

Problem is: the depth data a client receives initially is chopped off at the ends at a certain distance to the current last trade value.

Now whenever a trade happens, this window moves and some trades exit the window on the one end while others become "visible" on the other end.

There's 2 ways (at least) to solve this:

  • Screw that window completely, give the complete depth data to clients
  • On trade, determine entering/leaving orders and send them as new/cancelled over the channel

I'd go for the first option in combination with initially bursting change-messages instead of using the "classic API" to initially fill client depth table.

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
toffoo
Sr. Member
****
Offline Offline

Activity: 392



View Profile
May 17, 2011, 04:40:02 AM
 #50

Hi guys,
Does anybody have some example python code that just connects to the Mt.Gox websocket feed and maybe just spits out trades or order changes?
I'd like to start playing around with this, but I hate to try to re-create the wheel from scratch when I'm such a rusty python programmer.
Many thanks for any help!

mindtheft
Newbie
*
Offline Offline

Activity: 11


View Profile
May 17, 2011, 10:22:23 PM
 #51

A quick js chrome extension hack for WebSocket -

https://github.com/mindtheft/mtgox-Chrome-Extension

You can use the crx file to install the extension, or better yet check the source code and pack your own crx file to make sure there's no foul smells from mine.

0x6763
Newbie
*
Offline Offline

Activity: 24


View Profile
May 18, 2011, 06:16:43 PM
 #52

Hi guys,
Does anybody have some example python code that just connects to the Mt.Gox websocket feed and maybe just spits out trades or order changes?
I'd like to start playing around with this, but I hate to try to re-create the wheel from scratch when I'm such a rusty python programmer.
Many thanks for any help!


Here's a naive Mt. Gox websocket client I wrote in python: http://pastebin.com/FH6eWhFm
By default it just prints out trades as they happen, but you can modify it to do whatever you want.  If someone really wants an event-based version or something to make it a little easier to work with, you might be able to convince me (with a few bitcoins Wink) to put something together with pywsc or something.  I may eventually release an executable jar (clojure/java) file for those willing to buy a copy that will give you a GUI to view live trades and depth changes (though mtgox's websocket is still quite buggy in respect to depth changes...there would be a button to let you do manual refreshes as well).
martin
Full Member
***
Offline Offline

Activity: 150



View Profile WWW
May 18, 2011, 09:56:47 PM
 #53

I've been playing around a bit with this webcosket feed in C#, and I'm not sure if I'm doing things wrong or the feed is slightly broken.

Every time I receive a depth update I check the volume, if it's positive I add it to my local depth table, if it's negative I search the table for an exact matching depth entry, and remove it. After a while, this ends up with sell prices being less than buy prices - which is clearly wrong. Should this work, or am I missing something?

w.r.t all the people saying that sending the complete depth table on connection would be useful, I agree. It seems vital for the client to be able to request depth data, I would implement it as the client sending a request message for depth data, something like:

{ "op":"request-depth" }

And then the socket sends back the *complete* depth data in a *single message* in the same form that the web API currently returns it. In this way, once you receive the complete table you know you have a reliable dataset to perform updates to as they arrive.

The obvious caveat of this is that once you send the complete depth data, it must act as a "barrier", you can't send the complete state, and then an update to a stale state. (does this make sense?)

0x6763
Newbie
*
Offline Offline

Activity: 24


View Profile
May 19, 2011, 03:06:47 AM
 #54

I've been playing around a bit with this webcosket feed in C#, and I'm not sure if I'm doing things wrong or the feed is slightly broken.

Every time I receive a depth update I check the volume, if it's positive I add it to my local depth table, if it's negative I search the table for an exact matching depth entry, and remove it. After a while, this ends up with sell prices being less than buy prices - which is clearly wrong. Should this work, or am I missing something?

w.r.t all the people saying that sending the complete depth table on connection would be useful, I agree. It seems vital for the client to be able to request depth data, I would implement it as the client sending a request message for depth data, something like:

{ "op":"request-depth" }

And then the socket sends back the *complete* depth data in a *single message* in the same form that the web API currently returns it. In this way, once you receive the complete table you know you have a reliable dataset to perform updates to as they arrive.

The obvious caveat of this is that once you send the complete depth data, it must act as a "barrier", you can't send the complete state, and then an update to a stale state. (does this make sense?)

Mtgox currently has multiple bugs in the websocket's depth channel...your tables are behaving exactly the same as mine are.  Mtgox is moving to a new system in june which will hopefully have all of these problems taken care of.
martin
Full Member
***
Offline Offline

Activity: 150



View Profile WWW
May 19, 2011, 03:45:30 PM
 #55

Ah excellent, any information on the form of this new system?

zelyony
Newbie
*
Offline Offline

Activity: 23



View Profile
May 22, 2011, 01:25:33 PM
 #56

I wrote some code in C# for MonoDevelop or Visual Studio
MtGox & IMtGoxHandler & TestMtGox - for making orders
  FetchCash
  FetchOrders
  Place/CancelBuyOrder
  Place/CancelSellOrder
  .Handler - interface for responses & errors
WebSocket & Program - for read trades events
  .OnData - delegate received string (in JSON format) for any channel

https://rapidshare.com/files/3646016978/MtGoxApp.zip
zelyony
Newbie
*
Offline Offline

Activity: 23



View Profile
May 22, 2011, 04:38:11 PM
 #57

There's a new MTGox websocket API. This API works by subscription to channels, and each channel is represented by an UUID.
...

imho
It will be good to send 10-30 best Sell and Buy depths one new connection after subsribe to depth channel : price, type, ABSOLUTE-volume
After that depth data sent in relative manner: +volume, -volume
It gives knowledge to robots(or men) of current volumes for asks & bids.

technically:
JSON changes to this: add new attr to "depth": "abs"=X: where X = "1" for absolute volume; "0" or absent attr - relative volume as now
That data can be cashed on server for all new clients. Depth changes updates cache. Need to generate JSON (for sent) on request only - more expansive operation than just update some structs in cache.

PS
Sorry for my Russian English Smiley
zelyony
Newbie
*
Offline Offline

Activity: 23



View Profile
May 22, 2011, 05:09:17 PM
 #58

maybe
needs some authorization vs WebSockets' flood from bag guys (direction: from client to server)

something like this:
- some cookie received from Trade API for websocket connection to MtGox. this cookie valid only next hour (5-10-30min)
- on new connection to websocket client must send this cookie to server. if cookie is valid automatic subscribing will do..
cons: more difficult process for connection

or:
- using SSL/TLS encrypted WebSocekts.
pro: can refuse polling Trade API and do all orders and receive statuses over WS
cons: need implementations for different languages
zelyony
Newbie
*
Offline Offline

Activity: 23



View Profile
May 22, 2011, 09:27:32 PM
 #59

volumes and prices sometimes received as :X.YYYYYYYYYYY
who needed so many digits after dot?
for price enough.. 2 digits for cents.. and.. ok, let be 4digits after dot
for volume.. hmm.. similar numbers less than 0.001 are considered as zero.. = so 3 digits after dot for volumes
martin
Full Member
***
Offline Offline

Activity: 150



View Profile WWW
May 23, 2011, 02:08:23 PM
 #60

There's a new MTGox websocket API. This API works by subscription to channels, and each channel is represented by an UUID.
...

imho
It will be good to send 10-30 best Sell and Buy depths one new connection after subsribe to depth channel : price, type, ABSOLUTE-volume
After that depth data sent in relative manner: +volume, -volume
It gives knowledge to robots(or men) of current volumes for asks & bids.

I prefer my suggestion of a request packet which the client can send at any time to request the complete depth state, and then relative +volume -volume stream continues from there.

volumes and prices sometimes received as :X.YYYYYYYYYYY
who needed so many digits after dot?

We need as many digits as people can specify, which as far as I know is an unlimited amount.

Pages: « 1 2 [3] 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!