Bitcoin Forum

Economy => Trading Discussion => Topic started by: Clark on June 10, 2011, 07:13:11 AM



Title: Real Time MtGox Order Book
Post by: Clark on June 10, 2011, 07:13:11 AM
The second installment of WebSockets-enabled MtGox market data is here:

http://bitcoin.clarkmoody.com/ (http://bitcoin.clarkmoody.com/)

Update: MtGox now broadcasts with socket.io, which provides increased compatibility and maintains real time data streaming.

You should be able to view the order book in real time, with all bid and ask entries posted. The price grouping feature in the Controls section allows you to view the size of all orders within the corresponding price range. You may select a few different intervals on which to group prices. The service is set to show up to 1000 entries on each side of the market.

There is a calculator at the top of the screen that allows you to specify how many BTC you want to sell or USD you want to spend, and it tells you what the estimated outcome of that trade would be (in a market order sense).

The slippage field is a measure of how much profit you lose by selling at the market. For instance if the inside bid is for 10.00 and I have 100 BTC to sell, I should make $1000, right? Wrong. You only make $1000 if there are 100 BTC on that inside bid. If not, you have experience slippage due to accepting lower prices for the BTC greater than the size of the inside bid.

Please enjoy, and remember to donate ;) It keeps me motivated.



Title: Re: Real Time Order Book with WebSockets
Post by: molecular on June 10, 2011, 10:27:27 AM
http://bitcoin.clarkmoody.com/order-book/ (http://bitcoin.clarkmoody.com/order-book/)

nice!

When I looked at the site, I thought I'm seeing my own python app.

https://i.imgur.com/ixYzF.png

very similar display of depth data.

One differency strikes me, though: you're ordering the asks the other way around than me.

I think having the lowest ask at the bottom makes sense: You have the bids coming down on the left and the asks going back up on the right. So it's shaped like a "U" with the lowest point of the U being where the action is.

So by looking at the last line, you can already tell what the lowest ask and highest bids are. With your ordering, I have to look at the last and the first line.. confuses the hell out of me. While that's probably because I'm used to my own ordering, I would still like to suggest you think about this and maybe change it.

Thanks for your work, Clark, it's really pretty cool.


Title: Re: Real Time Order Book with WebSockets
Post by: ColdHardMetal on June 10, 2011, 10:31:20 AM
Some of the numbers in the "Sum" columns are white, while the rest are gray. Does that mean anything?

EDIT: Never mind, those are on the round price numbers 26,27 etc.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on June 10, 2011, 02:40:35 PM

One differency strikes me, though: you're ordering the asks the other way around than me.

I think having the lowest ask at the bottom makes sense: You have the bids coming down on the left and the asks going back up on the right. So it's shaped like a "U" with the lowest point of the U being where the action is.

So by looking at the last line, you can already tell what the lowest ask and highest bids are. With your ordering, I have to look at the last and the first line.. confuses the hell out of me. While that's probably because I'm used to my own ordering, I would still like to suggest you think about this and maybe change it.


I based my ordering on two things. First, I wanted to list all of the orders I got (up to 1000), and it would be really inconvenient to scroll all the way to the bottom to see the strike price. Second, I have only ever seen an order book listed with strike prices at the top (on professional trading software).

I'm glad you like it!


Title: Re: Real Time Order Book with WebSockets
Post by: bitoption on June 10, 2011, 02:46:09 PM
Clark, looks really nice!



Title: Re: Real Time Order Book with WebSockets
Post by: AtomicTrader on June 10, 2011, 03:10:01 PM
What would be really cool is a graphical representation of the order book, sort of like an equalizer on a stereo.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on June 10, 2011, 03:22:29 PM
What would be really cool is a graphical representation of the order book, sort of like an equalizer on a stereo.

Others have already done this. I don't know the links right now, but you could search for them on the forum.


Title: Re: Real Time Order Book with WebSockets
Post by: bitoption on June 10, 2011, 06:57:31 PM
Websockets are down. I am sad.


Title: Re: Real Time Order Book with WebSockets
Post by: zhalox on June 10, 2011, 07:04:55 PM
Yes, for some reason all my websockets are down, and my live SierraChart feed stopped working too, right when the Mt. Gox prices crashed.  The Mountain of Gox has fallen, and I am sad too :'(


Title: Re: Real Time Order Book with WebSockets
Post by: wahbasah on June 10, 2011, 07:07:30 PM
~

edit: oops.. appearently websockets are down


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on June 14, 2011, 05:24:55 AM
I've gone in and updated the code to be more careful about keeping the depth table honest. Some earlier versions seemed to let a series of quick trades get the depth table all out of whack (since the WebSocket sends deltas instead of absolute sizes at each price).

So you should see some 'Fixing depth table.' messages in the console when a ticker update comes in that disagrees with the current inside bids and asks.


Title: Re: Real Time Order Book with WebSockets
Post by: jaybny on June 15, 2011, 01:29:41 PM
anyway to see the full book all the way up and all the way down?

ty


Title: Re: Real Time Order Book with WebSockets
Post by: aandreas on June 15, 2011, 03:51:43 PM
hi clark. i stumbeld upon ur time & sales site a few days ago. tried it several times in the last days, but doesnt´t show anything. same with the new time & sales site. it says opening websocket, and about 10 sec later websocket closed.

my sierrachartfeed is running the whole time.

is there a regulation of users on ur site, or something? or some geoip thing? i am from germany.

or is it because i only use chrome for this and mtgoxlive and google wants me to open a gmailaccount andor browse other sites so they can track me  ;D


Title: Re: Real Time Order Book with WebSockets
Post by: YoYa on June 15, 2011, 07:38:02 PM
Clark,

Just to say thanks, your page is the BIZ!

Have sent a small donation already, will be happy to do so again should business be good.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on June 15, 2011, 07:42:03 PM
@jaybny: I have added a select box control that allows you to change the number of rows of the depth table displayed, all the way up to 1000. This should allow you to see the whole table, potentially at the expense of JavaScript performance.

@aandreas: It seems that you might be behind some sort of transparent proxy that cannot handle the WebSocket protocol handshake properly. There are no regulations from me on this. As I've pointed out other places, if MtGox implements the Secure WebSockets protocol, it will fix many of these compatibility issues.

@YoYa: Thank you for the donation!


Title: Re: Real Time Order Book with WebSockets
Post by: hlksis on June 17, 2011, 10:26:20 AM
Nice project. Which technology is behind your online order book? Are you using something like Socket.IO (+node.JS)?


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on June 17, 2011, 03:56:39 PM
I am not using Socket.IO. I simply used the built-in WebSocket object and jQuery to run everything else.


Title: Re: Real Time Order Book with WebSockets
Post by: cronopio on June 18, 2011, 04:12:55 PM
Hi Clarck

First, Thanks for that great site!!!

Second, Today i cant load the orders list. In the console bottom said "Buffering Ask" or "Buffering Bid". What this buffering thing?

I leave the window open for a while if at some time filled the buffer and showed me something but nothing.

Are you doing new improvements??

Thanks!!


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on June 18, 2011, 05:11:00 PM
There is some IP switching going on with the server, so after that finishes it should be back.


Title: Re: Real Time Order Book with WebSockets
Post by: cronopio on June 18, 2011, 05:15:21 PM
Great!!! Thanks a lot.

Thanks for your quick reply!!


Title: Re: Real Time Order Book with WebSockets
Post by: zhalox on June 27, 2011, 01:47:39 AM
Just noticed that the bids in the order book are upside-down - I had to display 1000 rows, and the highest bid was on the bottom of the list.  Did Mt. Gox change their websocket format?  On another note, I noticed that my live sierrachartfeed-0.2 plugin stopped working too, and will not stream live, but only shows historical data (I believe it also uses the Mt. Gox websockets by default).

Btw, I have another question, if I want to donate to you again, do you prefer the address in your signature, or the address displayed at the bottom of http://bitcoin.clarkmoody.com/order-book/ ?


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on June 27, 2011, 02:47:02 AM
Yeah, Gox changed the order in which the bids come over the JSON getDepth call. I didn't check for ordering, so that's why it's backwards. It should be fixed now.

As you have also noticed, the WebSockets are not working properly. The handshake happens just fine, but the socket doesn't send any data. Hopefully this is an indication that the WebSocket stuff is being upgraded. But one can only hope.

Feel free to donate to any of my addresses! And thank you very much for donating already!


Title: Re: Real Time Order Book with WebSockets
Post by: zhalox on June 27, 2011, 03:41:20 AM
Great, I'm glad to see it fixed already!  You're welcome, I've just sent 0.25 BTC :)

P.S. I see from your site/Twitter that you are a Christian - I am too!  A friend of mine at the Christian college I attend informed me about Bitcoin, and we began mining with our graphics cards together.  He is now a more serious Bitcoin miner, with several Radeon cards running.  As for me, I've only got a single 5870, but it's already paid itself off in BTC, plus extra :)


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on June 27, 2011, 05:06:51 AM
Awesome!  :)


Title: Re: Real Time Order Book with WebSockets
Post by: RandyMarsh on June 27, 2011, 11:47:07 AM
Since mtgox came back online the order book aint updating and the buy column is upsidedown :-(


Title: Re: Real Time Order Book with WebSockets
Post by: moneta on June 28, 2011, 09:25:32 PM
All websockets based feeds are broken ATM.

On the plus side, I am using this order book as a non-real-time service, manually reloading the page... it's a lot faster than MtGox own non-real-time order book.
Yay!


Title: Re: Real Time Order Book with WebSockets
Post by: stic.man on June 30, 2011, 02:38:31 AM
any word on the time and sales chart?  It's like heroin


Title: Re: Real Time Order Book with WebSockets
Post by: RandyMarsh on July 10, 2011, 02:55:52 PM
All websockets based feeds are broken ATM.

On the plus side, I am using this order book as a non-real-time service, manually reloading the page... it's a lot faster than MtGox own non-real-time order book.
Yay!

Its back realtime informative as ever! Thanks Clark! donation sent


Title: Re: Real Time Order Book with WebSockets
Post by: Bitcoin Swami on July 10, 2011, 07:03:49 PM
Maybe I'm not reading it right but the prices seem to be off.   Right now mt. gox is trading at 15 and this shows last trade at 15.11

EDIT nevermind its just super fast and sees into the future.  Awesome


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on July 10, 2011, 09:58:01 PM
I'm glad to see that it's back up and running!

As always, thanks for the donations!

I also updated the display of non-significant zeros in the size column. They are now very dark, so the view should be a little more like aligning to the decimal point.



Title: Re: Real Time Order Book with WebSockets
Post by: jjiimm_64 on July 11, 2011, 06:16:55 AM
I get::::

Your Browser does not Support WebSockets

Please try a better browser. You can get help here.


I am using firefox 5.  how much better can you get?


Title: Re: Real Time Order Book with WebSockets
Post by: MagicalTux on July 11, 2011, 07:29:05 AM
I get::::

Your Browser does not Support WebSockets

Please try a better browser. You can get help here.


I am using firefox 5.  how much better can you get?

Chrome, Firefox 6.0, MSIE+HTML Labs, Safari...

Details: http://en.wikipedia.org/wiki/WebSockets#Browser_support



Title: Re: Real Time Order Book with WebSockets
Post by: Clark on July 11, 2011, 02:51:25 PM
We have devised some solutions in the Time & Sales thread here: http://forum.bitcoin.org/index.php?topic=11560.0

It covers Opera & Firefox and enabling WebSockets for those browsers.


Title: Re: Real Time Order Book with WebSockets
Post by: dserrano5 on July 11, 2011, 03:07:31 PM
Here are a couple of features/changes that come to mind. I'm only sharing them so they don't die inside my head.

  • Option to not show all the Add/Rem Bid/Ask in the console. Personally I'm not interested in those.
  • Ability to click on a row to highlight it, like "I have an order at this point" (e.g. with a brighter background). Auto-remove the highlight only when that row has completely been fulfilled. My particular order may be long gone by then, but there's no way that can be detected.
  • Don't highlight round numbers, but big (≥2000Ⓑ?) orders instead.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on July 11, 2011, 03:36:48 PM
I like those ideas.

I'm thinking that it might be nice to simply add your MtGox trade api token and subscribe to your particular order feed on the WebSocket. Then it would be very easy to highlight events related to your orders (though there would be no persistence between sessions).

As a side note, I am currently working on a far more sophisticated real time quotes system that everyone will enjoy. Please stay tuned for developments.

In the mean time I will probably launch a side-by-side order book and time & sales so you can close the console altogether.


Title: Re: Real Time Order Book with WebSockets
Post by: DrKennethNoisewater on July 11, 2011, 05:25:36 PM
http://bitcoin.clarkmoody.com/order-book/ (http://bitcoin.clarkmoody.com/order-book/)

nice!

When I looked at the site, I thought I'm seeing my own python app.

https://i.imgur.com/ixYzF.png

very similar display of depth data.

One differency strikes me, though: you're ordering the asks the other way around than me.

I think having the lowest ask at the bottom makes sense: You have the bids coming down on the left and the asks going back up on the right. So it's shaped like a "U" with the lowest point of the U being where the action is.

So by looking at the last line, you can already tell what the lowest ask and highest bids are. With your ordering, I have to look at the last and the first line.. confuses the hell out of me. While that's probably because I'm used to my own ordering, I would still like to suggest you think about this and maybe change it.

Thanks for your work, Clark, it's really pretty cool.


This is looking good, the way it should be.

Offers on right, bids on left - Highest bid should be on top, lowest offer on top as well, just like a stock......add the TOS and we'll be off to the races.....

Keep up the great work!

DKN


Title: Re: Real Time Order Book with WebSockets
Post by: MagicalTux on July 11, 2011, 10:28:38 PM
I like those ideas.

I'm thinking that it might be nice to simply add your MtGox trade api token and subscribe to your particular order feed on the WebSocket. Then it would be very easy to highlight events related to your orders (though there would be no persistence between sessions).

As a side note, I am currently working on a far more sophisticated real time quotes system that everyone will enjoy. Please stay tuned for developments.

In the mean time I will probably launch a side-by-side order book and time & sales so you can close the console altogether.

If you need details on the future API and have preferences/requests on the way we push data on the stream, please contact me at admin@mtgox.com !


Title: Re: Real Time Order Book with WebSockets
Post by: Dom on July 15, 2011, 04:44:35 AM
Hi Clark, I just wanted to say the recent improvements to Order Book are quite nice. Thanks for your effort. I'll throw some bitchange your way on my next miner dump.


Title: Re: Real Time Order Book with WebSockets
Post by: GeniuSxBoY on July 23, 2011, 10:57:08 PM
Is the site down?


Title: Re: Real Time Order Book with WebSockets
Post by: Oldminer on July 30, 2011, 07:54:29 PM
Yup


Title: Re: Real Time Order Book with WebSockets
Post by: MagicalTux on July 31, 2011, 05:02:41 PM
I'd suggest ignoring depth updates for currency != USD for now, as it messes up the view :)


Title: Re: Real Time Order Book with WebSockets
Post by: Oldminer on August 05, 2011, 08:59:40 AM
site's down :(


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 05, 2011, 03:01:18 PM
And by 'site down,' do you mean that
  • My website failed to load?
  • Or the WebSocket would never connect to give any data?

It would be nice to have the details so I know if there's anything I can do about the problem.

It looks fine to me at this point though. I'm glad to see people finding these tools useful  :)


Title: Re: Real Time Order Book with WebSockets
Post by: Oldminer on August 05, 2011, 03:09:21 PM
Well the site came back up about 30 mins after I posted. From memory it was a time-out (page not found) error.


Title: Re: Real Time Order Book with WebSockets
Post by: toffoo on August 06, 2011, 03:18:42 AM
And by 'site down,' do you mean that
  • My website failed to load?
  • Or the WebSocket would never connect to give any data?

It would be nice to have the details so I know if there's anything I can do about the problem.

It looks fine to me at this point though. I'm glad to see people finding these tools useful  :)

I saw that it was down last night as well.  The entire clarkmoody.com domain wasn't responding for a few hours.  Mt.Gox's websocket feed actually goes down quite often, but this was the first time I'd seen your entire site down.

Clark, your Real Time Order Book page is just awesome (as are your other Bitcoin pages).  I tried to program something like that myself a few months ago and couldn't come close.  Your page is exactly what I was trying to achieve and I failed miserably.  If I were Mr. Gox I would offer you a job.

Your work is definitely deserving of a donation because now I cannot do a trade without it.  When I'm not feeling so Bitcoin poor I will certainly send you something.


Title: Re: Real Time Order Book with WebSockets
Post by: indicasteve on August 06, 2011, 06:16:07 AM
Hmm...

I wanted to try the site, but apparently firefox 4 and 5 have websockets disabled because it could be a security threat.

http://hacks.mozilla.org/2010/12/websockets-disabled-in-firefox-4/ (http://hacks.mozilla.org/2010/12/websockets-disabled-in-firefox-4/)

I'm sure the site is safe, but I don't really understand the websockets technology enough yet, so me thinks I will keep it disabled for now.





Title: Re: Real Time Order Book with WebSockets
Post by: Oldminer on August 06, 2011, 06:38:33 AM
Hmm...

I wanted to try the site, but apparently firefox 4 and 5 have websockets disabled because it could be a security threat.


Get Chrome


Title: Re: Real Time Order Book with WebSockets
Post by: indicasteve on August 06, 2011, 07:55:51 AM
Hmm...

I wanted to try the site, but apparently firefox 4 and 5 have websockets disabled because it could be a security threat.


Get Chrome

From what I've read so far, it's not the browser that's the problem...it's the protocol.  Or rather it's how 'transparent proxies' are vulnerable to cache exploits that don't handle the protocol properly.

I can edit my firefox config and enable it, but more research is needed before I will.


Title: Re: Real Time Order Book with WebSockets
Post by: thinkingBTC on August 06, 2011, 05:07:45 PM
I think Chrome 12 (stable) was the last version to enable websockets, although I'm not 100% sure (the beta might too). I'm running Chrome Canary (15) and it's disabled for security reasons. I just open it up in Chrome 12.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 10, 2011, 06:55:36 PM
Clark, your Real Time Order Book page is just awesome (as are your other Bitcoin pages).  I tried to program something like that myself a few months ago and couldn't come close.  Your page is exactly what I was trying to achieve and I failed miserably.  If I were Mr. Gox I would offer you a job.

Thanks very much!

As a side note, I have now added a real-time calculator to the order book. A longer description is in the first post in this thread.

Enjoy!


Title: Re: Real Time Order Book with WebSockets
Post by: zhalox on August 11, 2011, 12:58:26 AM
Thanks for the update with the calculator feature, I just sent you another 0.50 BTC.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 11, 2011, 06:15:40 PM
Thanks for the update with the calculator feature, I just sent you another 0.50 BTC.

Awesome!

I think people will find it very useful.


Title: Re: Real Time Order Book with WebSockets
Post by: YoYa on August 12, 2011, 11:42:07 PM
Clark, I remember watching you time sales page during the boom and the bust in July, never seen anything so cool in my life as watching those numbers fly. The order book is even more awesome when there is tonnes of activity, the calculator is just the cherry on the cake!

Thanks,
Will send on some btc when I buy some next.


Title: Re: Real Time Order Book with WebSockets
Post by: DrKennethNoisewater on August 13, 2011, 02:24:03 AM
This is really cool.



Title: Re: Real Time Order Book with WebSockets
Post by: GeniuSxBoY on August 14, 2011, 04:53:21 AM
Hmm, can you please remove the calculator and put it on a separate page that you can get to by a link on the page.


Ever since it was added, the website freezes waaaay more than it used to.  I can't load the 500 or 1000 views anymore.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 14, 2011, 08:02:02 PM
Hmm, can you please remove the calculator and put it on a separate page that you can get to by a link on the page.


Ever since it was added, the website freezes waaaay more than it used to.  I can't load the 500 or 1000 views anymore.

Here are some numbers that I gathered about the time (milliseconds) it takes to process a depth message. The calculator, when active, is set on selling 100 BTC and spending 100 USD

RowsCalcNo Calc
5039.737.7
10082.181.7
250350.3346.7
500832.2778.1

As you can see, the numbers do not reflect a significant processing overhead for the calculator. However, I believe I could be more efficient when computing the sums all the way down the book.

You must keep in mind that MtGox sends messages out much faster than this page can display them. I believe the WebSocket gives on the order of 10-20 Hz data.

Here's a few questions:

How important is the sum column to you, especially at 500+ rows? Are you planning on releasing a 10000+ BTC buy any time soon (and will you let me know when you do)?

How exactly do you use the book when set at those high display counts? What are you looking at far away from the strike price?

I'd really like to make this thing more efficient, so any help will be appreciated. And I'll go look into making this code more efficient.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 14, 2011, 08:40:33 PM
Ok, well I messed with the code a little bit, and I've eliminated a very inefficient jQuery call. Here is the updated table from my last post:

RowsCalcNo Calc
5010.19.3
10019.117.2
25047.145.7
50073.672.1

The new numbers are much more acceptable at the high row counts, especially considering that the book used to take almost a second (~800ms) to process a single message when displaying 500 rows.

GeniuSxBoy, thanks for pointing out how shamefully slow the update rate was and making me get in gear to produce more efficient code! The new code is now live.

Hint: for jQuery users, this is the power of the $.each() operator at work.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 15, 2011, 12:55:03 AM
New Feature Announcement:

I have completed implementation of price grouping on the order book. It is available under the Controls section. You may group by 0.01, 0.05, 0.10, and 0.25 ($/BTC). This range may always be expanded depending on demand.

I think this feature will enable much greater awareness of the order book as a whole, similar to the graphical order books you see on other sites. With prices grouped, the order book is not a jumpy as before since row entries are not being added and removed that often.

Enjoy, and send bug reports (and donations)!


Title: Re: Real Time Order Book with WebSockets
Post by: GeniuSxBoY on August 15, 2011, 02:35:36 AM
Site is working like a charm, thank you :)


Title: Re: Real Time Order Book with WebSockets
Post by: Kermee on August 15, 2011, 02:40:07 AM
Anyone using Chrome 14 Beta having issues?  Going from 12 to 14 seems to have broken WebSockets for me :(

Cheers,
Kermee


Title: Re: Real Time Order Book with WebSockets
Post by: toffoo on August 15, 2011, 08:24:34 AM
Hi Clark,

It looks like one of your very recent changes has altered the console display at the bottom of the screen.  I can't tell if this is a bug of some kind or if you changed this on purpose, but I no longer see the order ADD/REMOVE websocket messages scrolling by in the console, only trades and a few other info messages.

I'd prefer to see them, or ideally have the option to display them.

Thanks again for this great page.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 15, 2011, 02:31:30 PM
It looks like one of your very recent changes has altered the console display at the bottom of the screen.

I'd prefer to see them, or ideally have the option to display them.

Yes, I took those away, with the intent of adding an option, but the option never happened. Until now.

Enjoy.


Title: Re: Real Time Order Book with WebSockets
Post by: toffoo on August 16, 2011, 04:10:53 AM
Yes, I took those away, with the intent of adding an option, but the option never happened. Until now.

Truly awesome.  Looks perfect now.
You sure know how to milk me, and I'm feeling less Bitcoin-poor today, so donation sent! to: 184FgjfDC2JB3EDWfPpYrv6uX77FzfNE5k

Thanks again Clark, please don't ever take this page down.   ;)


Title: Re: Real Time Order Book with WebSockets
Post by: dserrano5 on August 17, 2011, 05:36:53 AM
I would gladly store a cookie in my browser so that my preferences (10-line console, no visible calculator, no visible controls, group by 0.01) were remembered ;).


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 17, 2011, 05:49:49 AM
I was just thinking about that as I clicked all of my preferred choices multiple times today. Stay tuned.

Edit: Of course you could also 'sponsor' the default configuration of the site  ;)


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 17, 2011, 07:14:12 AM
Ok. All settings are now saved transparently in a cookie and loaded on startup.

This, of course, doesn't mean that it's too late to sponsor the default settings for newcomers.

And thanks toffoo for the great donation!


Title: Re: Real Time Order Book with WebSockets
Post by: biohazord on August 18, 2011, 03:26:19 AM
Anyone using Chrome 14 Beta having issues?  Going from 12 to 14 seems to have broken WebSockets for me :(

Cheers,
Kermee

I'm having the same issue one two different PCs both running Chrome 14.0.835.94 Beta-m. I'm also having the issue with the canary channel build which is currently version 15.0.855.0 canary.

Currently the site works in Firefox 5 after changing the about:config setting network.websocket.override-security-block so that's what I'm using. The same setting does not work in FF6 so don't upgrade.

Heres a quick run down on my windows box.
Not working in...
Chrome 14.0.835.94 Beta-m
Chrome 15.0.855.0 canary
Firefox 6.0 (not even with network.websocket.override-security-block set to true)
Firefox Aurora 7.0a2 (2011-08-16)

Works in...
Firefox 5 (with network.websocket.override-security-block set to true)
Chrome 13.0.782.112 m


Title: Re: Real Time Order Book with WebSockets
Post by: Kermee on August 18, 2011, 04:58:41 AM
Anyone using Chrome 14 Beta having issues?  Going from 12 to 14 seems to have broken WebSockets for me :(

Cheers,
Kermee

I'm having the same issue one two different PCs both running Chrome 14.0.835.94 Beta-m. I'm also having the issue with the canary channel build which is currently version 15.0.855.0 canary.

Currently the site works in Firefox 5 after changing the about:config setting network.websocket.override-security-block so that's what I'm using. The same setting does not work in FF6 so don't upgrade.

Heres a quick run down on my windows box.
Not working in...
Chrome 14.0.835.94 Beta-m
Chrome 15.0.855.0 canary
Firefox 6.0 (not even with network.websocket.override-security-block set to true)
Firefox Aurora 7.0a2 (2011-08-16)

Works in...
Firefox 5 (with network.websocket.override-security-block set to true)
Chrome 13.0.782.112 m

Ah. Good to know about FFV.  Thanks.

I looked at the Dev logs... WebSockets was 'rehauled' in 14... So I'm not surprised it might of broken some things :(

Cheers,
Kermee


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 18, 2011, 05:44:28 AM
I think MtGox is using an older version of the WebSocket protocol, which may or may not play a role in the incompatibility.

If the data came in over Secure WebSockets with a newer protocol, then I bet the compatibility would be more widespread.


Title: Re: Real Time Order Book with WebSockets
Post by: klaus on August 21, 2011, 03:40:24 PM
@Clark
send you donation for your great tool


Title: Re: Real Time Order Book with WebSockets
Post by: oakpacific on August 23, 2011, 06:18:39 PM
Not working with Firefox 6, for which Websocket is enabled by default.


Title: Re: Real Time Order Book with WebSockets
Post by: Kermee on August 23, 2011, 08:12:36 PM
Not working with Firefox 6, for which Websocket is enabled by default.

MtGox needs to update their WebSockets 'stack'...

Cheers,
Kermee


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 23, 2011, 08:23:13 PM

MtGox needs to update their WebSockets 'stack'...

Cheers,
Kermee

I couldn't have said it better myself.


Title: Re: Real Time Order Book with WebSockets
Post by: GeniuSxBoY on August 26, 2011, 05:12:30 AM
LoL....It's me again. Another bug is making itself shown


The Websocket is too sensitive and keeps disconnecting for short periods of time. I used to be able to keep it open all day, now I have to reconnect every 15 minutes or so... sometimes after just 5 minutes.


Title: Re: Real Time Order Book with WebSockets
Post by: Kermee on August 26, 2011, 05:30:11 AM
LoL....It's me again. Another bug is making itself shown


The Websocket is too sensitive and keeps disconnecting for short periods of time. I used to be able to keep it open all day, now I have to reconnect every 15 minutes or so... sometimes after just 5 minutes.

That's MtGox's side...

Cheers,
Kermee


Title: Re: Real Time Order Book with WebSockets
Post by: osmosis on August 26, 2011, 09:01:27 PM
It wasnt working in Chrome in win for a google week. Websocket was failing. Im glad its back now.

Any plans to add support for other exchanges? Id love to see ExchB and Tradehill as options as well.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 26, 2011, 11:19:13 PM
The main advantage of this whole thing is that it works on the WebSocket, which no other exchanges provide.

I am working on a few products right now that will make trading on multiple exchanges way easier. Stay tuned!


Title: Re: Real Time Order Book with WebSockets
Post by: S3052 on August 27, 2011, 09:35:46 PM
Clark,

your site is just great.

One question on the depth of the order book on MtGox. Can you display the full order book or is it truncated?
If it is truncated, it should have no orders above and below a certain level.

The reason why I am asking is that there are only very small bids below 6 $ and only small asks above 12 $. What is the truth?

http://bit.ly/p1xOoc



Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 28, 2011, 12:29:35 AM

One question on the depth of the order book on MtGox. Can you display the full order book or is it truncated?
If it is truncated, it should have no orders above and below a certain level.

The reason why I am asking is that there are only very small bids below 6 $ and only small asks above 12 $. What is the truth?


Here is what happens:

Mt.Gox does not stream the full order book over the WebSocket; it streams only the changes as they come in. Therefore, the site polls the normal API at getDepth.php to get the 'whole' book. The problem is that the call to that API address seems to return only those orders within +/- $2 of the strike price. This is what you see when the order book loads initially.

As new orders come in, some lie outside of the initial 'full' book, so you get that light volume in the fringes of the book. There really is no way at this point to get an accurate representation of the entire order book.


Title: Re: Real Time Order Book with WebSockets
Post by: S3052 on August 28, 2011, 12:36:50 AM
Thanks for the clear and prompt answer, and.. keep your great service up!


Title: Re: Real Time Order Book with WebSockets
Post by: Tasty Champa on August 28, 2011, 02:20:59 AM
@Clark, do you have any advice or directional links to code or tuts for someone who wants to explore making charts similar to these?


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 28, 2011, 02:35:45 AM
@Clark, do you have any advice or directional links to code or tuts for someone who wants to explore making charts similar to these?

Learn jQuery (http://jquery.com/) and get familiar with the WebSocket in JavaScript. You can look on the Bitcoin Wiki for details on the messages that MtGox broadcasts on this page (https://en.bitcoin.it/wiki/MtGox/API).


Title: Re: Real Time Order Book with WebSockets
Post by: Tasty Champa on August 28, 2011, 02:56:45 AM
@Clark, do you have any advice or directional links to code or tuts for someone who wants to explore making charts similar to these?

Learn jQuery (http://jquery.com/) and get familiar with the WebSocket in JavaScript. You can look on the Bitcoin Wiki for details on the messages that MtGox broadcasts on this page (https://en.bitcoin.it/wiki/MtGox/API).

Thanks Clark!
Also, one last question if possible, I've been looking into Node.js, is that a wise direction?


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 28, 2011, 02:58:11 AM

Also, one last question if possible, I've been looking into Node.js, is that a wise direction?



I have no experience with Node.js, so I couldn't give you an answer one way or the other.


Title: Re: Real Time Order Book with WebSockets
Post by: cronopio on August 28, 2011, 06:00:32 PM
Yeah, Node.js its ideal for this. The problem is the API its a websocket interface from MtGox.

Node.js work perfectly only if mtgox be rewritten in Node.js

Here you have the relay on MtGox plataform.


Title: Re: Real Time Order Book with WebSockets
Post by: S3052 on August 29, 2011, 11:09:00 PM

Here is what happens:

Mt.Gox does not stream the full order book over the WebSocket; it streams only the changes as they come in. Therefore, the site polls the normal API at getDepth.php to get the 'whole' book. The problem is that the call to that API address seems to return only those orders within +/- $2 of the strike price. This is what you see when the order book loads initially.

As new orders come in, some lie outside of the initial 'full' book, so you get that light volume in the fringes of the book. There really is no way at this point to get an accurate representation of the entire order book.


Hi again, there is still some discrepancy between your order book completeness vs. MtGox (see here: https://mtgox.com/trade/history (https://mtgox.com/trade/history))

If you click on Market depth rounded, you see that the bids and ask go further than on your site. Any idea why? I remember it was better on your site days before. Anything changed?


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on August 29, 2011, 11:40:32 PM
Unfortunately, I am at the mercy of the getDepth API call. The data you see on my site comes from MtGox as-is.

My site reflects the most recent output of the MtGox API, which changes frequently, as you've noticed.


Title: Re: Real Time Order Book with WebSockets
Post by: S3052 on August 30, 2011, 04:50:53 AM
thanks.
I wish mtgox would deliver better data


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on September 12, 2011, 02:19:04 AM
I have launched the Time & Sales and Order Book in a combined product. Read about it here (https://bitcointalk.org/index.php?topic=43514.0).

Or visit http://bitcoin.clarkmoody.com/

Enjoy!


Title: Re: Real Time Order Book with WebSockets
Post by: S3052 on September 19, 2011, 11:43:07 AM
This is great! Wonderful.

Just one question: Is there an issue with the connection at the moment. In my case, it disconnects within a split second after hitting the connect button.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on September 19, 2011, 02:38:44 PM
Please see my latest post here: https://bitcointalk.org/index.php?topic=43514.msg533997#msg533997

I address these latest connection issues there.


Title: Re: Real Time Order Book with WebSockets
Post by: DrKennethNoisewater on September 19, 2011, 11:03:52 PM
For one reason or another, gox and crew are dragging there feet on this 1.

I hope they're not sand bagging because they're going to launch a competing product and lock up the api.


Title: Re: Real Time Order Book with WebSockets
Post by: molecular on September 24, 2011, 11:57:36 PM
For one reason or another, gox and crew are dragging there feet on this 1.

I hope they're not sand bagging because they're going to launch a competing product and lock up the api.

I think thats unlikely. That would p.ss off a lot of people. The websocket is also used by bots and custom trading clients, for example. MagicalTux has had websocket on low priority, it seems, and it's been like that for a long time. It's more important to have the trading engine working first ;)


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on September 25, 2011, 01:14:30 AM
For one reason or another, gox and crew are dragging there feet on this 1.

I hope they're not sand bagging because they're going to launch a competing product and lock up the api.

I think thats unlikely. That would p.ss off a lot of people. The websocket is also used by bots and custom trading clients, for example. MagicalTux has had websocket on low priority, it seems, and it's been like that for a long time. It's more important to have the trading engine working first ;)


Yeah, the main issue here is the changing browser support of the WebSocket protocols. Python certainly has no problem connecting, so the bots aren't hurting.


Title: Re: Real Time Order Book with WebSockets
Post by: DrKennethNoisewater on October 11, 2011, 06:13:13 PM
Installed Safari 5.1.....

Socket Still Closes......do you add any plugins or anything after the initial install?



Title: Re: Real Time Order Book with WebSockets
Post by: klaus on October 27, 2011, 02:54:21 PM

Socket Closed.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on October 27, 2011, 03:00:59 PM

Socket Closed.

It looks like MtGox has taken the socket down for the last 8 hours or so.

For something a little different, you could try http://bitcoin.clarkmoody.com/ticker/


Title: Re: Real Time Order Book with WebSockets
Post by: molecular on October 27, 2011, 08:22:42 PM

Socket Closed.

It looks like MtGox has taken the socket down for the last 8 hours or so.

For something a little different, you could try http://bitcoin.clarkmoody.com/ticker/

you never fail to surprise.

Quote from: #mtgox
<molecular> is the websocket offline?
<MagicalTux> the routing is still down

didn't bug him any further


Title: Re: Real Time Order Book with WebSockets
Post by: molecular on November 10, 2011, 02:30:51 PM
some news: obviously websocket was replaced with socket.io

Quote from: #mtgox
<shockdiode> websocket is officially deprecated now?
<MagicalTux> shockdiode: yep, replaced by socket.io server
<MagicalTux> which is more stable
<MagicalTux> but not using the same protocol
<MagicalTux> client libs are however in most languages, however
<shockdiode> when will that be up?

<MagicalTux> [18:23:22] <shockdiode> when will that be up? <- IT IS
<molecular> MagicalTux, any doc/wiki/??? on socket.io server?

<MagicalTux> molecular: http://socket.io got a lot of doc
<MagicalTux> it's basically the same websocket data, passed through socket.io

<molecular> MagicalTux, thanks. so you merely switched the transport layer? everything else like before...?
<MagicalTux> molecular: yep, switched transport layer, which will allow us to make it work in any browser

EDIT: I might add I have hope it will also work through transparent proxies that have probs with websocket, will check it out soon with my python client


Title: Re: Real Time Order Book with WebSockets
Post by: MagicalTux on November 10, 2011, 02:49:32 PM
I should also add that it works with most browsers including MSIE 5.5+, and has various fallback mechanisms depending if websocket fails.

To use it:

Code:
<script type="text/javascript" src="https://socketio.mtgox.com/socket.io/socket.io.js"></script>
<script type="text/javascript">
var conn = io.connect('https://socketio.mtgox.com/mtgox');
conn.on('message', function(data) {
    var message = event.data;
    // do things with message (same format as before)
});
</script>

That should be simple enough, and is cross browser.


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on November 10, 2011, 06:10:20 PM
Thanks for working on this! It will be nice to see more browsers able to use the live quote services!


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on November 10, 2011, 08:30:20 PM
Awesome!

I've completed the transition of bitcoin.clarkmoody.com to the new socket.io interface.


Title: Re: Real Time Order Book with WebSockets
Post by: MagicalTux on November 11, 2011, 02:15:24 AM
Awesome!

I've completed the transition of bitcoin.clarkmoody.com to the new socket.io interface.

Hi,

Just a detail, now the data sent to the function is no longer a string with some json inside, but an object (it didn't make much sense to just send strings).

I didn't expect you'd have this implemented already, if you can just comment out the JSON.parse that'll fix everything.


Mark


Title: Re: Real Time Order Book with WebSockets
Post by: Clark on November 11, 2011, 02:16:57 AM
Just a detail, now the data sent to the function is no longer a string with some json inside, but an object (it didn't make much sense to just send strings).

I didn't expect you'd have this implemented already, if you can just comment out the JSON.parse that'll fix everything.

Actually I was!

I was just sitting here developing and I saw the messages change from strings to JSON!


Title: Re: Real Time MtGox Order Book
Post by: throughput on November 16, 2011, 10:55:18 AM
socket.io simply does not work atm.

sooner or later getting the "7:::1+0" message.
Firefox at  http://bitcoin.clarkmoody.com/
or handwritten socket.io application connection to https://socketio.mtgox.com/  - that does not matter.
Interesting, though, connection passes handshake stage and receives "connect" message.

Please, anyone possessing the knowledge about the "new technology", comment on what does the 1+0 message mean?

socket.io-spec says:
Quote
(7) Error

'7::' [endpoint] ':' [reason] '+' [advice]

For example, if a connection to a sub-socket is unauthorized.
just a nonsense.
BTW, I still have not found more documentation on the protocol.
OMFG. Is it undocumented?


Title: Re: Real Time MtGox Order Book
Post by: Clark on November 16, 2011, 04:04:28 PM
sooner or later getting the "7:::1+0" message.

Please, anyone possessing the knowledge about the "new technology", comment on what does the 1+0 message mean?

I have not seen this message, and I have a bit of experience with the socket.io interface now.


BTW, I still have not found more documentation on the protocol.
OMFG. Is it undocumented?

The protocol still follows the WebSocket protocol listed on the wiki (https://en.bitcoin.it/wiki/MtGox/API#Websocket_API). The incoming data are straight objects instead of JSON.


Title: Re: Real Time MtGox Order Book
Post by: eldentyrell on January 31, 2012, 02:35:43 AM
sooner or later getting the "7:::1+0" message.

Please, anyone possessing the knowledge about the "new technology", comment on what does the 1+0 message mean?

I have not seen this message, and I have a bit of experience with the socket.io interface now.

I am getting the "7:::1+0" as well.


Title: Re: Real Time MtGox Order Book
Post by: eldentyrell on January 31, 2012, 09:31:29 PM
I am getting the "7:::1+0" as well.

FWIW I am using the standard Java (not javascript) Socket.IO client (https://github.com/Gottox/socket.io-java-client).


Title: Re: Real Time MtGox Order Book
Post by: eldentyrell on February 02, 2012, 12:39:20 AM
Here is a self-contained piece of code that demonstrates the problem:

Code:
import io.socket.*;
import org.json.*;
import net.tootallnate.websocket.*;

public class MtGoxStreaming {
   
    private static final String tickerChannel = "d5f06780-30a8-4a48-a2f8-7ed181b4a13f";
    private static final String tradesChannel = "dbf1dee9-4f2e-4a08-8cb7-748919a71b21";
    private static final String depthChannel  = "24e67e0d-1cad-4cc0-9e7a-f8523ef460fe";

    public static void main(String[] args) throws Exception {
        SocketIO socket = new SocketIO("https://socketio.mtgox.com/mtgox");
        socket.connect(new IOCallback() {
                @Override
                    public void onMessage(JSONObject json, IOAcknowledge ack) {
                    try {
                        System.out.println("We received a message: " + json.toString(2));
                    } catch (JSONException e) {
                        e.printStackTrace();
                    }
                }

                @Override
                    public void onMessage(String data, IOAcknowledge ack) {
                    System.out.println("We received a message:" + data);
                }
               
                @Override
                    public void onError(SocketIOException socketIOException) {
                    System.out.println("Something went wrong: " + socketIOException);
                    socketIOException.printStackTrace();
                    //System.exit(0);
                }

                @Override
                    public void onDisconnect() {
                    System.out.println("Disconnected");
                    System.exit(0);
                }

                @Override
                    public void onConnect() {
                    System.out.println("Connected");
                }

                @Override
                    public void on(String event, IOAcknowledge ack, Object... args) {
                    System.out.println("got: " + event);
                    ack.ack("Roger that!");
                    //socket.emit("answer", new JSONObject().put("msg", "Hello again Socket.io!"));
                }
            });

        // This will be cached until the server is connected.
        //socket.emit("hello", new JSONObject().put("msg", "Hello Socket.io! :D"));
 
        JSONObject obj = new JSONObject();
        obj.put("op",      "subscribe");
        //obj.put("channel", depthChannel);
        obj.put("channel", "d5f06780-30a8-4a48-a2f8-7ed181b4a13f");
        socket.send(obj);
 
    }
}

Run it and you get:

Code:
Connected
> 1::/mtgox
> 4::/mtgox:{"op":"subscribe","channel":"d5f06780-30a8-4a48-a2f8-7ed181b4a13f"}
> 2::
> 2::
> 2::
> 2::
> 2::
> 2::
> 2::
< 7:::1+0
Something went wrong: io.socket.SocketIOException: 1+0
io.socket.SocketIOException: 1+0


Title: Re: Real Time MtGox Order Book
Post by: wareen on February 17, 2012, 11:08:40 PM
I am getting the "7:::1+0" as well.

FWIW I am using the standard Java (not javascript) Socket.IO client (https://github.com/Gottox/socket.io-java-client).
The problem was with the Java-WebSocket implementation. Gottox' socket.io recently switched to Weberknecht and it fixed the problem. Just pull the new version from github and you should be fine.


Title: Re: Real Time MtGox Order Book
Post by: brox on March 03, 2012, 07:41:50 PM
I am getting the "7:::1+0" as well.

FWIW I am using the standard Java (not javascript) Socket.IO client (https://github.com/Gottox/socket.io-java-client).
The problem was with the Java-WebSocket implementation. Gottox' socket.io recently switched to Weberknecht and it fixed the problem. Just pull the new version from github and you should be fine.

Man, you saved my life! It works like a charm now!


Title: Re: Real Time MtGox Order Book
Post by: TheFootMan on December 11, 2013, 09:13:26 PM
Is clark moody frozen?


Title: Re: Real Time MtGox Order Book
Post by: Clark on December 11, 2013, 11:35:58 PM
Is clark moody frozen?

Mt. Gox has disconnected its real time WebSocket.


Title: Re: Real Time MtGox Order Book
Post by: TrustedPartner on February 16, 2014, 10:43:08 AM
Anybody buying or selling within MtGox to other MtGox traders for real cash for MtGoxBTC to each other?  Any talk about that?