Real Time Charting, Order Book, and Time & Sales

[molecular]

Can you add weekly mouthly chart in you web?

week 1:

[Steve]

I have a suggestion...can you make it such that the width of the chart can be expanded?  We are putting a display in our offices (a sort of startup hotel in Atlanta where we'll get a lot of traffic) that we would like to show the Bitcoin candlestick chart on...clarkmoody is good for that, except when you put it on a large 1080p monitor, the chart doesn't take advantage of the full width of the display.  You can pull up the website full screen on any 1080p to get an idea of the wasted real estate on the sides.  Even if you had just a few different width options (small, medium, large), it would be great. 

Also, we're using a Raspberry Pi to drive the display...it pushes that little box hard...anything you can do to make the JavaScript more efficient would be good.  It works, but just barely (mtgoxlive.com won't work on it).

[Clark]

when you put it on a large 1080p monitor, the chart doesn't take advantage of the full width of the display.  You can pull up the website full screen on any 1080p to get an idea of the wasted real estate on the sides.

"Wasted" is such a harsh word 

Try this and see what you think: http://bitcoin.clarkmoody.com/widget/chart/

...anything you can do to make the JavaScript more efficient would be good.

Reduce the number of rows displayed on the depth table: one of the most intensive operations is computing the cumulative sum and updating the cells in the sum columns.

The launch of RTBTC will provide a new chart package that has gotten a lot of development attention from me in recent months. You will be able to create charts for any screen size with the platform. A little preview of an older version: https://twitter.com/RT_BTC/status/261543364984442880/photo/1

[goodlord666]

I have a suggestion...can you make it such that the width of the chart can be expanded?  We are putting a display in our offices (a sort of startup hotel in Atlanta where we'll get a lot of traffic) that we would like to show the Bitcoin candlestick chart on...clarkmoody is good for that, except when you put it on a large 1080p monitor, the chart doesn't take advantage of the full width of the display.  You can pull up the website full screen on any 1080p to get an idea of the wasted real estate on the sides.  Even if you had just a few different width options (small, medium, large), it would be great. 

Also, we're using a Raspberry Pi to drive the display...it pushes that little box hard...anything you can do to make the JavaScript more efficient would be good.  It works, but just barely (mtgoxlive.com won't work on it).

Using the Chrome browser and zooming the page to fit the screen works well too (Chrome anti-aliases well.).

[Clark]

Notice: you may now save the chart without taking a screenshot. The resulting image is something like this:



I might add a full-screen option shortly, or some equivalent large size chart on wide monitors.

[Clark]

The interface now goes to a wide configuration on large monitors:



Around 25% of all site visitors come in with large monitors, so this feature should reach a lot of people. Also, you will not be able to hide the chart in widescreen mode, but with only 10% of users hiding the chart at any time, there shouldn't be a problem.

[dooglus]

Any idea what's going on here?  The bids are higher than the asks, and the red numbers are consistently higher than the green.  And what does the grey background on a sale indicate?

[Clark]

Any idea what's going on here?  The bids are higher than the asks, and the red numbers are consistently higher than the green.  And what does the grey background on a sale indicate?

You're looking at a "crossed" order book. I have put in logic to try and avoid that situation; usually when a new 'ticker' message comes over the socket, it repairs a crossed book. I guess it didn't fix yours. Usually a page refresh will fix a crossed book, but often I leave the site up for long periods without the book becoming crossed.

On the Time & Sales, the alternating background colors indicate trades that come in blocks. If a certain amount of time passes, the platform considers it a new trade and changes the background color. As for the color of the trades, the green color is on trades that occur at the bid (someone is adding a bid that makes a trade), and red trades occur at the ask (someone adding a sell order that makes a trade). The situation you observed seems to be a period of price jitter.

[genuise]

You're looking at a "crossed" order book. I have put in logic to try and avoid that situation; usually when a new 'ticker' message comes over the socket, it repairs a crossed book. I guess it didn't fix yours. Usually a page refresh will fix a crossed book, but often I leave the site up for long periods without the book becoming crossed.

Hi, can I ask what can be a cause of the crossed orderbook state? Is it on the mtgox side?

And what is the logic of the reparement process you use?

Am I right ? you use ticker channel messages to validate orderbook state which itself
 is updated from depth channel?

[Clark]

Hi, can I ask what can be a cause of the crossed orderbook state? Is it on the mtgox side?

And what is the logic of the reparement process you use?

Am I right ? you use ticker channel messages to validate orderbook state which itself
 is updated from depth channel?

The order book comes in from my server as a snapshot, then MtGox sends updates to the book in real time. If those updates get out of whack, then the book can get crossed.

I repair the book whenever ticker messages come in, listing the Bid & Ask prices. If the book prices are incompatible, then I remove those entries.

You are correct in your estimation.

[genuise]

Hi, Clark,
thank you for your answers.

We faced those problems with invalid orderbook state which have to be repaired. In fact those crossed states appear to be quite often.

I wonder what is the primary cause of those crossings?
My first guess was that gox api in time of high frequency trades sends update messages in incorrect order. Thus my strategy was to wait short period of time till the order book is fixed before reinitialization. But it seems that this strategy is not working well.

and your solution probably is much robust.

but the main question remains, what is the cause of the problem? I thought that having socket connection should eliminated any transport related problems and all updated messages have to arrive as they were sent.

My guess is the this can be some problem on  gox side. What do you think?

[Clark]

but the main question remains, what is the cause of the problem? I thought that having socket connection should eliminated any transport related problems and all updated messages have to arrive as they were sent.

My guess is the this can be some problem on  gox side. What do you think?

The problem is that Gox doesn't deliver the order book over the websocket, so my server grabs the order book, which is cached for 10 seconds by their server. Then the user gets it and updates it with changes from the real time feed. I believe that synchronization errors lead to crossed books, etc.

[genuise]

Yes, this can be a problem, especially in case of additional difference between time your server synchronized orderbook state and time when user requests your page and the moment his client establishes connection/subscribtions to mtgox channels.

In our case server maintains orderbook state first requesting over http, and exactly at this moment we start buffering depth messages. Then when http request returned successfully we play buffered messages and continue listening to new messages.

Our client communicates directly with our server and updates its orderbook state once per 15 seconds.

But the problem is that while we have less point of error on server orderbook crossings appear very often.
So there is evidence that the cause is not only, not mainly the delays during the initalization, but rather during normal depth messages updates.

At least we can agree that initialization procedure have to be revised on the mtgox side and somehow they have to provide the errorless initialization procedure.

 

[molecular]

Yes, this can be a problem, especially in case of additional difference between time your server synchronized orderbook state and time when user requests your page and the moment his client establishes connection/subscribtions to mtgox channels.

In our case server maintains orderbook state first requesting over http, and exactly at this moment we start buffering depth messages. Then when http request returned successfully we play buffered messages and continue listening to new messages.

Our client communicates directly with our server and updates its orderbook state once per 15 seconds.

But the problem is that while we have less point of error on server orderbook crossings appear very often.
So there is evidence that the cause is not only, not mainly the delays during the initalization, but rather during normal depth messages updates.

At least we can agree that initialization procedure have to be revised on the mtgox side and somehow they have to provide the errorless initialization procedure.

I can agree with that.

I never got my client to correclty sync orderbook. There's always (even after hours of operation) cases where there are impossible orders in my book. Leftovers. I've put hours into debugging this, just can't find the problem.

Initial sync is unsolved in my mind, too. Maybe orderbook change messages could have timestamps and also the orders in orderbook download could have "last updated" timestamps. That way, we could start buffering change channel, download the orderbook and the replay the changes, but only if timestamp < last_updated.

[Clark]

Initial sync is unsolved in my mind, too. Maybe orderbook change messages could have timestamps and also the orders in orderbook download could have "last updated" timestamps. That way, we could start buffering change channel, download the orderbook and the replay the changes, but only if timestamp < last_updated.

That's what I do. I buffer a certain time period of realtime messages, enough to be more than the cache period of the order book. Then I save the latest stamp from that book and update it with any realtime messages that occurred after that stamp.

[genuise]

Hi, I am very interested, but I still have questions to clear.
Can you explaine some details please?

I buffer a certain time period of realtime messages, enough to be more than the cache period of the order book. Then I save the latest stamp from that book and update it with any realtime messages that occurred after that stamp.

What exactly do you mean with "cache period of the order book"? Mtgox's cache period? or your server cache?

What is exactly "the latest stamp from that book" ? how exactly do you choose it?
stamp is timestamp or what? is it mtgox's stamp or you server stamp?

thank you

[Clark]

What exactly do you mean with "cache period of the order book"? Mtgox's cache period? or your server cache?

What is exactly "the latest stamp from that book" ? how exactly do you choose it?
stamp is timestamp or what? is it mtgox's stamp or you server stamp?

thank you

Cache time is the combined cache time from both my server and MtGox.

MtGox sends the timestamp of when the order went on the book for each order. Also, each depth message contains a timestamp. So I just use all of that together.

[Herodes]

are you considering making an affordable subscription based service with more information than what's currently available ?

Things that would be interesting would be stuff such as:

Getting tot. volume, avg. trade size and num of trades for last n minutes. There could be a sliding scale so the user could decide what he wanted to look into.

Also it would be very cool to have an overview of price development for the last n days, basically the user could decide what the X and Y scales should look like, and could look at the data that he desired to look at.

Sorry if anything of this has already been covered.

[genuise]

MtGox sends the timestamp of when the order went on the book for each order. Also, each depth message contains a timestamp.

MtGox sends the timestamp of when the order went on the book

In what channel this timestamp is provided?

[Clark]

In what channel this timestamp is provided?

From the MtGox public data API.

Code:

{"result":"success","return":{"asks":[{"price":29.79999,"amount":9.1649916,"price_int":"2979999","amount_int":"916499160","stamp":"1361479711579718"}, ...

MtGox sends the 'now' parameter in the streaming depth data. I'm assuming that both of those are stamped from the same server.