Real Time Charting, Order Book, and Time & Sales

[Clark]

What do the blue and cyan lines in the chart?

10- and 21-period exponential moving averages.

c.f. https://bitcointalk.org/index.php?topic=60501.0

[1713911126]

1713911126

[1713911126]

1713911126

[1713911126]

1713911126

[Herodes]

I feel a bit uneasy complaining, as you provide an excellent service for free, but anyways, here goes:

I mentioned earlier that not all orders show up in your order book. I recently placed an ask order, and then I waited 10-15 secs but it still didn't show up in the orderbook, now waiting even more, perhaps 20-40 secs, I see it show up. How frequently do you fetch or receive this data ?

I thought the intention was to have orderbook changes show up as quickly as possible ? Thanks.

According to bitcoinity.org there was no lag at the time that this happened.

[Clark]

I feel a bit uneasy complaining, as you provide an excellent service for free, but anyways, here goes:

I mentioned earlier that not all orders show up in your order book. I recently placed an ask order, and then I waited 10-15 secs but it still didn't show up in the orderbook, now waiting even more, perhaps 20-40 secs, I see it show up. How frequently do you fetch or receive this data ?

I thought the intention was to have orderbook changes show up as quickly as possible ? Thanks.

According to bitcoinity.org there was no lag at the time that this happened.

The real-time data comes directly from MtGox. There have been other posts complaining about the lag in the socket.io stream.

The only data I fetch concerning the order book is the snapshot from MtGox that gets delivered shortly after the page loads. There is a cache time on MtGox and a cache time on my server, so that I don't poll too often (and get blocked).

[Luno]

I feel a bit uneasy complaining, as you provide an excellent service for free, but anyways, here goes:

I mentioned earlier that not all orders show up in your order book. I recently placed an ask order, and then I waited 10-15 secs but it still didn't show up in the orderbook, now waiting even more, perhaps 20-40 secs, I see it show up. How frequently do you fetch or receive this data ?

I thought the intention was to have orderbook changes show up as quickly as possible ? Thanks.

According to bitcoinity.org there was no lag at the time that this happened.

The real-time data comes directly from MtGox. There have been other posts complaining about the lag in the socket.io stream.

The only data I fetch concerning the order book is the snapshot from MtGox that gets delivered shortly after the page loads. There is a cache time on MtGox and a cache time on my server, so that I don't poll too often (and get blocked).

There has been a problem with your page for me since the 7'th. :
Orderbook loads only partially with an equally small number of around 1100 buy and 1100 asks. Last trade, highest buy, lowest sell never updates (no connected message in the console)

I'm using Firefox in Ubuntu. Noone else complains about this, so the problem must be with me, any suggestions as what to do?

[Clark]

Noone else complains about this

Look back about 5 posts 

Short answer: Gox has placed restrictive limits on how often I can poll the full book (5 times per hour), so as to make it a little more work to get it. I will be creating a solution shortly.

[Luno]

Noone else complains about this

Look back about 5 posts 

Short answer: Gox has placed restrictive limits on how often I can poll the full book (5 times per hour), so as to make it a little more work to get it. I will be creating a solution shortly.

Thanks, I will. BTW, Gox live pulls 45 MB a minute, but it's their own page of course, which is too much for my connection and must put a heavy load on their servers compared to your service.

[Herodes]

Hi Clark, thanks for the explanation.

[zhalox]

I have some good news Clark, I received an update from the Mt. Gox support, and they sent me the following messages regarding the hard limit polling issue, which is causing much of the market depth's order book to be missing.  They said you should use "data.mtgox.com" instead of "mtgox.com" for calling the API, and that you should contact Mt. Gox at support if you have any further inquiries on how to fix this:

Hello,

Would you please use data.mtgox.com instead of mtgox.com to call the API at bitcoin.clarkmoody.com and try again. Also there is no limit for the number of calls. Get back to us for any further assistance.

Thanks,
MtGox.com Team
March 18, 2013 20:34

Mt.Gox Support
Support Desk
Hello,

Clarkmoody will have to use data.mtgox.com instead of mtgox.com to call the API. Please let them know and there will be no limit for the number of calls for them. Please let them know that they can also contact us at Support should they have any further queries.

Thanks,
MtGox.com Team
March 19, 2013 00:40

[Clark]

I have some good news Clark, I received an update from the Mt. Gox support, and they sent me the following messages regarding the hard limit polling issue, which is causing much of the market depth's order book to be missing.  They said you should use "data.mtgox.com" instead of "mtgox.com" for calling the API, and that you should contact Mt. Gox at support if you have any further inquiries on how to fix this:

I had switched to the data.mtgox.com source soon after it was announced. But at that time, the /full wasn't allowing me to poll. However, it is up and running at this time will the full order book.

Thanks for the detective work.

[Herodes]

I have some good news Clark, I received an update from the Mt. Gox support, and they sent me the following messages regarding the hard limit polling issue, which is causing much of the market depth's order book to be missing.  They said you should use "data.mtgox.com" instead of "mtgox.com" for calling the API, and that you should contact Mt. Gox at support if you have any further inquiries on how to fix this:

I had switched to the data.mtgox.com source soon after it was announced. But at that time, the /full wasn't allowing me to poll. However, it is up and running at this time will the full order book.

Thanks for the detective work.

I think hiring 1 super intelligent and helpful support desk worker at MtGox would be a good idea. I think such a person could provide more help to the userbase then 4 non-skilled support representatives. Give him twice the current wage, and have 2 such persons instead of 4 providing the current support level.

Off-topic anyway, back to topic!

[Herodes]

I don't know if anybody told you, but when there's a selloff or rally caused by one big buy or sell, the two upper lines in the buy and bid columns seems to be stuck, while the rest of the lines updates. If you haven't got a report of it yet, I will try to make a screenshot the next time i see it.

[Clark]

I don't know if anybody told you, but when there's a selloff or rally caused by one big buy or sell, the two upper lines in the buy and bid columns seems to be stuck, while the rest of the lines updates. If you haven't got a report of it yet, I will try to make a screenshot the next time i see it.

Are you talking about when the order book is grouped by price?

I've noticed that also, and it corrects itself once a new ticker message comes in to update the inner bid and ask values.

[genuise]

Hi, speaking about mtgox socket.io api usual problems during high load.

Is it possible to implement that your server provides not only orderbook data but also broadcasts depth, ticker and trades channels to your clients?

In this case your server can take all of your clients load and in general this would be correct approach, while it definitly will help unload mtgox server.

What is your opinion?

[Clark]

Hi, speaking about mtgox socket.io api usual problems during high load.

Is it possible to implement that your server provides not only orderbook data but also broadcasts depth, ticker and trades channels to your clients?

In this case your server can take all of your clients load and in general this would be correct approach, while it definitly will help unload mtgox server.

What is your opinion?

I have been thinking about this particular problem for months, and I might provide this type of solution in the future.

[jordan.dev]

I'm having an issue getting the HTML5 Canvas to display in any of my browsers on OS X 10.8.3

When I originally opened the site, I scrolled in with my touchpad and it eventually went out of scope and I couldn't get it back.

So far I've tried:

• Double-clicking (reset) no change.
• I've also tried clearing my cache in all browsers

Any idea what the issue might be?

I've got it working in Firefox, and moving my scroll wheel up scrolls in, but I can't scroll back out (OS X Natural Scrolling enabled) -- but if I switch it works in Firefox, but no change in Safari in Chrome.




Also some weird output:

[Herodes]

I don't know if anybody told you, but when there's a selloff or rally caused by one big buy or sell, the two upper lines in the buy and bid columns seems to be stuck, while the rest of the lines updates. If you haven't got a report of it yet, I will try to make a screenshot the next time i see it.

Are you talking about when the order book is grouped by price?

I've noticed that also, and it corrects itself once a new ticker message comes in to update the inner bid and ask values.

Yes, when it's grouped by price. Ie. the normal view with Buying colum to the left, and Selling column to the right. When there's a selloff, it seems like the two uppermost lines on the buy-side doesn't update to reflect the selloff - but Time & Sales column is updated correctly.

[RoadTrain]

Yesterday I noticed a problem when using this service.
When I reload the page it seems to connect properly.
But after some time it tries to reload depth table

Code:

18:05:42 > Loading depth of market table.

18:05:42 > Opening streaming data connection.

And then (as far as I understood) it opens another socket connection to the ticker so I get each trade twice and even three times.

I'm using Opera 12.14.
Thanks.

[Clark]

Yesterday I noticed a problem when using this service.
When I reload the page it seems to connect properly.
But after some time it tries to reload depth table

Code:

18:05:42 > Loading depth of market table.

18:05:42 > Opening streaming data connection.

And then (as far as I understood) it opens another socket connection to the ticker so I get each trade twice and even three times.

I'm using Opera 12.14.
Thanks.

Interesting. I've changed the platform so that it attempts to connect to the plain WebSocket first and then falls back to socket.io. The normal WebSocket tends to disconnect a few times when first trying to connect (during high-traffic times). I am transparently reconnecting to that protocol each time it closes, but eventually I am having it connect to the socket.io endpoint.

Now, I'm really not sure what the socket.io does when it reconnects, but I think there shouldn't be multiple messages coming across the normal WebSocket, since I close old connections before attempting to open new ones.

Are you 100% sure that the messages coming across are duplicates?

[RoadTrain]

Interesting. I've changed the platform so that it attempts to connect to the plain WebSocket first and then falls back to socket.io. The normal WebSocket tends to disconnect a few times when first trying to connect (during high-traffic times). I am transparently reconnecting to that protocol each time it closes, but eventually I am having it connect to the socket.io endpoint.

Now, I'm really not sure what the socket.io does when it reconnects, but I think there shouldn't be multiple messages coming across the normal WebSocket, since I close old connections before attempting to open new ones.

Are you 100% sure that the messages coming across are duplicates?

I'm 100% sure. After these "reconnects" I get each trade multiple times with exact time and volume. I could make a screenshot but I doubt it can help.
Maybe it's something opera-related as I tried another PC with Opera and it still ends up in getting duplicate trades.

There's a full log when it started to get three exact trades and stopped to get any shortly after that.

Code:

02:16:15 > Got 1629 asks and 7367 bids.

02:16:13 > Opening streaming data connection.

02:16:13 > Loading full depth of market table.

02:15:15 > Got 1629 asks and 7367 bids.

02:15:13 > Opening streaming data connection.

02:15:13 > Loading full depth of market table.

02:14:34 > Got 1629 asks and 7367 bids.

02:14:32 > Opening streaming data connection.

02:14:32 > Loading full depth of market table.

02:14:25 > MtGox: Connected.

02:14:19 > Attempting connection over websocket

02:14:19 > Got 1629 asks and 7367 bids.

02:14:17 > Opening streaming data connection.

02:14:17 > Loading full depth of market table.

02:14:13 > MtGox: Connection error.

02:14:11 > Got 1629 asks and 7367 bids.

02:14:10 > Loading depth of market table.

02:14:10 > Opening streaming data connection.

02:13:07 > Unwinding buffered depth messages.

02:13:07 > Got 1635 asks and 7375 bids.

02:13:07 > Current trend: +0.10000 USD, Volume:247.9849 BTC

02:13:07 > Current trend: +0.10000 USD, Volume:243.0350 BTC

02:13:07 > Current trend: +0.10000 USD, Volume:238.0851 BTC

02:13:07 > Current trend: +0.10000 USD, Volume:233.1352 BTC

02:13:07 > Current trend: +0.10000 USD, Volume:155.4234 BTC

02:13:07 > Direction change up. Volume: 77.7117 BTC

02:13:06 > Current trend: -0.10000 USD, Volume:37.9800 BTC

02:13:06 > Current trend: -0.10000 USD, Volume:28.9800 BTC

02:13:06 > Current trend: -0.10000 USD, Volume:19.9800 BTC

02:13:05 > Synchronizing the order book.

02:11:58 > Got 1635 asks and 7375 bids.

02:11:57 > MtGox: Connected

02:11:56 > Loading depth of market table.

02:11:56 > Opening streaming data connection.

02:11:28 > Got 1629 asks and 7360 bids.

02:11:27 > MtGox: Connected

02:11:26 > Loading depth of market table.

02:11:26 > Opening streaming data connection.

02:11:18 > Current trend: -0.10000 USD, Volume:10.9800 BTC

02:11:15 > Current trend: -0.10000 USD, Volume:10.8800 BTC

02:11:14 > Direction change down. Volume: 0.8800 BTC

02:11:05 > Current trend: 0.00000 USD, Volume:9.0383 BTC

02:11:02 > Unwinding buffered depth messages.

02:11:02 > Got 1629 asks and 7360 bids.

02:11:02 > Buffering ask 0.00936355 @ 74.17896

02:11:02 > Buffering ask 0 @ 74.17896

02:11:02 > MtGox: Connected

02:11:00 > Loading depth of market table.

02:11:00 > Opening streaming data connection.

02:10:56 > MtGox: Connection error.

02:10:55 > Got 1629 asks and 7360 bids.

02:10:53 > Loading depth of market table.

02:10:53 > Opening streaming data connection.

02:10:45 > MtGox: Connection error.

02:10:44 > Got 1629 asks and 7360 bids.

02:10:42 > Loading depth of market table.

02:10:42 > Opening streaming data connection.

02:10:42 > Changing depth display to 100 rows.

02:10:42 > Resizing console to 5 lines.

[Clark]

Yeah, I have seen this problem before. I instruct socket.io to force a new connection each time. That could be the issue: an 'error' connection comes back to life and starts getting data or something.

Going with something like socket.io means that there is a bit of black-box to it.

I will continue to tweak and see if it helps.