Real Time Charting, Order Book, and Time & Sales

[molecular]

Thanks so much for the details!

Here is a screenshot from the site as it stands currently:



It looks as if the historic chart data that loads from the server is correct at least. That means the problem is happening with the real time feed. I have confirmed that my server-side database contains just one trade at that time (I also show 5 trades total with that volume - 1700BTC. Just a bit of trivia.)

Also note that the candles shown in my screenshot differ from those created in real time by the streaming feed.

I think bitcoincharts creates its graphics using polled historic data only, so it will not experience this problem. I'm not sure about bitcoin-analytics and whether the charts are created from streaming or polled data.

The discrepancy in candle shapes could be chalked up to timing differences / lag in the streaming feed over the stored MtGox database information. The double-volume bug, however, is more elusive. It would be nice to have a screenshot of the Time & Sales from that moment, so see if the trade showed up twice in that feed. If it so happened that the platform somehow had two connections to the streaming data (through the disconnect/reconnect behavior of the MtGox feed resetting every hour), then it would get double the data, and the chart would add trades twice.

As for a solution to this problem, it seems like it's probably going to be complicated. Since we're using socket.io to get the streaming data, we have to deal with black-box software. I'm not convinced that we don't have multiple connections to the streaming data sometimes. Diagnosing bugs like this usually involve waiting for something to happen in the real time feed again, which could take time.

I could put together a hack that reloads the chart data every so often, but that would ramp my server bandwidth usage and simply patch over this problem. I could instruct socket.io to NOT reconnect upon failures, requiring a site refresh, but our current reconnect solution is the result of complaints over having to refresh the page.

Maybe we could all keep looking more closely at the feed to notice if the Time & Sales starts showing double trades. I will look over the code again to see if there's any fix to be had there. Thanks for pointing this out to me.

couldn't you just store the trade IDs (tid) from the socket.io feed and check for duplicates when receiving new trades?

[molecular]

Oh wow, you are correct. I guess I was looking at the result of the streaming construction of the chart on a tab that has been open for ~14 hours. So this means that the streaming chart was correct in this case.

So the problem might lie in the server implementation of bar generation. The trade shows up once in the database, but the way that trades are turned into bars might have been corrupted.

This sounds possible. Maybe that quirk occurs when there is exactly one trade in the candles timespan (1 minute in this case and exactly one trade in that minute). It might be counting the volume twice because this trade is the last trade and the first trade within that candle at the same time or something.

[keewee]

This has been happening since early September at least. See this post by adamstgBit and my reply: https://bitcointalk.org/index.php?topic=82626.msg1180152#msg1180152

Confused me at the time but I assumed it was just a difference in the way the figures were calculated. I would have sent a bug report but didn't know enough about it to know it actually was a bug. Seems to be fixed now though 

[Yuhfhrh]

By the way, bitcoin.clarkmoody.com has been down. 

[Clark]

By the way, bitcoin.clarkmoody.com has been down. 

Yeah, I don't know what's happening. I'm contacting the host now. I assumed it would come back after a short time, but it's been down for hours. Please stand by.

[phungus]

Hi Clark,

I hope this was resolved for you. We're going through some transition issues with the hosting service and a number of us have been trying to bootstrap the support thing while RG has personal-life issues.

Please let us know if something is still not right and we'll get it working again. We hope to avoid these kinds of unresolved outages in the future because they don't help anyone. :-)

I think I know what server this one was and I think it was fixed yesterday. I hope that is the case here.

Thanks,
-p

[ThiagoCMC]

Hi!

 Why http://bitcoin.clarkmoody.com/ is outdated?

 The buy orders does not reflect the real live from mtgox.

 I already opened it from 3 different machines, same result.

 Mtgoxlive is working... Also, the Market Depth here is okay too: http://bitcoincharts.com/markets/mtgoxUSD_depth.html

Tks!
Thiago

[Clark]

The buy orders does not reflect the real live from mtgox.

I already opened it from 3 different machines, same result.

Mtgoxlive is working... Also, the Market Depth here is okay too: http://bitcoincharts.com/markets/mtgoxUSD_depth.html

Are you getting a connection error in the console?

[thph]

There is no error message - just the wrong orders

14.22888 0.0508 1228.0
14.22893 0.4000 1228.4
14.22900 0.0534 1228.5
14.22999 0.2002 1228.7
14.23000 1658.0102 2886.7
14.23193 0.0711 2886.8
14.23260 0.0766 2886.8
14.23425 0.0203 2886.9
14.23623 0.1000 


ask side

[Clark]

There is no error message - just the wrong orders

So what are the right orders? I need to know what it should be in order to fix any problems.

[thph]

Hi,
the orders are shown in clarkmoody are not right,
i for example have a selling order at 14.28026 since quite a while, it dont show up.
Mtgoxlive is working, all asks and bids are OK.

Never saw it like that, the first row in clarkmoody is OK.
sorry for my english

[Clark]

Hi,
the orders are shown in clarkmoody are not right,
i for example have a selling order at 14.28026 since quite a while, it dont show up.
Mtgoxlive is working, all asks and bids are OK.

Never saw it like that, the first row in clarkmoody is OK.
sorry for my english

It should be fixed now. I see your order on there now.

[thph]

Thanks - workin now !
Great !
Thanks

[phatsphere]

one bug, one feature request:

1. in firefox 18, when i set the controls to group by price of 0.01, and i reload. the form-checkbox is enabled, but it isn't grouped. when i un-set the checkbox, it is grouped. somehow, this seems to be stored but not properly parsed or set when reloaded.

2. would it be hard to do the following: when i hover over a bid/ask in the long list at the bottom, to highlight the corresponding row in the other column, where the cumulative bid/ask value is "almost" the same? that would help to assess the inequality of bids vs. asks!

p

[Clark]

1. in firefox 18, when i set the controls to group by price of 0.01, and i reload. the form-checkbox is enabled, but it isn't grouped. when i un-set the checkbox, it is grouped. somehow, this seems to be stored but not properly parsed or set when reloaded.

2. would it be hard to do the following: when i hover over a bid/ask in the long list at the bottom, to highlight the corresponding row in the other column, where the cumulative bid/ask value is "almost" the same? that would help to assess the inequality of bids vs. asks!

1. Are you getting any messages in the JavaScript console when you load the page? Any script errors?

2. This is certainly an interesting thing to consider. I might tool around with something like this in the future.

[BTCdude007]

One feature request...hide small trades and quotes.

There are a lot of 0.01 and like trades and quotes that lead to clutter. It would be nice to have a user selectable control
whereby you can hide these.

I would like to set my limit to 1BTC, so all ungrouped quotes < 1BTC and Time&Sales < 1BTC are hidden.

Thanks for such a useful website. I sent you a 0.5BTC donation a few days back.

[m_yaw]

Maybe this has been asked before but why do I see completely different ask/bids on the EUR view compared to the real EUR ask/bids on mtgox?

[Clark]

Maybe this has been asked before but why do I see completely different ask/bids on the EUR view compared to the real EUR ask/bids on mtgox?

You should be able to refresh and have the proper order book. I'm still working out the kinks in a new data-delivery platform.

[m_yaw]

k, thx. Now it looks much more reasonable!

[PAPASO]

Can you add weekly mouthly chart in you web?