 Show Posts
|
|
Pages: [1] 2 »
|
<chart>
Just so I can say I called it if it does happen.
are your predictions ever not bullish? For all of this decline from 100 to 65 I was bearish, as for my predictions, they're mostly bullish. I can remember very well your bullish predictions just before we bounced off 114 into 60's He still owes people money for that cuz he bet it'll hit 120 before 100. |
|
|
|
Perhaps (buys-sells)/(buys+sells) (if buys+sells=0 then 0) in that period then. The idea is to compare the relative volume of buys and sells in a time window to see if the market is bullish or bearish during that period.
I just was thinking about how visually this shart will appear - as I can understand it will got positiv if buys greater than sells and negative otherwise. The value will not exceed +/-1. But how this an be modified to be ale to visualize not only side of trade but also the absolute siginificance. I mean to tell whether there are only small sells or big. Can it be some how combine in this idea? Of course we always can have existing already absolute trades volume chart but I am just curious. What is your opinion? To show the absolute scale just don't divide by total volume, though I'm of the opinion that if you have a stacked bar chart for the trade volume then that will be obvious. |
|
|
|
There can be several causes of this
1) some times exchange just does not return request - in this case the best option we have to assume the last available data as actual
the problem can be on exchange side - btc-e was ddos'ed or our hosting did not get connectivity.
2) in case of gox - we use quite complex logic for stream api client wich accounts for several cases:
- stream api can connect us but does not subsribe to any channel in this case we need to detect that stream is dead after some timout and reconnect
- if we could not reconnect for 15 minutes we force orderbook reinitialization - request full ordrbook via http
- also gox can disconnect/reconnect us arbitrarily often and each time we connected again and subscribed we need to reinitialize orderbook state requesting full orderbook via http but we cannot request too often otherwise we will be blocked by gox. thus we maintain hysotry of requests for each currency and wait in timeout if necessary some times for on hour and rarely for 24 hours.
all those timeouts is subject to changes and tweaks cause mtgox every time shows different and new patterns of how their engine can behave.
in all those cases we use last know state of orderbook to be able to assign trades to bid or aks side.
Note however that sells are being updated but buys are not. That rules out connectivity issues since buys and sells are on the same channel at least for mtgox. Also the same issue on three different exchanges at the same time is fishy. 3) bitcoincharts can:
- be down
- we just do not connection to bitcoincharts
- bitcoincharts delays some trades - in this case after some time bitcoincharts cn send all dealyed trades and if our application was still up it sorts all of them by timestamp to its time slot and assigns bid/ask side, but if our application happend to restart due its own uncought exceptions then delayed trades could not come to us at all.
So if orderbook state is not changing you can see flat line.
If trade time is increasing then trades are not coming.
so you can imagine that thigns are a bit complex here.
thank you for your attention to those details
To clarify, do you get your data from bitcoincharts or using each exchange's API directly? |
|
|
|
Another one that would be good to have is a moving average of the buys volume:sells volume ratio over windows of e.g. 10m, 1h, 3h, etc.
Here is one question, as I can understand you meant trading volumes here. What if buys is zero in current slot? or more over what if sells volume is zero or very small? In those cases the metric will have no meaning. Or what did you mean? Perhaps (buys-sells)/(buys+sells) (if buys+sells=0 then 0) in that period then. The idea is to compare the relative volume of buys and sells in a time window to see if the market is bullish or bearish during that period. Also the charts seem to be glitching out periodically where nothing is updated except sells. Happens at the same time on multiple exchanges so I assume it's not on mtgox's end, what's up with that?    |
|
|
|
4) On the VWAP line there are markers for each point, but not on the other price lines. I think it looks better without markers.
If you zoom the VWAP chart you will see something like this:  The markers are not for every point, but for each point (15-second "slot" as we call it) with trading activity. So here you can see that a large trade activity at 14:29:45 shifted VWAP, but a larger trade volume at 14:33:00 had almost no impact on the price. Removal of the markers will obscure this information. Of course when you zoom out to default zoom level it starts looking ugly, but we didn't knew how we can improve it without information loss. I'm not sure I understand what extra information is conveyed by the points on the VWAP line that's not already in the volume chart - as far as I can tell the set of points on the sell line show that a bunch of trades happened around 14:33 but only one of them had large volume as indicated by the corresponding point on the volume chart. The fact that the price didn't change seems incidental. In addition you can see here why we chose to plot "buy" and "sell" trades separately instead of joining them with a single line as our competitors do. We show that trades are performed at two different steady prices instead of zigzag pattern you often see on last trade price charts without bid-ask separation and volume weighting:  Yeah, I like the separate volumes for buys and sells a lot. Though the problem with using two line plots for them is 1) the line one with the higher volume in a timestep is going to obscure the lower one and 2) it's hard to tell how much total volume was traded for that timestep. A stacked bar graph with the buy volume on top of the sell volume would solve both problems and as a bonus is visually more clear - with a line plot the slope between points suggests the value is changing between points while a bar better represents the "slot" being filled with volume. |
|
|
|
|
Seems like that would do the trick. By the way by "/" I meant the ratio, editing to clarify.
|
|
|
|
Another idea, the raw total volumes could be used for a total bid USD/total ask BTC chart, would be interesting to see how that correlates with price.
This what I suggested some time ago for discussion. And I think in this case it would be good to use filtered raw data to have meaningful estimation for the price. How can we call this parameter - expected market price or general market expected price? or? Is there any existing parameter for this in financial world already? I'm not sure if there's a term for it, but total bids:total asks seems clear enough. I'm not sure if filtering would improve it and if so which filter, so again it would be nice to capture some data and compare. Another one that would be good to have is a moving average of the buys volume:sells volume ratio over windows of e.g. 10m, 1h, 3h, etc. |
|
|
|
|
Another idea, the raw total volumes could be used for a total bid USD/total ask BTC chart, would be interesting to see how that correlates with price.
|
|
|
|
I'm also curious if the spam orders on both sides simply cancel out, what does the chart look like if you just multiply without discounting? I mean you prefer to have both pair series or only one pair? Nevermind that request, I made a chart for the data and "fake" orders do indeed have a large impact. also as each filtering method will produce differernt series would it be practical to maintain all of them (thus multiplying serires) and toggle between them or just one. If one which one would be the most practical for analyzes on your opinion?
I think picking one is fine, which one to use depends on the purpose of the chart. The exponential function (a=0.1) prioritizes orders that are very close to the current price and hence should be a better indicator for short term market movement. The bandpass function (a=0.15) is closer to the "true" total volume but is more sensitive to volume changes that are very far away from the current price. The linear function is midway between the two but I'm not sure if it can match the benefit of either. I would run all three for a while and see what they look like, and then analyze the strength and weakness of each. More UI suggestions: 1) Make the height of the chart area scale with window size, if you've got chart height scaling working it would be really nice to hide a few of the charts and have the rest be taller. 2) Move the axis labels to the right side of the charts, that's where new data is coming in so that's where people will be looking. 3) Add an option for a dark colour scheme, I think most people prefer something like http://bitcoinity.org/markets or http://bitcoin.clarkmoody.com/. 4) On the VWAP line there are markers for each point, but not on the other price lines. I think it looks better without markers. |
|
|
|
|
Without a detailed rundown of your system my ability to evaluate your performance capabilities is limited but I'll try.
1) Server-side performance shouldn't be a problem for these filters, they only take O(n) additional arithmetic operations per update (n = number of orders in the orderbook) and the same amount of memory as whatever you're using to store the volume data right now.
2) When sending data to the client during an update, do you send all data or just the update delta? I'm not sure what sort of data the client keeps track of, but assuming you have all the calculations done server side the update delta should be very small, for each graph you should only need to send one additional datapoint per line per update.
3) It seems the client is receiving data for all currencies and exchanges, even the ones that aren't shown. You can avoid some data transfer by sending it selectively. For example, I only care about mtgox data in USD, there are 13 other tabs containing data I don't need. What I would do is have an option in the user's preferences for the default exchange and currency, and only load and display that data by default. Request and subscribe to the other tabs only when the user requests it.
4) Similar to 3), you don't have to show all charts by default and the client doesn't need data for hidden charts. As I suggested before, add a control to hide/show each chart, and have an option for a default set of charts in user preferences. Only request and subscribe to data for a chart if and when it's shown.
For each series you'll have 24*60*4 = 6240 datapoints. Each datapoint is two floats, i.e. 8 bytes, yielding about 49kB of raw floats per series (not sure how the data is structured, e.g. if you're using JSONs there will be more overhead). I'm not sure what kind of compression ratios you're getting, a cursory search suggests gzip doesn't do a very good job of compressing floats. Still, it shouldn't be a problem for most people and you may be able to reduce it further by using a compression algorithm that's optimized for floats.
With regards to the filtering I suggested, I think there's no real point sending the raw data to the client; as it is the data is only useful for showing relative change in volume for either bids or asks. Seeing as orders that are very far from where the trades are happening don't really affect the prices, I don't think you lose anything by reducing the weights assigned to them. In fact by discounting spam orders and far-out orders it would better link relative change in volume with change in price since the orders that matter are given more importance.
|
|
|
|
|
One more thing instead of using a line plot for depth and trade volumes it might be better to use a stacked bar chart. Right now it's really hard to hover over one exact datapoint to see its values, and it doesn't really take advantage of horizontal scaling.
|
|
|
|
Ok, I made some charts to demonstrate. The total discounted volume for bids and asks in USD is shown beneath each. Here's the raw data I was working with  Bids: 12170767.4644583 Asks: 27476862169015.9 And here's the depth chart up to twice the top bid  Here is the exponential discount factor I described before, it prioritizes prices close to the spread. The formula for the discount on bids is e^(-a(highestBid-bid)), and e^(-a(lowestAsk-ask)) for asks. Here's the discounted volume:  Bids: 105023.402931941 Asks: 23900.6565377794 That was a little too restrictive, relax the scaling parameter a bit to a=0.1:  Bids: 1522439.36415067 Asks: 1236490.55085024 It can be relaxed further to include points further down the line:  Bids: 3262099.42799612 Asks: 2777887.40515259 Alternatively to give full weight to the majority of the orders, use 1-e^(-a*bid) for bids, and max(0, 1-e^(a*(ask-lowestAsk)-highestBid))) to create a symmetrical bandpass filter about the spread. Here's what it looks like with a = 1:  Bids: 11970104.7601015 Asks: 9177170.77805910 Restrict the further out values a little with a = 0.15:  Bids: 11091817.7195716 Asks: 8799423.98610072 A simple linear discount function, use 1-(highestBid-bid)/highestBid for bids and max(0, 1-(ask-lowestAsk)/lowestAsk) for asks  Bids: 6158532.21541063 Asks: 5225588.99409854 Depending on what kind of analysis you want to make I'd say either the bandpass function with a=0.15, exponential function with a=0.1, or linear function would all work fine. |
|
|
|
|
Yeah, there seems to be some problems with the USD tooltips, prices jump erratically. Also it would be nice to be able to toggle the entire chart to display in USD instead of BTC.
Also is it just me or is the depth chart updating a lot less frequently than before?
|
|
|
|
It is not a problem to multiply usd bid price and volume of btc in each bid order.
But the problem is, as nimnul explained, that simple multiplication will not get meaningful number.
The problem is that not all bid orders have meaningful, as nimnul stated a "real", price. e.g. if someone places a bid order to buy 1000 000 000 btc for 1 USD, in other words buyer expects to buy 1bn btc at 0,000000001 usd/btc. does this price make sence for youe analyzes? Such order will be never executed.
Thus such orders represents spam, not real , fake data in the orderbook.
There are a lot of such orders, and their combined volume just obscurs the true picture.
If we exclude those fake orders we will see the true picture.
It this clear?
So the questions is how to tell which orders have to be excluded and which not?
The problem is here that there is no common agreement on which order have to be counted as meaningful and which is not.
So how to find this common filtering creteria is the main question.
Do you have any idea on how can we solve it?
One way to filter it would be use an discount factor based on distance from the top order. The discount factor can be exponential or linear. An exponential discount factor would be something like e^(-ax) where a is some scaling factor and x is the distance. A linear discount factor could be something very simple like a(x/x_top) For example, say current top bid is $60 and there's a bid for 1000000000 BTC at $1. The exponential discounted volume with a = 1 is 1000000000 * e^(-(60-1)) = 2.3802664e-17, i.e. negligible. If we want to give it a bit more weight we can decrease a to 0.1 which yields 1000000000 * e^(-0.1(60-1)) = 2739444.81877. The linear discounted volume with a = 1 is 1000000000 * (1/60) = 16666666.6667. Give it more weight by increasing a. The same thing could be used on asks. Personally I'd like to see orderbook volume charts in USD and BTC (option to show volume in USD or BTC). I'm also curious if the spam orders on both sides simply cancel out, what does the chart look like if you just multiply without discounting? If I could get some raw data I can play around with the parameters in MATLAB and see what values are reasonable. For the depth chart it's not necessary to discount, simply convert BTC to USD and vice versa at the price of each order to make volume on each side use the same currency. Again it would be nice to have an option to toggle between BTC and USD. Another suggestion, add another series in the VWAP chart for both buys and sells, and have a control to show/hide each chart. |
|
|
|
|
How about calculate the "real" volume yourself by e.g. multiplying the USD volume by the order price to get the true BTC volume?
|
|
|
|
|
Hi, as a subscriber is it possible to access data older than a day? Also can we get an option to have the volumes in either BTC or USD on the same axis, instead of having bids in USD and asks in BTC on separate axes? It's hard to compare two plots on different axes in e.g. the order book volume chart, you can only see the relative fluctuations on either axis.
|
|
|
|
Is current data available for download anywhere? How do you collect your data? Any plans to make interactive charts available on http://blockchained.com instead of static ones, or making a public API for the data? |
|
|
|
Provide the past predictions over a series of times at fixed intervals. Provide an objective standard for saying that your algorithm works.
That doesn't prove anything, I could claim my "algorithm" "predicted" 240 on the 9th, 120 on the 10th, and 60 on the the 12th. I say provide us with price predictions for the next 7 days. Yesterday's "prediction" with the crude growth line drawn over the increase from 60 to 120 transposed sideways a little has so far failed to pan out. Agreed that future predictions means he can't simply lie. That wasn't my presumption. It seems plausible he has something he calls an algorithm. I suspect, like some technical analysis people I've met. He's just fooling himself. However the problem with future predictions is that they themselves are information and could affect the outcome (e.g. knowing that $200 is going to be hit tomorrow could change what a large group of people do today) You could mitigate this buy using "sealed envelop" predictions. I really don't think enough people would base their decisions on some guy's kooky "predictions" without waiting to see if they're actually accurate. If they do I guess he succeeded in what he set out to do. |
|
|
|
Provide the past predictions over a series of times at fixed intervals. Provide an objective standard for saying that your algorithm works.
That doesn't prove anything, I could claim my "algorithm" "predicted" 240 on the 9th, 120 on the 10th, and 60 on the the 12th. I say provide us with price predictions for the next 7 days. Yesterday's "prediction" with the crude growth line drawn over the increase from 60 to 120 transposed sideways a little has so far failed to pan out. |
|
|
|
|
