SierraChart bridge - Realtime Bitcoin charts [v0.5] (MtGox, Intersango, ...)

[deepceleron]

Something weird is happening here.
The hist api is up and accessible in the browser.
It doesn't look like any api specs have changed also.
But the lovely bridge can't get historical data at all.
Is this user-agent filtering of some sort or something?

The nginx gateway has become very intolerant of the type of requests that were being sent, or the api server was taking to long to return the result. For some reason the default URL http://api.bitcoincharts.com/v1/trades.csv?symbol=mtgoxUSD returns the most recent 20,000 trades again and again with no error, but when specifying a time range, anything other than a very small amount of data gives a 502 gateway timeout error.

I altered the parameters of sierrachartfeed to make many small requests instead of desiring larger data blocks. This seems to be working OK. I also increased the default max history to 14 days:

http://we.lovebitco.in/schart/sierrachartfeedUTC-0.6.020.py (requires github source files also)
(new version available)/sierrachartfeedUTC-0.6.020.exe (Win32)

md5sum:
a163db9dc95a5352bac68f4e223d0250 *sierrachartfeedUTC-0.6.020.exe


History files are also updated to today's date; refer back to this full post for sierrachart instructions:

SCID full history files:
from first trade 2010-07-17 23:09:17, to 2013-05-26 23:59:59 (UTC, tick accurate, precision 2):
(extract to C:\SierraChart\data\ before starting SierraChart)

http://we.lovebitco.in/schart/mtgoxUSD.scid.UTC.7z (19.9MB/200MB)
http://we.lovebitco.in/schart/otherALL.scid.UTC.7z (12.0MB/128MB)

[1713406583]

1713406583

[1713406583]

1713406583

[skywalk]

this no longer work with trial version, sierrachart patched this hack, u need to buy subscription.

[xeverse]

this no longer work with trial version, sierrachart patched this hack, u need to buy subscription.

They can not patch the old builds unless you are talking about the most recent.

Their subscription doesn't even unlock some advanced charting tools..

[skywalk]

well mine stop working, and creating new user didnt fix. very sad

[deepceleron]

Delete everything Sierrachart from your system, and try this build:

http://we.lovebitco.in/schart/SierraChartSetup.exe

[Loozik]

A temporary solution for those who like high quality charting for free

https://www.tradingview.com/e/?symbol=MTGOX:BTCUSD#

[arklan]

well mine stop working, and creating new user didnt fix. very sad

mines been ticking away problem free, and i haven't touched it for weeks.

[bitdraw]

hey =)

Can you use this software to chart bitstamp as well or is it only for mtgox?

(if not, does anyone know a software to chart bitstamp with?)

greetings

[xeverse]

Can you use this software to chart stamp..

Yes you can. Anything listed @ bitcoincharts all markets list.
But i think the quality wouldn't be the same. Hey try & tell us how do you like it!

[bitdraw]

Can you use this software to chart stamp..

Yes you can. Anything listed @ bitcoincharts all markets list. Hey try & tell us!

thx! Gonna try it when i get back from work =)

[servicehidden]

slush,

Is there any linux clone?

Or how often do you make a linux port of Sierra?

[xeverse]

Is there any linux clone?

What GNU/Linux port of Sierra? Use wine. They say it runs..

[servicehidden]

Port would be better.

I'm actually looking something like zoo of rss readers, but about trading.

[deepceleron]

New version of Sierrachartfeed and direct-download history files, now with old exchanges for your entertainment

Update 021:

• Corrected crash downloading trade data with negative volume amounts in history (btc24EUR and ruxumUSD),
• Fixed downloading of symbols with no recent trades for months (undocumented API issue),
• Error handling of blank or invalid exchange symbols,
• Faster history download on long periods of no trades.

Update 020:
-Tweak data request sizes.

Update 019:
-Fix error on initial request too large.

Update 018:
-All times in UTC instead of local time (allows easy data file interchange with other users and comparison with other ticker sources),
  REQUIRED: remove all existing C:\SierraChart\data\*.scid files before using this new bridge version.
-properly close IP sockets (not a fix for all 502 errors, apparently)
-faster update,
-remove about 1-5MB of redundant downloading at initialization,
-cosmetic tweaks: error stats and download times on console.

Code:

Usage: sierrachartfeedUTC-0.6.021.exe [options]

Options:
  -h, --help            show this help message and exit
  -d DATADIR, --datadir=DATADIR
                        Data directory of SierraChart software
  -y, --disable-history
                        Disable downloads from bitcoincharts.com
  -p PRECISION, --volume-precision=PRECISION
                        Change decimal precision for market volume.
  -s SYMBOLS, --symbols=SYMBOLS
                        Charts to watch, comma separated. Use * for streaming
                        all markets. (default = mtgoxUSD only)
  -l HISTORY, --history=HISTORY
                        Maximum days of history to retrieve (default = 14).

note: when using the single-letter option, there is no space between the option and the parameter


Download links:
(new version available)/sierrachartfeedUTC-0.6.021.exe (3.9MB) (windows 32 bit standalone exe)
MD5: fb686793172f142ecdb2150d701c9c99 *sierrachartfeedUTC-0.6.021.exe

http://we.lovebitco.in/schart/sierrachartfeedUTC-0.6.021.py (python 2.7 source - needs github files also)

SCID full history files:
from first recorded trade to Wed, 21 Aug 2013 23:59:59 GMT:
(extract to C:\SierraChart\data\ before starting SierraChartfeed)

MtGox only: http://we.lovebitco.in/schart/mtgoxUSD.scid.UTC.7z (22.6MB/227MB)

• mtgoxUSD

Other current exchanges: http://we.lovebitco.in/schart/otherALL.scid.UTC.7z (19.0MB/200MB)

• bit2cILS, bitboxUSD, bitcashCZK, bitcurexEUR, bitcurexPLN, bitkonanUSD, bitnzNZD, bitstampUSD, btcdeEUR, btceEUR, btceRUR, btceUSD, btchkexHKD, btcnCNY, cbxUSD, crytrEUR, crytrUSD, fbtcEUR, fbtcUSD, fybseSEK, fybsgSGD, icbitUSD, intrsngEUR, intrsngGBP, intrsngPLN, justLTC, justNOK, justXRP, kptnSEK, localbtcEUR, lybitCAD, lybitUSD, mrcdBRL, mtgoxAUD, mtgoxCAD, mtgoxCHF, mtgoxCNY, mtgoxDKK, mtgoxEUR, mtgoxGBP, mtgoxHKD, mtgoxJPY, mtgoxNZD, mtgoxPLN, mtgoxRUB, mtgoxSEK, mtgoxSGD, mtgoxTHB, rippleEUR, rippleUSD, rippleXRP, rmbtbCNY, rockEUR, rockSLL, rockUSD, vcxEUR, vcxUSD, virtexCAD, virwoxSLL, weexAUD, weexCAD, weexUSD

Past/retired exchanges (Tradehill, etc): http://we.lovebitco.in/schart/historic.scid.UTC.7z  (3.4MB/32.9MB)

• aqoinEUR, b2cUSD, b7BGN, b7EUR, b7PLN, b7SAR, b7USD, bbmBRL, bcEUR, bcGBP, bcLREUR, bcLRUSD, bcmBMAUD, bcmBMGAU, bcmBMUSD, bcmLRUSD, bcmMBUSD, bcmPPUSD, bcmPXGAU, bcPGAU, bitchangePLN, bitfloorUSD, bitmarketAUD, bitmarketEUR, bitmarketGBP, bitmarketPLN, bitmarketRUB, bitmarketUSD, bitmeUSD, bitomatPLN, britcoinGBP, btc24EUR, btc24USD, btcexEUR, btcexJPY, btcexRUB, btcexUSD, btcexWMR, btcexWMZ, btcexYAD, btctreeUSD, cryptoxAUD, cryptoxUSD, exchbUSD, freshPLN, globalEUR, globalGBP, globalPLN, globalUSD, imcexEUR, imcexUSD, intrsngUSD, ruxumAUD, ruxumCHF, ruxumEUR, ruxumGBP, ruxumHKD, ruxumHUF, ruxumJPY, ruxumPLN, ruxumRUB, ruxumSEK, ruxumSGD, ruxumTHB, ruxumUAH, ruxumUSD, ruxumZAR, snwcnXRP, thAUD, thCLP, thEUR, thINR, thLRUSD, thUSD, wbxAUD

[myself]

Can you add a option to export the data to CSV

[deepceleron]

Can you add a option to export the data to CSV

There are already csv files for you to download at http://api.bitcoincharts.com/v1/csv/

Added:

SierraChart can export data into CSV format for you, after opening your exchange, choose Edit-> Export Intraday data to txt file. It exports to CSV in this format:

Code:

Date, Time, Open, High, Low, Last, Volume, Number of Trades, Bid Volume, Ask Volume
08/03/2013, 03:54:45, 104.00497, 104.00497, 104.00497, 104.00497, 1655, 1, 0, 0
08/03/2013, 03:54:46, 104.00502, 104.00502, 104.00502, 104.00502, 4244, 1, 0, 0
08/03/2013, 03:54:55, 104.00502, 104.00502, 104.00502, 104.00502, 1, 1, 0, 0

Note that these volumes are 100x larger than actual.

[S3052]

New version of Sierrachartfeed and direct-download history files, now with old exchanges for your entertainment

Update 021:

• Corrected crash downloading trade data with negative volume amounts in history (btc24EUR and ruxumUSD),
• Fixed downloading of symbols with no recent trades for months (undocumented API issue),
• Error handling of blank or invalid exchange symbols,
• Faster history download on long periods of no trades.

Update 020:
-Tweak data request sizes.

Update 019:
-Fix error on initial request too large.

Update 018:
-All times in UTC instead of local time (allows easy data file interchange with other users and comparison with other ticker sources),
  REQUIRED: remove all existing C:\SierraChart\data\*.scid files before using this new bridge version.
-properly close IP sockets (not a fix for all 502 errors, apparently)
-faster update,
-remove about 1-5MB of redundant downloading at initialization,
-cosmetic tweaks: error stats and download times on console.

Code:

Usage: sierrachartfeedUTC-0.6.021.exe [options]

Options:
  -h, --help            show this help message and exit
  -d DATADIR, --datadir=DATADIR
                        Data directory of SierraChart software
  -y, --disable-history
                        Disable downloads from bitcoincharts.com
  -p PRECISION, --volume-precision=PRECISION
                        Change decimal precision for market volume.
  -s SYMBOLS, --symbols=SYMBOLS
                        Charts to watch, comma separated. Use * for streaming
                        all markets. (default = mtgoxUSD only)
  -l HISTORY, --history=HISTORY
                        Maximum days of history to retrieve (default = 14).

note: when using the single-letter option, there is no space between the option and the parameter


Download links:
http://we.lovebitco.in/schart/sierrachartfeedUTC-0.6.021.exe (3.9MB) (windows 32 bit standalone exe)
http://we.lovebitco.in/schart/sierrachartfeedUTC-0.6.021.py (python 2.7 source - needs github files also)

SCID full history files:
from first recorded trade to 2013-07-18 11:59:59UTC:
(extract to C:\SierraChart\data\ before starting SierraChart)

MtGox only: http://we.lovebitco.in/schart/mtgoxUSD.scid.UTC.7z (21.9MB/221MB)

• mtgoxUSD

Other current exchanges: http://we.lovebitco.in/schart/otherALL.scid.UTC.7z (17.4MB/218MB)

• bit2cILS, bitboxUSD, bitcurexEUR, bitcurexPLN, bitkonanUSD, bitnzNZD, bitstampUSD, btcdeEUR, btceEUR, btceRUR, btceUSD, btchkexHKD, btcnCNY, cbxUSD, crytrEUR, crytrUSD, fbtcEUR, fbtcUSD, fybsgSGD, icbitUSD, intrsngEUR, intrsngGBP, intrsngPLN, justLTC, justNOK, justXRP, kptnSEK, lybitCAD, lybitUSD, mrcdBRL, mtgoxAUD, mtgoxCAD, mtgoxCHF, mtgoxCNY, mtgoxDKK, mtgoxEUR, mtgoxGBP, mtgoxHKD, mtgoxJPY, mtgoxNZD, mtgoxPLN, mtgoxRUB, mtgoxSEK, mtgoxSGD, mtgoxTHB, rippleEUR, rippleUSD, rippleXRP, rmbtbCNY, rockEUR, rockSLL, rockUSD, vcxEUR, vcxUSD, virtexCAD, virwoxSLL, weexAUD, weexCAD, weexUSD

Past/retired exchanges (Tradehill, etc): http://we.lovebitco.in/schart/historic.scid.UTC.7z  (3.4MB/32.9MB)

• aqoinEUR, b2cUSD, b7BGN, b7EUR, b7PLN, b7SAR, b7USD, bbmBRL, bcEUR, bcGBP, bcLREUR, bcLRUSD, bcmBMAUD, bcmBMGAU, bcmBMUSD, bcmLRUSD, bcmMBUSD, bcmPPUSD, bcmPXGAU, bcPGAU, bitchangePLN, bitfloorUSD, bitmarketAUD, bitmarketEUR, bitmarketGBP, bitmarketPLN, bitmarketRUB, bitmarketUSD, bitmeUSD, bitomatPLN, britcoinGBP, btc24EUR, btc24USD, btcexEUR, btcexJPY, btcexRUB, btcexUSD, btcexWMR, btcexWMZ, btcexYAD, btctreeUSD, cryptoxAUD, cryptoxUSD, exchbUSD, freshPLN, globalEUR, globalGBP, globalPLN, globalUSD, imcexEUR, imcexUSD, intrsngUSD, ruxumAUD, ruxumCHF, ruxumEUR, ruxumGBP, ruxumHKD, ruxumHUF, ruxumJPY, ruxumPLN, ruxumRUB, ruxumSEK, ruxumSGD, ruxumTHB, ruxumUAH, ruxumUSD, ruxumZAR, snwcnXRP, thAUD, thCLP, thEUR, thINR, thLRUSD, thUSD, wbxAUD

This works all very well, but there is one problem:

Recently I see some gaps in the chart. Is this appearing if I am not online 24/7? Is there any way to avoid those gaps without the need to be online and connected to the feed all the time?
thanks for your help

[xeverse]

Recently I see some gaps in the chart. Is this appearing if I am not online 24/7?

Sometimes. Only if the internet connection is lost.
Usually the bridge checks for missing data upon starting up and fetch what's missing.

[Han]

Is anyone else having trouble getting the BitstampUSD feed to update continuously like the MtGoxUSD feed?

[deepceleron]

Bitcoincharts seems to have changed the API again, breaking software (sierrachartfeed might be the only software using it?) It is now returning a list of trades in order from oldest->newest. Epoch time query parameters do strange things:

• The most recent 20,000 trades are returned if there are no time parameters,
• If the &start time parameter is used alone, 20,000 trades since the start time are returned,
• If the &start and &end parameters are used and return > 20,000 results, 20,000 trades since the start time are returned,
• If the &start and &end parameters are used and return <= 20,000 results, the requested range is returned,
• If the &end parameter is used alone and if the &end time parameter is after than the first of the most recent 20,000 trades, the most recent 20,000 trades list is truncated to the end time (unexpected/odd)
• If the &end parameter is used alone and if the &end time parameter is before the first of the most recent 20,000 trades, no results are returned (unexpected/odd)

It would appear that "end" only serves to truncate the results of either the 20,000 trades from "start=" or the most recent 20,000 trades.

Notable is that if you get chunks of 20,000 results chronologically, software can't continue retrieval at the next epoch second after the last trade in the reply - there still might be more trades with the same timestamp. You must request that second again to continue, de-duping or discarding the previous last-second result data, or re-request a smaller time interval (which is what my fixed sierrachartfeed did on the reverse-sorted data). The end of a data request might look like the example below, but there are actually five trades with timestamp 1350659454 - two more trades still to get:

Code:

...
1350659256,11.680250000000,0.970000000000
1350659256,11.699890000000,6.722307690000
1350659287,11.680240000000,0.010000000000
1350659454,11.680140000000,3.000000000000
1350659454,11.680240000000,0.990000000000
1350659454,11.680240000000,1.000000000000

A new download algorithm will need to be written equivalent in work to my first rewrite...