Bitcoin Forum

Economy => Trading Discussion => Topic started by: c0inbuster on May 09, 2013, 11:23:57 AM



Title: Crypto currencies historical data
Post by: c0inbuster on May 09, 2013, 11:23:57 AM
Hello,

I did a Python/Pandas script to retrieve data from Bitcoincharts.

I also performed some tests to ensure that data are ok
(no missing data, no overlapping data, constant timeframe...)

Every columns are stored as integer.

You can download M5 candlestick data for mtgox|BTC/USD and mtgox|BTC/EUR

mtgox|BTC/USD from Sun, 18 Jul 2010 03:40:00 GMT to Wed, 08 May 2013 00:00:00 GMT
 CSV format
http://www.4shared.com/file/7yNh5s14/mtgoxBTCUSD-M5-1279424400_1367.html?

 HDF5 format (more suitable for large files containing tables - a standard financial industry format)
http://www.4shared.com/file/SsXEjf7C/mtgoxBTCUSD-M5-1279424400_1367.html?

mtgox|BTC/EUR from Sat, 27 Aug 2011 08:00:00 GMT to Thu, 09 May 2013 00:00:00 GMT
 CSV format
http://www.4shared.com/file/04uR0jUN/mtgoxBTCEUR-M5-1314432000_1368.html?

 HDF5 format
http://www.4shared.com/file/xY2BxsAW/mtgoxBTCEUR-M5-1314432000_1368.html?

I can now get historical data for any markets supported by Bitcoincharts...

I'm also in the process of making a tick generator from candlestick data
(using a method quite similar to what Metatrader 5 is using)
http://www.mql5.com/en/articles/75

Stay tuned...

working4coins


Title: Re: Crypto currencies historical data
Post by: c0inbuster on May 09, 2013, 07:58:28 PM
My script to download historical data from Bitcoincharts is available for free (both as speech and as beer)

https://sites.google.com/site/working4coins/data/btccharts-tick2candlestick

this version download CSV tick data from
http://api.bitcoincharts.com/v1/csv/

and output CSV file with candlestick OHLCV.

I also have an other script which use this API
http://bitcoincharts.com/charts/chart.json?m=mtgoxUSD&r=30&i=15-min&c=1&s=2013-01-01&e=2013-01-31
it's not tick data but directly candlestick...
The advantage of the latter script is that there is no need to download a large file like this one

I will release it if I get some feedback...


Title: Re: Crypto currencies historical data
Post by: dscotese on May 12, 2013, 05:34:10 PM
I was looking for historical hourly HLOCV for MtGox trade in USD and I found your thread.  Do you have that data, and are you selling it?


Title: Re: Crypto currencies historical data
Post by: c0inbuster on May 12, 2013, 06:30:41 PM
You can download data here

http://www.4shared.com/file/mnXToC4n/mtgoxBTCUSD-H1-201007180000-20.html?

(data are coming from Bitcoincharts and python-pandas script to resample tick data to OHLCV is available here
https://sites.google.com/site/working4coins/data/btccharts-tick2candlestick

Feel free to donate what you want !
https://sites.google.com/site/working4coins/donate

(I own less than 20 BTC and less than 120 LTC)


Data are integer (Unix/Epoch timestamp, price*10^5, vol*10^8)


Title: Re: Crypto currencies historical data
Post by: dscotese on May 12, 2013, 07:47:10 PM
Excellent, Thanks!  I'm a developer too, so let me know if you ever get some work you want to share.  I sent you 0.2 BTC.


Title: Re: Crypto currencies historical data
Post by: c0inbuster on May 12, 2013, 09:01:42 PM
Thanks for your reward. That's nice.

If you want some work...

Have a look at this thread
https://bitcointalk.org/index.php?topic=202808.0
it deals with zipline a python pandas framework for backtesting
strategy.

If you know Python and Pandas
it will be a good think to transform Time-OHLCV data to
Time Price data (pseudo tick)

A first basic idea could be to transform to something like
t0 OPEN_dt0
t0+timeframe/4 LOW_dt0
t0+2*timeframe/4 HIGH_dt0
t0+3*timeframe/4 CLOSE_dt0
t1 OPEN_dt1
t1+timeframe/4LOW_dt1
t1+2*timeframe/4 HIGH_dt1
t1+3*timeframe/4 CLOSE_dt1
...

I 'd prefer a vectorized code (no ugly for loop) as my goal will be to feed ziplinewith M30 or H1 data
(it can be a very long if we also want to process M1 timeframe data)

A second step could be to be able to transform OHLCV data to "pseudo tick" data
(control points) using this method
http://www.mql5.com/en/articles/75
you could begin with only 3-5-3 pattern

code must be open source


Other work to do... but that's harder...
integrate zipline with goxtool
in order to be able to test a strategy
and have this same strategy be applied to real market.