Bitcoin Forum
December 09, 2016, 10:07:02 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: R quantmod xts load bitcoincharts.com data  (Read 4001 times)
br00t
Newbie
*
Offline Offline

Activity: 16



View Profile WWW
May 29, 2012, 03:32:33 AM
 #1

Code:
library('quantmod')
library('zoo')

TEMP_FILE <- './data/temp.csv'

ohlc <- function(ttime,tprice,tvolume,fmt) {
  # function which converts raw bitcoincharts ticker data to ohlc data
  # possible formats include: "%Y%m%d %H %M" (minute ticker), "%Y%m%d %H" (hourly ticker), "%Y%m%d" (daily ticker)
  ttime.int <- format(ttime,fmt)
  data.frame(time = ttime[tapply(1:length(ttime),ttime.int,function(x) {head(x,1)})],
             .Open = tapply(tprice,ttime.int,function(x) {head(x,1)}),
             .High = tapply(tprice,ttime.int,max),
             .Low = tapply(tprice,ttime.int,min),
             .Close = tapply(tprice,ttime.int,function(x) {tail(x,1)}),
             .Volume = tapply(tvolume,ttime.int,function(x) {sum(x)}),
             .Adjusted = tapply(tprice,ttime.int,function(x) {tail(x,1)}))
}

getTicker <- function(symbol,period,datasource,filename='') {
  # this method retrieves ticker data from bitcoincharts.com or a a CSV downloaded from bitcoincharts.com for any symbol listed on the site
  # usage: ticker <- data.frame(getTicker('mtgoxUSD|virtexCAD\...', 'daily|hourly|minutes', 'web|file','./data/mtgoxusd-recent.csv'))
  # filename parameter is optional and only used when datasource = 'file'
  ColClasses = c('numeric','numeric','numeric')
  if(datasource == 'web') {
    RECENT_TRANSACTIONS_FILE <- paste('./data/', symbol, sep='')
    RECENT_TRANSACTIONS_FILE <- paste(RECENT_TRANSACTIONS_FILE, '-recent.csv', sep='')
    # let's be good and only download the data that we have to ...
    if(file.exists(RECENT_TRANSACTIONS_FILE)) {
      # get the last id
      mawk_command <- paste("mawk 'END {print}' ", RECENT_TRANSACTIONS_FILE, sep='')
      lastline <- system(mawk_command, intern = TRUE)
      lastid <- strsplit(lastline[1], split=',')[[1]][1]
      url <- paste('http://bitcoincharts.com/t/trades.csv?symbol=', symbol, sep='')
      url <- paste(url, '&start=', sep='')
      url <- paste(url, lastid, sep='')
      download.file(url, destfile=TEMP_FILE, method='auto', quiet = FALSE, mode = "w", cacheOK = TRUE, extra = getOption("download.file.extra"))
      cat("\n",file=RECENT_TRANSACTIONS_FILE,append=TRUE)
      # now we need some additional sanity checking here to see if this delta already exists in the
      # current local file or we will sometimes wind up repeatedly downloading the same transactions
      file.append(RECENT_TRANSACTIONS_FILE, TEMP_FILE)
      file.remove(TEMP_FILE)
    }
    else {
      download.file(url=paste('http://bitcoincharts.com/t/trades.csv?symbol=', symbol, sep=''), destfile=RECENT_TRANSACTIONS_FILE, method='auto', quiet = FALSE, mode = "w", cacheOK = TRUE, extra = getOption("download.file.extra"))
    }
    ticker <- read.zoo(RECENT_TRANSACTIONS_FILE, colClasses = ColClasses, sep = ",", header = FALSE)
  }
  else {
    ticker <- read.zoo(filename, colClasses = ColClasses, sep = ",", header = FALSE)
  }
  index(ticker) <- as.POSIXct(index(ticker), origin="1970-01-01", tz="GMT")
  if(period == 'monthly')  ohlcObj <- ohlc(index(ticker),ticker$V2,ticker$V3,"%Y%m")
  if(period == 'daily')  ohlcObj <- ohlc(index(ticker),ticker$V2,ticker$V3,"%Y%m%d")
  if(period == 'hourly') ohlcObj<- ohlc(index(ticker),ticker$V2,ticker$V3,"%Y%m%d %H")
  if(period == 'minutes') ohlcObj <- ohlc(index(ticker),ticker$V2,ticker$V3,"%Y%m%d %H %M")
  # clean up column names a bit
  names(ohlcObj)[names(ohlcObj) == ".Open"] <- 'Open'
  names(ohlcObj)[names(ohlcObj) == ".Close"] <- 'Close'
  names(ohlcObj)[names(ohlcObj) == ".Volume"] <- 'Volume'
  names(ohlcObj)[names(ohlcObj) == ".Adjusted"] <- 'Adjusted'
  names(ohlcObj)[names(ohlcObj) == ".High"] <- 'High'
  names(ohlcObj)[names(ohlcObj) == ".Low"] <- 'Low'
  names(ohlcObj)[names(ohlcObj) == "time"] <- 'Time'
  row.names(ohlcObj) <- NULL
  ohlcObj$row.names <- NULL
  return(ohlcObj)
}

getChartable <- function(ohlcObj) {
  # this method takes a raw ohlc data frame and rebuilds it in a way that emits an xts objects that can be used by chartSeries
  rownames(ohlcObj) <- ohlcObj$Time
  ohlcObj$Time <- NULL
  ohlcObjXts <- as.xts(ohlcObj)
  return(ohlcObjXts)
}

# Let's try it out now
#ticker <- data.frame(getTicker('mtgoxUSD', 'minutes', 'file','./data/mtgoxusd-recent.csv'))
#ticker <- data.frame(getTicker('virtexCAD', 'monthly', 'file','./data/cavirtex-2012.csv'))

# populate a data frame from web
ticker <- data.frame(getTicker('mtgoxUSD', 'hourly', 'web'))

# get xts object that can be used for charting
tickerxts <- getChartable(ticker)
chartSeries(tickerxts)

#Add a few technical indicators to the chart
addEMA(n=6*7,col='red')
addEMA(n=3*7,col='green')
addMACD()

Useful for modelling of bitcoincharts.com ticker data from all (?) exchanges in R. R data frame can also be imported into RapidMiner with the R plugin.

15DVPSkHh44MEk9NdkBW3hSumVcYECBZE8
A tautology is something which is tautological
1481278022
Hero Member
*
Offline Offline

Posts: 1481278022

View Profile Personal Message (Offline)

Ignore
1481278022
Reply with quote  #2

1481278022
Report to moderator
1481278022
Hero Member
*
Offline Offline

Posts: 1481278022

View Profile Personal Message (Offline)

Ignore
1481278022
Reply with quote  #2

1481278022
Report to moderator
1481278022
Hero Member
*
Offline Offline

Posts: 1481278022

View Profile Personal Message (Offline)

Ignore
1481278022
Reply with quote  #2

1481278022
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481278022
Hero Member
*
Offline Offline

Posts: 1481278022

View Profile Personal Message (Offline)

Ignore
1481278022
Reply with quote  #2

1481278022
Report to moderator
Seal
Donator
Hero Member
*
Offline Offline

Activity: 833


View Profile WWW
May 29, 2012, 07:30:33 AM
 #2

I'd love to have a play with R and learn the ropes.

Can you recommend any guides? I have a reasonable amount of programming experience so any tutorials or examples would help.

Also, what's your take on using R to programmatically trade? Thumbs-up or thumbs down?

bb113
Hero Member
*****
Offline Offline

Activity: 728


View Profile
May 29, 2012, 02:50:08 PM
 #3

I've been learning it the last few months (with little programming experience), I think the best way is to just have a problem you are motivated to solve and solve it by messing around and using google.
nimnul
Sr. Member
****
Offline Offline

Activity: 255


View Profile WWW
May 29, 2012, 07:46:03 PM
 #4

It's great to see someone actually developing models for bitcoin markets and not merely fortune telling using chart patterns. Keep going!

Seal
Donator
Hero Member
*
Offline Offline

Activity: 833


View Profile WWW
May 30, 2012, 12:11:01 AM
 #5

Regarding R for trading programmatically, it depends on what you're after.  I use Mathematica and R (depending on what's easiest for the task) for pricing and decision making over models but orchestrate the actual trades using other languages.

Great tutorials, it seems pretty powerful. Especially when paired with something like http://www.quantmod.com/

I have various scripts to place orders already in PHP, I guess R should be able to place url calls based upon its own statistical analysis.

Mathematica is licenced software by the looks of it.

How do the languages hold up for live price feeds? Can they crunch on the fly when given a data feed, say from mtgox?

br00t
Newbie
*
Offline Offline

Activity: 16



View Profile WWW
May 31, 2012, 01:31:51 AM
 #6

You know when I posted this I didn't realize that stochastic had already written this post https://bitcointalk.org/index.php?topic=61495.0 months ago, which goes into much greater detail and is probably of far greater value to someone looking to develop a quantitative model based on ticker data from bitcoincharts... definitely worth a read!

15DVPSkHh44MEk9NdkBW3hSumVcYECBZE8
A tautology is something which is tautological
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!