Sure - although tbh I've only been using rgp for a couple of weeks, before that I was using eureqa but I didn't find it flexible enough (it's really just for symbolic regression).
rgp is well documented but if you don't have a lot of experience with GAs (i don't) then the tutorials at rsymbolic.org are a bit incomplete. Still, I have plenty of algorithms generated. Some overfit and just buy below a certain level and sell above it. It all comes down to your fitness function and i think mine leaves a bit to be desired.
Some of the useful trading algo's I've got are of the longlag-shortlag=0 variety, just like the EMA10 EMA21 variety.
I'll be happy to share code I've done so far - I'm sure you can improve on it.
Thanks, if you can put some here or pm to me I will take a look at it. I requested some books on the subject.
|
|
|
Using modified code from here and here. I ran some backtesting in R to compare the EMA 5 EMA 21 Crossover and a random buy/sell strategy based on a coin flip. Why did I do this? I was making my own trading strategies but got stuck on making one for bullish/bearish divergence and I thought some practice may help me find out what I am doing wrong. Why not generate your own? The R package rgp lets you evolve GAs. I've got some that consistently give me better than 'buy and hold'. In artificial intelligence, genetic programming (GP) is an evolutionary algorithm-based methodology inspired by biological evolution to find computer programs that perform a user-defined task. It is a specialization of genetic algorithms (GA) where each individual is a computer program. It is a machine learning technique used to optimize a population of computer programs according to a fitness landscape determined by a program's ability to perform a given computational task. Wow, thanks for the recommendation. I am an evolutionary biologist (or trying to be) and this makes a lot of sense. How have you applied GP to trading strategies?
|
|
|
Dear community,
...I unofficially give Bitscalper a Bitcoin Risk Level of 99%...
This is a great point. I won't do business with others that won't be more open especially with bitcoin. It would be great if there was a Bitcoin Better Business Bureau or something like you mention with a Risk Level.
|
|
|
***EDIT*** Calculated Expectancy Expectancy = (Probability of Win * Average Win) – (Probability of Loss * Average Loss) Flipist Daily Flip (1) buy / (-1) sell Signal # Trades % Win Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L 1 -1 93 52.68817 7.132123 -6.507944 3.636423 -4.839239 1.095910 0.7514452 2 0 1 0.00000 NaN NaN NA NA NaN NA 3 1 90 46.66667 4.551028 -4.431499 2.366660 -2.242254 1.026973 1.0554826 Flipist - Hourly Flip (1) buy / (-1) sell Signal # Trades % Win Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L 1 -1 2208 50.00000 1.123302 -1.034769 0.6283153 -0.6485819 1.085558 0.9687524 2 0 1 0.00000 NaN NaN NA NA NaN NA 3 1 2206 49.50136 1.063690 -1.027253 0.6050835 -0.5874410 1.035470 1.0300328 Expectancy = -4.0892 EMA10/21 Crossover using Daily data. There are 18 less trades when looking at the EMA10/21, buy (1) when EMA10 > EMA21, sell (-1) when EMA10 < EMA21 Signal # Trades % Win Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L 1 -1 132 56.81818 6.341372 -5.713935 2.840616 -4.183966 1.109808 0.678929 2 0 21 0.00000 NaN NaN NA NA NaN NA 3 1 52 57.69231 5.068756 -3.911978 3.199519 -1.747152 1.295702 1.831277 EMA10/21 Crossover using Hourly data. EMA10/21 Buy (1) when EMA10 crosses over EMA21, sell (-1) when EMA10 crosses below EMA21 Signal # Trades % Win Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L 1 -1 83 51.80723 0.8337976 -0.8799018 0.4516124 -0.5958119 0.9476030 0.7579782 2 0 4249 0.00000 NaN NaN NA NA NaN NA 3 1 83 46.98795 0.6240971 -1.4239146 0.4776158 -0.5503347 0.4382968 0.8678642 Expectancy = -45.00281
|
|
|
You need emotional investor and EMA10/21 contrarian controls. I'm not sure how to implement the emotional investor one...
How would you do the controls then?
|
|
|
Now for the Flipist strategy. This is the same strategy as before but now we can flip every hour if needed. First the code. I added in everything to this code, from making the signals, making the charts, and outputting the statistics. ############################################ #Backtesting Flipist Method ############################################ # We will need the quantmod package for charting and pulling # data and the TTR package to calculate RSI(2). # You can install packages via: install.packages("packageName") # install.packages(c("quantmod","TTR")) library(quantmod) library(TTR)
# Load data x = last(y,4415) #gets the last 184 daily positions from the data (starts at July 23, 2011)
# Calculate the random indicator set.seed(43) #include this to reproduce my results x$flip <- rbinom(4415,1,0.5) #50% probability that 1 (heads/buy) will show and 50% probability that 0 (tails/sell) will show
# Create the buy (1) and sell (-1) signals sigbuy <- ifelse(x$flip == 1, 1, 0) sigsell <- ifelse(x$flip == 0, -1, 0)
# Lag signals to align with days in market, # not days signals were generated sigbuy <- lag(sigbuy,1) # Note k=1 implies a move *forward* sigsell <- lag(sigsell,1) # Note k=1 implies a move *forward*
# Replace missing signals with no position # (generally just at beginning of series) sigbuy[is.na(sigbuy)] <- 0 sigsell[is.na(sigsell)] <- 0
# Combine both signals into one vector sig <- sigbuy + sigsell # Calculate Close-to-Close returns ret <- ROC(Cl(x)) ret[1] <- 0
# Calculate equity curves eq_up <- exp(cumsum(ret*sigbuy)) eq_dn <- exp(cumsum(ret*sigsell*-1)) eq_all <- exp(cumsum(ret*sig))
# Equity Chart png("flipist.png",width=720,height=720) plot.zoo( cbind(eq_up, eq_dn), ylab=c("Long","Short"), col=c("green","red"), main="Flipist Strategy: 07-23-2011 to 01-22-2012" ) dev.off()
# Create a chart showing mtgoxUSD png("flipistchart.png",width=720,height=720) chartSeries(x, subset="last 4415 hours", type="line")
# Add the total equity line addTA(eq_all) dev.off()
# Evaluate the Strategy
# install.packages("PerformanceAnalytics") require(PerformanceAnalytics) # chart equity curve, daily performance, and drawdowns png("performance.png",height=720,width=720) charts.PerformanceSummary(ret) dev.off()
# This function gives us some standard summary # statistics for our trades. tradeStats <- function(signals, returns) { # Inputs: # signals : trading signals # returns : returns corresponding to signals
# Combine data and convert to data.frame sysRet <- signals * returns * 100 posRet <- sysRet > 0 # Positive rule returns negRet <- sysRet < 0 # Negative rule returns dat <- cbind(signals,posRet*100,sysRet[posRet],sysRet[negRet],1) dat <- as.data.frame(dat)
# Aggreate data for summary statistics means <- aggregate(dat[,2:4], by=list(dat[,1]), mean, na.rm=TRUE) medians <- aggregate(dat[,3:4], by=list(dat[,1]), median, na.rm=TRUE) sums <- aggregate(dat[,5], by=list(dat[,1]), sum)
colnames(means) <- c("Signal","% Win","Mean Win","Mean Loss") colnames(medians) <- c("Signal","Median Win","Median Loss") colnames(sums) <- c("Signal","# Trades")
all <- merge(sums,means) all <- merge(all,medians)
wl <- cbind( abs(all[,"Mean Win"]/all[,"Mean Loss"]), abs(all[,"Median Win"]/all[,"Median Loss"]) ) colnames(wl) <- c("Mean W/L","Median W/L")
all <- cbind(all,wl) return(all) }
print(tradeStats(sig,ret))
|
|
|
Here is the EMA10/21 crossover using hourly data instead of daily. This is still all buy or all sell. I also found how to get the number of trades. # This function gives us some standard summary # statistics for our trades. tradeStats <- function(signals, returns) { # Inputs: # signals : trading signals # returns : returns corresponding to signals
# Combine data and convert to data.frame sysRet <- signals * returns * 100 posRet <- sysRet > 0 # Positive rule returns negRet <- sysRet < 0 # Negative rule returns dat <- cbind(signals,posRet*100,sysRet[posRet],sysRet[negRet],1) dat <- as.data.frame(dat)
# Aggreate data for summary statistics means <- aggregate(dat[,2:4], by=list(dat[,1]), mean, na.rm=TRUE) medians <- aggregate(dat[,3:4], by=list(dat[,1]), median, na.rm=TRUE) sums <- aggregate(dat[,5], by=list(dat[,1]), sum)
colnames(means) <- c("Signal","% Win","Mean Win","Mean Loss") colnames(medians) <- c("Signal","Median Win","Median Loss") colnames(sums) <- c("Signal","# Trades")
all <- merge(sums,means) all <- merge(all,medians)
wl <- cbind( abs(all[,"Mean Win"]/all[,"Mean Loss"]), abs(all[,"Median Win"]/all[,"Median Loss"]) ) colnames(wl) <- c("Mean W/L","Median W/L")
all <- cbind(all,wl) return(all) }
print(tradeStats(sig,ret)) Signal # Trades % Win Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L 1 -1 83 51.80723 0.8337976 -0.8799018 0.4516124 -0.5958119 0.9476030 0.7579782 2 0 4249 0.00000 NaN NaN NA NA NaN NA 3 1 83 46.98795 0.6240971 -1.4239146 0.4776158 -0.5503347 0.4382968 0.8678642 Here are the graphs. The equity curve graph with long (green) and short (red). ***EDIT*** This graph's title should say EMA10/21, not EMA5/21 I tried to plot the EMA10 and 21 on here but the time span is so large it is hard to see. Here is the cumulative return along with hourly return and draw-down. Surprisingly or not it is the same as the other daily data set.
|
|
|
... when I run the data there is only 1 trade at the beginning of December. So what to do? I can change the data to hourly.
do you suppose if you ran the flippest method many times and averaged the result at each point in time, you'd end up with a 'returns' chart that was pretty-much the same as the market price? I would expect in a trending market the ratio would be about breaking even, but I never looked before so I can only guess.
|
|
|
To convert the data to hourly is pretty easy. There is some function in one of the quantitative financial packages in R. But I forget which one. But that one function I had before can do the same thing with only a slight modification. ohlc <- function(ttime,tprice,tvolume,fmt) { ttime.int <- format(ttime,fmt) data.frame(time = ttime[tapply(1:length(ttime),ttime.int,function(x) {head(x,1)})], mtgoxUSD.Open = tapply(tprice,ttime.int,function(x) {head(x,1)}), mtgoxUSD.High = tapply(tprice,ttime.int,max), mtgoxUSD.Low = tapply(tprice,ttime.int,min), mtgoxUSD.Close = tapply(tprice,ttime.int,function(x) {tail(x,1)}), mtgoxUSD.Volume = tapply(tvolume,ttime.int,function(x) {sum(x)}), mtgoxUSD.Adjusted = tapply(tprice,ttime.int,function(x) {tail(x,1)})) }
data <- ohlc(data$time,data$price, data$volume,"%Y%m%d%H") #converts data in CSV to OHLC,[b] notice the %H[/b] The original data is taken from bitcoincharts and then put through this function. The output is hourly data. I also modified the code so that only crossovers indicate buy and sell signals. Here is the new code for the EMA10/EMA21 crossover. ***EDIT*** Named the graph wrong. library(quantmod) library(TTR) # Load presorted data x = last(y,4436) #gets the last 4436 hourly positions from the data
# Calculate the EMA indicators x$ema10 <- EMA(Cl(x),10) x$ema21 <- EMA(Cl(x),21) x$ema10.old = as.double(lag(x$ema10)) x$ema21.old = as.double(lag(x$ema21))
#omit NA's x = na.omit(x)
# Create the long and short signals sigbuy = ifelse ((x$ema10 > x$ema21) & (x$ema10.old < x$ema21.old), 1, 0) #buy signal. yesterday's EMA10 was below yesterday's EMA21 and today it crossed over sigsell = ifelse ( (x$ema10 <= x$ema21) & (x$ema10.old > x$ema21.old), -1, 0) #sell signal. yesterday's EMA10 was above yesterday's EMA21 and today it crossed under
# Lag signals to align with days in market, # not days signals were generated sigbuy <- lag(sigbuy,1) # Note k=1 implies a move *forward* sigsell <- lag(sigsell,1) # Note k=1 implies a move *forward*
# Replace missing signals with no position # (generally just at beginning of series) sigbuy[is.na(sigbuy)] <- 0 sigsell[is.na(sigsell)] <- 0
# Combine both signals into one vector sig <- sigbuy + sigsell
# Calculate Close-to-Close returns ret <- ROC(Cl(x)) ret[1] <- 0
# Calculate equity curves eq_up <- exp(cumsum(ret*sigbuy)) eq_dn <- exp(cumsum(ret*sigsell*-1)) eq_all <- exp(cumsum(ret*sig))
# Equity Chart png(filename="emacrossnew.png",width=720,height=720) plot.zoo( cbind(eq_up, eq_dn), plot.type="single", ylab=c("Long","Short"), col=c("green","red"), main="EMA10/21 Crossover Strategy:\n 2011-07-23 to 2012-01-22" ) dev.off()
# Create a chart showing mtgoxUSD png("EMAcrosschart-new.png",width=720,height=720) chartSeries(x, subset="last 4415 hours", type="line") # Add the total equity line and EMA lines addEMA(n=c(10,21),col=c("orange","blue") ) addTA(eq_all) dev.off()
# Evaluate the Strategy # install.packages("PerformanceAnalytics") require(PerformanceAnalytics) # chart equity curve, daily performance, and drawdowns png("performance-EMAcrossnew.png",height=720,width=720) charts.PerformanceSummary(ret) dev.off()
|
|
|
I was going to modify the buy sell signals of the EMA10/EMA21 crossover strategy to: sigbuy = ifelse ((x$ema10 > x$ema21) & (x$ema10.old < x$ema21.old), 1, 0) #buy signal. yesterday's EMA10 was below yesterday's EMA21 and today it crossed over sigsell = ifelse ( (x$ema10 <= x$ema21) & (x$ema10.old > x$ema21.old), -1, 0) #sell signal. yesterday's EMA10 was above yesterday's EMA21 and today it crossed under Which means, if yesterday's EMA10 is below yesterday's EMA21 AND EMA10 is greater than or equal to EMA21 today then buy Also, if yesterday's EMA10 is above yesterday's EMA21 AND EMA10 is less than or equal to EMA21 today then sell But when I run the data there is only 1 trade at the beginning of December. So what to do? I can change the data to hourly.
|
|
|
The EMA 10 / 21 Crossover Strategy is: 1. Buy when EMA 10 crosses over EMA 21. So EMA 10 > EMA 21, BUY 2. Sell when EMA 10 crosses below EMA 21. So EMA 10 < EMA 21, SELL
I think most systems based on SMA and EMA crossovers (eg GMMA) give 'buy' or 'sell' signal at the actual point of crossover - ie when (in this case) EMA10==EMA21. The rest of the time they indicate a hold. You shouldn't be changing positions that often. Edit: Nice chartage though. What package are you using for the bottom one? The bottom chart comes from PerformanceAnalytics, the middle is from quantmod, and the first is from from the zoo package. I will try this for the signals. sigbuy = ifelse ((ema10 >= ema 21) & (ema10.old < ema21.old), 1, 0) #buy signal. yesterday's EMA10 was below yesterday's EMA21 and today it crossed over sigsell = ifelse ( (ema10 =< ema 21 & (ema10.0ld > ema21.old), -1, 0) #sell signal. yesterday's EMA10 was above yesterday's EMA21 and today it crossed under
|
|
|
The EMA 10 / 21 Crossover Strategy is: 1. Buy when EMA 10 crosses over EMA 21. So EMA 10 > EMA 21, BUY 2. Sell when EMA 10 crosses below EMA 21. So EMA 10 < EMA 21, SELL
I think most systems based on SMA and EMA crossovers (eg GMMA) give 'buy' or 'sell' signal at the actual point of crossover - ie when (in this case) EMA10==EMA21. The rest of the time they indicate a hold. You shouldn't be changing positions that often. That is true. I will update the code. There is a way to see how many trades was done but I can't remember right now. I will search for that.
|
|
|
So what do others think of these graphs?
If I am reading the graphs right. The flipist method quickly starts out badly, losing equity in both the short and long trades. It never recovers its equity back.
In the EMA10/21 crossover strategy. It starts with no change in equity, but it starts to lose money because it is bad at predicting when to sell. Its long predictions seem pretty good but even the flipist method recovered pretty well in the December bull market. I would predict anyone that bought during that time did well.
If you look at the cumulative return. The flipist method actually did slightly better. Flipist -0.78 versus EMA10/21 -0.80.
Of course this is assuming I did this correctly.
Comments? Questions? Screams of fury?
|
|
|
Now for the EMA10-EMA21 crossover. ***EDIT*** Added the results table Signal # Trades % Win Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L 1 -1 132 56.81818 6.341372 -5.713935 2.840616 -4.183966 1.109808 0.678929 2 0 21 0.00000 NaN NaN NA NA NaN NA 3 1 52 57.69231 5.068756 -3.911978 3.199519 -1.747152 1.295702 1.831277 ****EDIT**** The following graphs title should say EMA10/21, not EMA5/12
|
|
|
Using modified code from here and here. I ran some backtesting in R to compare the EMA 5 EMA 21 Crossover and a random buy/sell strategy based on a coin flip. Why did I do this? I was making my own trading strategies but got stuck on making one for bullish/bearish divergence and I thought some practice may help me find out what I am doing wrong. Why not generate your own? The R package rgp lets you evolve GAs. I've got some that consistently give me better than 'buy and hold'. I just used a simple strategy as that is what these two strategies were doing. Blotter along with quantstrat allows for better trading strategies. I have not heard of the rgp. I will check it out. This was all done with the TTR and quantmod packages.
|
|
|
Flipist Results **EDIT** Added this results table: Signal # Trades % Win Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L 1 -1 93 52.68817 7.132123 -6.507944 3.636423 -4.839239 1.095910 0.7514452 2 0 1 0.00000 NaN NaN NA NA NaN NA 3 1 90 46.66667 4.551028 -4.431499 2.366660 -2.242254 1.026973 1.0554826 The first graph is the equity change based on the long (green) and short (red) positions. Main graph. This graph shows the exchange rate at Mtgox at the close of each day, along with the volume and the short and long positions. Finally, we see a chart giving the cumulative return.
|
|
|
EMA5 / EMA21 Crossover Strategy This code will also input the data, create the indicators, and then create the signals. From the daily return and the buy or sell signals, equity curves are calculated. Last charts are produced to show the performance and draw-down. Since the EMA21 requires the past 21 days of history before trading can start, the data file used for this contains the last 205 days but trading cannot start until July 23, 2011. # install.packages(c("quantmod","TTR")) library(quantmod) library(TTR) # Load presorted data x = last(y,205) #gets the last 205 daily positions from the data
# Calculate the EMA indicators ema10 <- EMA(Cl(x),10) ema21 <- EMA(Cl(x),21)
# Create the long (up) and short (dn) signals sigbuy <- ifelse(ema10 > ema21, 1, 0) sigsell <- ifelse(ema10 < ema21, -1, 0)
# Lag signals to align with days in market, # not days signals were generated sigbuy <- lag(sigbuy,1) # Note k=1 implies a move *forward* sigsell <- lag(sigsell,1) # Note k=1 implies a move *forward*
# Replace missing signals with no position # (generally just at beginning of series) sigbuy[is.na(sigbuy)] <- 0 sigsell[is.na(sigsell)] <- 0
# Combine both signals into one vector sig <- sigbuy + sigsell
# Calculate Close-to-Close returns ret <- ROC(Cl(x)) ret[1] <- 0
# Calculate equity curves eq_up <- exp(cumsum(ret*sigbuy)) eq_dn <- exp(cumsum(ret*sigsell*-1)) eq_all <- exp(cumsum(ret*sig))
# Equity Chart png("EMAcross.png",width=720,height=720) plot.zoo( cbind(eq_up, eq_dn), ylab=c("Long","Short"), col=c("green","red"), main="EMA5-EMA21 Crossover Strategy: 07-23-2011 to 01-22-2012" ) dev.off()
png(filename="emacross.png",width=720,height=720) plot.zoo( cbind(eq_up, eq_dn), plot.type="single", ylab=c("Long","Short"), col=c("green","red"), main="EMA Crossover Strategy:\n 2011-07-23 through 2012-01-22" ) dev.off()
# Create a chart showing mtgoxUSD png("EMAcrosschart.png",width=720,height=720) chartSeries(x, subset="last 184 days", type="line") # Add the total equity line addTA(eq_all) dev.off()
# Evaluate the Strategy
# install.packages("PerformanceAnalytics") require(PerformanceAnalytics) # chart equity curve, daily performance, and drawdowns png("performance-EMAcross.png",height=720,width=720) charts.PerformanceSummary(ret) dev.off()
|
|
|
Flipist Strategy Methods/code This code will input the data, create the indicators, and then create the signals. From the daily return and the buy or sell signals, equity curves are calculated. Last charts are produced to show the performance and draw-down. # Load data # install.packages(c("quantmod","TTR")) library(quantmod) library(TTR) x = last(y,184) #gets the last 184 daily positions from the data (starts at July 23, 2011)
# Calculate the random indicator set.seed(43) #include this to reproduce my results x$flip <- rbinom(length(x$.Open),1,0.5) #50% probability that 1 (heads/buy) will show and 50% probability that 0 (tails/sell) will show
# Create the buy (1) and sell (-1) signals sigbuy <- ifelse(x$flip == 1, 1, 0) sigsell <- ifelse(x$flip == 0, -1, 0)
# Lag signals to align with days in market, # not days signals were generated sigbuy <- lag(sigbuy,1) # Note k=1 implies a move *forward* sigsell <- lag(sigsell,1) # Note k=1 implies a move *forward*
# Replace missing signals with no position # (generally just at beginning of series) sigbuy[is.na(sigbuy)] <- 0 sigsell[is.na(sigsell)] <- 0
# Combine both signals into one vector sig <- sigbuy + sigsell # Calculate Close-to-Close returns ret <- ROC(Cl(x)) ret[1] <- 0
# Calculate equity curves eq_up <- exp(cumsum(ret*sigbuy)) eq_dn <- exp(cumsum(ret*sigsell*-1)) eq_all <- exp(cumsum(ret*sig))
# Equity Chart png(filename="flipist.png",width=720,height=720) plot.zoo( cbind(eq_up, eq_dn), plot.type="single", ylab=c("Long","Short"), col=c("green","red"), main="Flipist Strategy:\n 2011-07-23 through 2012-01-22" ) dev.off()
# Create a chart showing mtgoxUSD png("flipistchart.png",width=720,height=720) chartSeries(x, subset="last 184 days", type="line")
# Add the total equity line addTA(eq_all) dev.off()
# Evaluate the Strategy
# install.packages("PerformanceAnalytics") require(PerformanceAnalytics) # chart equity curve, daily performance, and drawdowns png("performance-flipist.png",height=720,width=720) charts.PerformanceSummary(ret) dev.off()
|
|
|
Methods Data aquisition Data was acquired from bitcoincharts.com. The entire data file was downloaded from the Bitcoincharts API ( http://bitcoincharts.com/t/trades.csv?symbol=bcmPPUSD&start=0 ~45mb) The data was then converted from tick data to Open/High/Low/Close/Adjusted/Volume format. The code for this conversion is: #install.packages("xts") #install the xts package if you don't already have it. Available on CRAN require(xts) ohlc <- function(ttime,tprice,tvolume,fmt) { ttime.int <- format(ttime,fmt) data.frame(time = ttime[tapply(1:length(ttime),ttime.int,function(x) {head(x,1)})], .Open = tapply(tprice,ttime.int,function(x) {head(x,1)}), .High = tapply(tprice,ttime.int,max), .Low = tapply(tprice,ttime.int,min), .Close = tapply(tprice,ttime.int,function(x) {tail(x,1)}), .Volume = tapply(tvolume,ttime.int,function(x) {sum(x)}), .Adjusted = tapply(tprice,ttime.int,function(x) {tail(x,1)})) } data <- ohlc(data$time,data$price, data$volume,"%Y%m%d%H") #converts data in CSV to OHLC data <- xts(data[,-1], order.by=data[,1]) #converts data frame to an XTS object
|
|
|
|