Let's Backtest Some Indicators

[let_me_backtest_that]

I will be posting a series of backtests for various indicators on BTC historical data.   I posted on the Aroon last week, but thought I would begin a larger thread to see if anyone is interested in this research project.   This time I am backtesting the ADX indicator

The ADX is one of the more complicated indicators in a trader's arsenal.   The reason why the ADX is complex is because it is a compilation of many indicators created by J. Welles Wilder in his book New Concepts in Technical Trading.  The book is worth the read because his ideas have been so influential on previous and contemporary traders.  (I also heard from a little bird that it is easy to find a pdf of the book from a basic google search.)

I am going to give a basic overview of the indicator then backtest it on historical BTC data.   The ADX is designed to indicate changes in momentum and the direction and strength of a trend.   The indicator does this by first calculating the Directional Movement Index (DMI).

The DMI generally consists of the difference between two different values the DM+(blue or green) and the DM-(red).  The DM+ and DM- consist of the net difference between the most recent highs and lows.  Changes in momentum are generally detected when the DM+ crosses the DM- and vice versa.    Additionally, the direction of the trend is determined by whether or not the DM+ or DM- has a higher value.

The creation of the ADX is based on the DMI and various smoothing techniques.  This involves taking the difference between the absolute value of the DM+ and DM- and dividing it by the sum of the DM+ and DM-.  The calculation is somewhat complex and can be found in more detail at stockcharts.com

http://stockcharts.com/school/doku.php?id=chart_school:technical_indicators:average_directional_index_adx

The traditional interpretation of the indicator involves using the ADX line (green or blue) to indicate a trend.  When the ADX is below 20, then the markets are moving sideways.  If it is above 20, then the markets are trending.  Trading signals are determined when there is a crossover of the DM+ and DM-.   Here is an example:

https://i.imgur.com/QC1vLGn.png

Now that we have the basics of the ADX down let's do some historical research. I am going to do two backtests using the default parameters for the ADX and a basic non-optimized position sizing strategy that trades approx $1,000 (or less) on each trade.  The position sizing strategy helps to focus on the performance of the indicator itself.  Since this is only research on an indicator and not a fully developed system, then our focus consists on how it works on various individual trades which I give below.

I will create two separate strategies.  The first will be to trade when the DM+ crosses above the DM- and sell with the reverse.  Also, we will use the ADX line above 20 as a filter.  Call this ADX Research.   

The second will merely use the DMI without the ADX filter (DMI Research).  As usual, I will use NinjaTrader for the backtest and analysis.  I will post images of the results and provide links for the code below.

Here is a summary of  the results of the ADX results:

https://i.imgur.com/0kLTgHV.png

Here is a summary of each trade

https://i.imgur.com/LwmtVkg.png

Here is short simulation of the strategy on Youtube to give you a picture of each trade.

https://www.youtube.com/watch?v=OCWIXg-xxGg&feature=youtu.be

Here is a summary of the DMI results

https://i.imgur.com/Zkif51J.png

Here is a summary of each trade

https://i.imgur.com/m3gH4sR.png

Here is a short simulation on Youtube to give you a picture of each trade.

https://www.youtube.com/watch?v=9dR97Uq6Gww&feature=youtu.be

Here are some things to notice:

1. The ADX system does succeed at filtering out many whipsaw trades when the market is flat.  However, this increase in % profitable also comes with the cost of less total profit and a lower profit fact.

2. The total drawdown of each system is the same.  Importantly, the ADX filter does not decrease drawdown risk.  The ADX filter does not perform better on a return to risk basis.

3. One reason that the DMI indicator does well is because BTC prices trend and have low volatility when moving sideways.  If these features were to change, then the DMI indicator would do less well.   I am merely pointing out one of the many limitations that occur through backtesting.

You can download the code for on my blog
http://www.signalstrengthfinance.com/better-know-an-indicator-adx/

You can also find a free daily updating link to upload historical BTC data into NinjaTrader here

http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

Trade Well and Prosper

[wobber]

Absolutely brilliant! Waiting for more! Thanks!

[HeliKopterBen]

Nice!  You should charge for your services.

[let_me_backtest_that]

DOES MACD BEAT BUY AND HOLD?

The MACD is a fairly straightforward momentum indicator.  It consists of the difference between a 12 and 26 bar EMA.  The difference between these two moving averages is the MACD.  It is smoothed by a 9 Day EMA that is called the "signal line."

The MACD  is usually plotted with a histogram that represents the difference between the MACD and the signal line.  The histogram can give either positive or negative values.  On the traditional interpretation, a buy signal occurs when the signal line crosses the MACD and a sell signal occurs when the signal line crosses below the MACD.   See the example below:

https://i.imgur.com/Llda1qf.png

The MACD is a popular indicator on BTC forums.  How does it hold up historically? Does it beat  buy and hold?  How does a long and short strategy perform?  We will perform two backtests.  The first will be a MACD long and short strategy.  The second will be a long only strategy.  Finally, we will compare this to buy and hold.

We will make the following assumptions: (1) The backtest will cover 1/'1/2012-6/1/2012.  (2) We will reinvest profits beginning with a $1,000 account.  (3) We will keep the default parameters for the MACD.

EDIT: The backtest is performed over 1/1/2012-6/1/2014

How does the long/short strategy perform?

The strategy entered 41 total trades - 59% were profitable- net profit was $9,820- there was a 98% a drawdown and a profit factor of 6.42.  See below

https://i.imgur.com/UcJD6aH.png

I am also attaching a youtube replay of each trade in the system to get a better feel for how the system would've worked.

https://www.youtube.com/watch?v=3mwCZOr-6fc&feature=youtu.be

You will notice that the system trades a different number of BTC depending on the account size.  Once it hit a large drawdown at the end of April 2013, then the system did not have the equity to trade larger quantities of BTC.

https://i.imgur.com/vwQTIqI.png

Now let's backtest the long only system.

The strategy entered a total of 21 trades - 76.19% were profitable - net profit was $135,000,  there was a 19% drawdown and a profit factor of 60.41.

https://i.imgur.com/ZFZLGI5.png

Here is the youtube replay.

https://www.youtube.com/watch?v=jE_eDDAe10w&feature=youtu.be

Ultimately, MACD outperforms a buy/hold strategy as seen below.  The Buy/Hold had a net profit of $126,600 and a drawdown of 73%.  (For Buy/Hold I entered a buy at the open of everyday with the same amount of BTC 202 on each trade to make the performance equivalent. )

https://i.imgur.com/kz70wD9.png

Here are a few takeaways:

1. Ultimately, MACD long only does beat buy and hold on both a net profit and drawdown.   It caught every major uptrend even though it lagged the market.

2. However, MACD long only underperformed buy and hold during the large price increases.  At one point, someone would've seen their BTC holdings drop from 202 to 58 due to price increases, lag and some losing trades.

3.  The drawdown  in USD on the MACD strategy is superior to buy/hold.  Returns should always be related to risk.  Someone trading the MACD would've had a $1,982 drawdown in USD, while the buy/hold drawdown equaled $159,000.

You can download updated BTC historical data formatted for NinjaTrader here:

http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

You can download the code for your own analysis on my blog;

http://www.signalstrengthfinance.com/bitcoinchartsmacd/


Trade Well and Prosper

[oda.krell]

Very interesting, as before. Thanks!

"long/short" MACD strategy means on a buy signal, the system buys in with the entire account value, and on a sell signal, it sells and actively shorts? While "long" only means the system only buys and sells, without entering a margin position?

[let_me_backtest_that]

Very interesting, as before. Thanks!

"long/short" MACD strategy means on a buy signal, the system buys in with the entire account value, and on a sell signal, it sells and actively shorts? While "long" only means the system only buys and sells, without entering a margin position?

Yes, thanks for clarifying that.  I think its important to show how a going long and actively selling short with one's full account value can have some pretty horrendous consequences.

EDIT: I had the order backwards in the post.  It now makes more sense.

[oda.krell]

I keep being surprised by how well daily MACD still seems to work for this market, even after it being mentioned and discussed over and over again. In fact, I suspect during the 2013/2014 bear market it would have performed even better than in your 2012 data backtest.

[fallinglantern]

Interesting to read that trading MACD beats buy hold. Does that factor in the tax savings (long term capital gains vs. short term capital gains) inherent in the buy+hold strategy that MACD trading doesn't have?

[BitchicksHusband]

Outperformed "buy and hold DAILY".  That's not a valid measure.  Nobody buys and holds daily.

Also: "The backtest will cover 1/'1/2012-6/1/2012", the cherry-picked deadest period possible for buy and hold.

[oda.krell]

Outperformed "buy and hold DAILY".  That's not a valid measure.  Nobody buys and holds daily.

AFAIK, he doesn't include slippage and fees in his calculations, so assuming no gaps between daily candles, I believe selling at candle close and buying at candle open is just his way to simulate a buy&hold strategy. It is (for the reasons mentioned above) equivalent to "real" b&h.

[let_me_backtest_that]

Outperformed "buy and hold DAILY".  That's not a valid measure.  Nobody buys and holds daily.

AFAIK, he doesn't include slippage and fees in his calculations, so assuming no gaps between daily candles, I believe selling at candle close and buying at candle open is just his way to simulate a buy&hold strategy. It is (for the reasons mentioned above) equivalent to "real" b&h.

Yes, that's right.  I am buy on the open and selling on the close without slippage and commissions to simulate both the gains and losses of buy/hold.  Also, I don't see how the data is cherry picked, not intentionally anyway, I am using the same data set as the other indicator backtests.  The bitstamp data is pretty shoddy before 1/1/2012.   In fact, starting at the beginning of the most recent move to 1,200USD and decline would've given the best results because the MACD has had a very good record of shorting during the recent bear market.

[let_me_backtest_that]

Interesting to read that trading MACD beats buy hold. Does that factor in the tax savings (long term capital gains vs. short term capital gains) inherent in the buy+hold strategy that MACD trading doesn't have?

Good point. I am not factoring in slippage, commissions or taxes which should be thoroughly taken into account before using this as a stand alone strategy.  I want to emphasize that this thread is just for testing indicators not fully developed strategies.

[BitchicksHusband]

Outperformed "buy and hold DAILY".  That's not a valid measure.  Nobody buys and holds daily.

AFAIK, he doesn't include slippage and fees in his calculations, so assuming no gaps between daily candles, I believe selling at candle close and buying at candle open is just his way to simulate a buy&hold strategy. It is (for the reasons mentioned above) equivalent to "real" b&h.

Yes, that's right.  I am buy on the open and selling on the close without slippage and commissions to simulate both the gains and losses of buy/hold.  Also, I don't see how the data is cherry picked, not intentionally anyway, I am using the same data set as the other indicator backtests.  The bitstamp data is pretty shoddy before 1/1/2012.   In fact, starting at the beginning of the most recent move to 1,200USD and decline would've given the best results because the MACD has had a very good record of shorting during the recent bear market.

Change it to first six months of 2013 and see if it still beats it.

[oda.krell]

@BitchicksHusband Let it go, man. Just by eyeballing MACD, I can already tell that during the first 6 months of 2013, b&h would perform better. Then skip forward to 2013/2014, and MACD signal is better again.

What matters are overall profits, and this entire thread is basically an invitation to backtest your own ideas. Instead, you come in here, with your "only b&h makes sense" attitude, and take cheap shots. Why not _download_ the motherf'ing program and test it for yourself?

EDIT: that said, any reason why you don't run the test over the entire trading history, let_me_backtest_that?

EDIT 2: sounded more aggressive than intended. apologies.

[let_me_backtest_that]

@BitchicksHusband Let it go, man. Just by eyeballing MACD, I can already tell that during the first 6 months of 2013, b&h would perform better. Then skip forward to 2013/2014, and MACD signal is better again.

What matters are overall profits, and this entire thread is basically an invitation to backtest your own ideas. Instead, you come in here, with your "only b&h makes sense" attitude, and take cheap shots. Why not _download_ the motherf'ing program and test it for yourself?

EDIT: that said, any reason why you don't run the test over the entire trading history, let_me_backtest_that?

EDIT 2: sounded more aggressive than intended. apologies.

Thanks oda.krell-  You are right, this post is a an invitation to test people's own ideas with a some data and code to get started with.  I am responsible for some of the misunderstanding because the backtest length was a typo.  I am including the MACD backtest info up till 6-29-14.  People can see that it is similar to the other results.

https://i.imgur.com/bFcicsE.png

[Wary]

Thanks for the tests!

I've looked at the MACD video. The indicator seems to be good for quiet periods, but not for the rally time.

At April's rally it
    1 Completely missed the rally's top
    2 Sold exactly at the following bottom
    3 Bought exactly at the following top

At October's rally it was planning to replicated April's disaster   , but was saved by an accident: it was going to sell at the bottom again, but since the rally had two overlapping peaks, it sold around the second peak.

Ergo: 1d MACD is good at quiet periods, but at rally time, because of the lag, it's signals becoming worse than useless (namely harmful).

(Sorry for harsh words. My anger is directed not to participants of this discussion, but to the indicator itself. I guess it wouldn't mind )

EDIT: grammar.

[let_me_backtest_that]

Thanks for the tests!

I've looked at the MACD video. The indicator seems to be good for quiet periods, but not for the rally time.

At April's rally it
    1 Completely missed the rally's top
    2 Sold exactly at the following bottom
    3 Bought exactly at the following top

At October's rally it was planning to replicated April's disaster   , but was saved by an accident: it was going to sell at the bottom again, but since the rally had two overlapping peaks, it sold around the second peak.

Ergo: 1d MACD is good at quiet periods, but at rally time, because of the lag, it's signals becoming worse than useless (namely harmful).

(Sorry for harsh words. My anger is directed not to participants of this discussion, but to the indicator itself. I guess it wouldn't mind )

EDIT: grammar.

My purpose here is to give the backtested information so people can determine the usefulness of the indicator for themselves.  One reason I provide the videos is so that people can see the exact issues that you point out.  I will say this about the MACD, as with any lagging momentum indicator, it will not catch the exact top.  Also, any price that decreases precipitously in one or two bars will give back a lot of gains. 

What I am less comfortable with is the following inference: If the MACD lags (and thereby doesn't sell at the top of a rally), then it is only useful in non-trending periods.  Also, I want to point out that my purpose is not to give  full blown strategy.  I think a person could find various exit parameters alongside the MACD indicator that would be useful.  Just something to think about. 

[Wary]

Thanks for the tests!

I've looked at the MACD video. The indicator seems to be good for quiet periods, but not for the rally time.

At April's rally it
    1 Completely missed the rally's top
    2 Sold exactly at the following bottom
    3 Bought exactly at the following top

At October's rally it was planning to replicated April's disaster   , but was saved by an accident: it was going to sell at the bottom again, but since the rally had two overlapping peaks, it sold around the second peak.

Ergo: 1d MACD is good at quiet periods, but at rally time, because of the lag, it's signals becoming worse than useless (namely harmful).

(Sorry for harsh words. My anger is directed not to participants of this discussion, but to the indicator itself. I guess it wouldn't mind )

EDIT: grammar.

My purpose here is to give the back-tested information so people can determine the usefulness of the indicator for themselves.  One reason I provide the videos is so that people can see the exact issues that you point out.  I will say this about the MACD, as with any lagging momentum indicator, it will not catch the exact top.  Also, any price that decreases precipitously in one or two bars will give back a lot of gains.  

What I am less comfortable with is the following inference: If the MACD lags (and thereby doesn't sell at the top of a rally), then it is only useful in non-trending periods.  Also, I want to point out that my purpose is not to give  full blown strategy.  I think a person could find various exit parameters alongside the MACD indicator that would be useful.  Just something to think about.  

Your back-tests gave us very useful information and it is much appreciated. Especially appreciated is the videos, which gives much more clear picture for table-challenged persons like myself  
I realize that no single indicator can give perfect signals. I also realize that catching the exact tops&bottoms is Holy Grail that can be searched for but hardly can be found.  What I'm trying to figure out about 1d MACD is when it should be used and when avoided. It seems to be it is more useful for quiet time, i.e. not during the parabolic rise or immediately after. It is too lagging for it. The quiet time can be trending, just things should be moving slow enough for the indicator to catch up. Especially good it is for "buy" signals. For violent time and "sell" signals it should be used more cautiously, with more confirmations or not at all. Does it make any sense?

[BitchicksHusband]

First off, Oda, he said it beats buy and hold and I intend to make him prove it more than his simple statements.  He brought it up, he should have expected criticism.

My point: Buy and Hold is easier to perform perfectly than ADX.  So I have the following questions:

How do you perform all these trades?

How do you perform all these trades perfectly? 

Do you ever sleep?

Do you program a bot to do them for you? 

Do you leave your money on an exchange such as Mt. Gox?  If so, your carefully traded stash just went to zero.  If you say Bitstamp, how do you know Bitstamp isn't the next Mt. Gox?

All of these questions must be answered adequately in order to beat buy and hold in a cold storage wallet, which takes zero effort and gives 93% of the same result (by the numbers given).

Mistakes can easily lose you that 7%.

[oda.krell]

@BitchicksHusband

Okay, I'll jump to let_me_backtest_that's defense one more time... I'm starting to feel I'm whiteknighting here :D

Anyway. Your points are all absolutely valid questions.

However, let_me_backtest_that said clearly, that he doesn't provide a fully developed strategy. Call it "a hypothetical strategy based on a single signal", if you want.

This is not just a semantic difference. Understanding the difference between a single indicators's signal and a strategy is the difference between making profitable trades and incurring losses in the long run. I learned that lesson the last year.

The backtest is just the evaluation of a single indicator signal, and it potentially invites the interested trader to continue investigating it. Since the signal all by itself slightly outperforms b&h (in this very theoretical, non-practical setting), it is a somewhat interesting invitation.

[let_me_backtest_that]

Thanks for the tests!

I've looked at the MACD video. The indicator seems to be good for quiet periods, but not for the rally time.

At April's rally it
    1 Completely missed the rally's top
    2 Sold exactly at the following bottom
    3 Bought exactly at the following top

At October's rally it was planning to replicated April's disaster   , but was saved by an accident: it was going to sell at the bottom again, but since the rally had two overlapping peaks, it sold around the second peak.

Ergo: 1d MACD is good at quiet periods, but at rally time, because of the lag, it's signals becoming worse than useless (namely harmful).

(Sorry for harsh words. My anger is directed not to participants of this discussion, but to the indicator itself. I guess it wouldn't mind )

EDIT: grammar.

My purpose here is to give the back-tested information so people can determine the usefulness of the indicator for themselves.  One reason I provide the videos is so that people can see the exact issues that you point out.  I will say this about the MACD, as with any lagging momentum indicator, it will not catch the exact top.  Also, any price that decreases precipitously in one or two bars will give back a lot of gains.  

What I am less comfortable with is the following inference: If the MACD lags (and thereby doesn't sell at the top of a rally), then it is only useful in non-trending periods.  Also, I want to point out that my purpose is not to give  full blown strategy.  I think a person could find various exit parameters alongside the MACD indicator that would be useful.  Just something to think about.  

Your back-tests gave us very useful information and it is much appreciated. Especially appreciated is the videos, which gives much more clear picture for table-challenged persons like myself  
I realize that no single indicator can give perfect signals. I also realize that catching the exact tops&bottoms is Holy Grail that can be searched for but hardly can be found.  What I'm trying to figure out about 1d MACD is when it should be used and when avoided. It seems to be it is more useful for quiet time, i.e. not during the parabolic rise or immediately after. It is too lagging for it. The quiet time can be trending, just things should be moving slow enough for the indicator to catch up. Especially good it is for "buy" signals. For violent time and "sell" signals it should be used more cautiously, with more confirmations or not at all. Does it make any sense?

I think it depends on what you mean by quiet time.  During times of really low volatility, the MACD can provide whipsaws.  Maybe you are asking this question, "Should you rely solely on the MACD when deciding to enter or exit the market during times of high volatility?"  IMO, no.

[let_me_backtest_that]

First off, Oda, he said it beats buy and hold and I intend to make him prove it more than his simple statements.  He brought it up, he should have expected criticism.

My point: Buy and Hold is easier to perform perfectly than ADX.  So I have the following questions:

How do you perform all these trades?

How do you perform all these trades perfectly? 

Do you ever sleep?

Do you program a bot to do them for you? 

Do you leave your money on an exchange such as Mt. Gox?  If so, your carefully traded stash just went to zero.  If you say Bitstamp, how do you know Bitstamp isn't the next Mt. Gox?

All of these questions must be answered adequately in order to beat buy and hold in a cold storage wallet, which takes zero effort and gives 93% of the same result (by the numbers given).

Mistakes can easily lose you that 7%.

I am fine with criticism.  I should note that I said it outperforms buy/hold on a risk adjusted basis.  In other words, you get a higher net profit and less of a drawdown.  I should also remind everyone that I did note that it underperformed for a significant amount of time.  I am not trying to promote the MACD as some holy grail.  Now to your question about trading?

Where and how do you perform these trades?  Well, I set it up for daily trading, so one could do this manually merely by trading at the opening of each bar UTC time.  You can also create automatic trading strategies.  I am using NinjaTrader for both backtesting and automated strategy execution. 

However, I want to emphasize AGAIN....that this is only backtested research and not a complete trading strategy. People can draw their own inferences as to its value.

[twiifm]

First off, Oda, he said it beats buy and hold and I intend to make him prove it more than his simple statements.  He brought it up, he should have expected criticism.

My point: Buy and Hold is easier to perform perfectly than ADX.  So I have the following questions:

How do you perform all these trades?

How do you perform all these trades perfectly? 

Do you ever sleep?

Do you program a bot to do them for you? 

Do you leave your money on an exchange such as Mt. Gox?  If so, your carefully traded stash just went to zero.  If you say Bitstamp, how do you know Bitstamp isn't the next Mt. Gox?

All of these questions must be answered adequately in order to beat buy and hold in a cold storage wallet, which takes zero effort and gives 93% of the same result (by the numbers given).

Mistakes can easily lose you that 7%.

I am fine with criticism.  I should note that I said it outperforms buy/hold on a risk adjusted basis.  In other words, you get a higher net profit and less of a drawdown.  I should also remind everyone that I did note that it underperformed for a significant amount of time.  I am not trying to promote the MACD as some holy grail.  Now to your question about trading?

Where and how do you perform these trades?  Well, I set it up for daily trading, so one could do this manually merely by trading at the opening of each bar UTC time.  You can also create automatic trading strategies.  I am using NinjaTrader for both backtesting and automated strategy execution. 

However, I want to emphasize AGAIN....that this is only backtested research and not a complete trading strategy. People can draw their own inferences as to its value.

Thats pretty cool.   Can I ask which software you use to backtest?   Did you automate yourself?

I use an indicator called 3-10 oscillator for trading stocks.   Its a MACD set to 3/10/16

Works pretty well but I'm manually doing the trades and sometimes I miss the signals.   I'd be interested to backtest this and also automate my trading system

[let_me_backtest_that]

First off, Oda, he said it beats buy and hold and I intend to make him prove it more than his simple statements.  He brought it up, he should have expected criticism.

My point: Buy and Hold is easier to perform perfectly than ADX.  So I have the following questions:

How do you perform all these trades?

How do you perform all these trades perfectly? 

Do you ever sleep?

Do you program a bot to do them for you? 

Do you leave your money on an exchange such as Mt. Gox?  If so, your carefully traded stash just went to zero.  If you say Bitstamp, how do you know Bitstamp isn't the next Mt. Gox?

All of these questions must be answered adequately in order to beat buy and hold in a cold storage wallet, which takes zero effort and gives 93% of the same result (by the numbers given).

Mistakes can easily lose you that 7%.

I am fine with criticism.  I should note that I said it outperforms buy/hold on a risk adjusted basis.  In other words, you get a higher net profit and less of a drawdown.  I should also remind everyone that I did note that it underperformed for a significant amount of time.  I am not trying to promote the MACD as some holy grail.  Now to your question about trading?

Where and how do you perform these trades?  Well, I set it up for daily trading, so one could do this manually merely by trading at the opening of each bar UTC time.  You can also create automatic trading strategies.  I am using NinjaTrader for both backtesting and automated strategy execution. 

However, I want to emphasize AGAIN....that this is only backtested research and not a complete trading strategy. People can draw their own inferences as to its value.

Thats pretty cool.   Can I ask which software you use to backtest?   Did you automate yourself?

I use an indicator called 3-10 oscillator for trading stocks.   Its a MACD set to 3/10/16

Works pretty well but I'm manually doing the trades and sometimes I miss the signals.   I'd be interested to backtest this and also automate my trading system

Thanks...glad you like the thread.

I use NinjaTrader for my backtesting and order execution.  You can download NT for free here: http://ninjatrader.com/download-registration.php

We provide a continually updating link for BTC data that can be imported into NT for free as well as real time data in our Beta release of Bitconnector.   This will allow you to backtest and automate strategies fairly easily.   You can find both here: http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

I was interested in how the 3-10 oscillator would work. Here is the backtest for it on BTC (assuming that 3=fast, 10=slow and 16 = signal line)

 https://i.imgur.com/CiMkrda.png

[twiifm]

Thanks for doing the backtest.  I guess it doesn't work as well as the standard MACD long only

The slow/fast signals are on shorter periods so its expected to have more trades.  I'm curious to why more drawdowns

[let_me_backtest_that]

The " Turtle Traders" are a legendary group of traders taught by commodity trader Richard Dennis.   He and his partner had a debate about whether or not trading could be taught.  They conducted an experiment by placing an ad in the Wall Street Journal and giving a group of traders some accounts and a system.  The turtles ended up making more than $150 million.    Richard Dennis came up with the idea after visiting a turtle farm in Singapore and thought that traders could be "grown" like turtles.

The Turtle Trading System's entry and exit parameters are quite simple.  They are based on the Donchian channel breakout indicator.  The Donchian channel merely measures the highest high and the lowest low in a series of bars.  For example, if you set the lookback to 10 bars, then the system will give you the highest high/lowest low in the last 10 bars.

The turtle trading system had two different entry systems that could be used. Each one based is based on the Donchian Channel breakout.  The one we'll consider goes long/short at the highest high/lowest low of the last 20 days.   It exits when when there is a new high/low 10 bars ago.   Here is an example:

https://i.imgur.com/SEIUCFq.png

If we backtest the indicator, then we will get some information about how it performed historically on BTC prices.  This information can be valuable when thinking about designing a system.   However, my primary purpose in this post is to distinguish between an indicator and a strategy.

Let's take the Turtle System which is given for fee by one of the former turtles:

http://bigpicture.typepad.com/comments/files/turtlerules.pdf

Notice that there are at least six things that go into the complete strategy:

1. Markets - What to buy or sell

2. Position Sizing - How much to buy or sell

3. Entries - When to buy or sell

4.  Stops - When to get out of a losing position

5. Exits - When to get out of a winning position

6. Tactics - How to buy or sell

Its important to notice that the entry/exit is only a small part of the entire strategy.  I will illustrate by running two  backtests and compare this with buy/hold.

The first will be a backtest of the Donchian indicator itself as determined by the Turtle trading strategy.  The second will be a backtest that includes the position sizing parameters of the Turtle system.  Each backtest will assume an account of $10,000 with profits reinvested and trade from 1/1/2012 thru 6/1/2014.

Here are the results for trading the Donchian channel indicator itself.  (Note: all profits are reinvested every trade because there is no position sizing strategy.)  The results include a 22 trades- 50% profitable- a profit factor of 1- a drawdown of 98%- and a net profit of $3,482.

https://i.imgur.com/5REjbmN.png

Here is a youtube playback of the system.  (You will notice the large drawdown occurs in April 2013).

http://youtu.be/aTlpsat74dY

As is obvious, buy/sell signals generated from the indicator severely underperformed buy/hold.    Buying $10,000 worth of Bitcoin would've netted you a nice profit of 1.38 mill.  As you can see below:

https://i.imgur.com/gqFHam9.png

Now, let's consider what happens when we include the position sizing algorithm that the Turtle's used.  We will follow their rules of only entering 4 units per trade.  Where a unit is defined by (0.01 x AccountSize)/ Dollar per point x the Average True Range).  This position sizing algorithm leads them to enter less money when markets are volatile and more when they are not.   Here are the results using the same time period.

There were 22 trades-50% profitable-a profit factor of 5.27- a drawdown of 98% and a net profit of 3.61 mill.  See below:

https://i.imgur.com/4efSiUa.png

I will also add the Youtube playback so you that you can see how the position sizing affected the trading.  Notice the awful trade that occurs in April 2013 doesn't  destroy all the equity because the trade size is risk adjusted for larger volatility.

http://youtu.be/cT5wu_ODap0

 

Here are some takeaways:

1. An indicator is not a system.  It should not be used as a complete system when trading.  It should not be evaluated as a complete system when backtesting.
2. An indicator that, by itself, underperforms buy/hold can outperform buy/hold when placed in a good strategy.

3.You can learn a lot from a Turtle.  Personally, the 34 pages providedis one of the most concise and helpful pieces on trading that I've read.  http://bigpicture.typepad.com/comments/files/turtlerules.pdf

As always, the code is given at my blog

http://www.signalstrengthfinance.com/distinguishing-a-strategy-from-an-indicator-a-lesson-from-the-turtles/

Historical Bitcoin Data can be downloaded for free in NinjaTrader format  at the link below:

http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

Trade Well and Prosper

[twiifm]

The " Turtle Traders" are a legendary group of traders taught by commodity trader Richard Dennis.   He and his partner had a debate about whether or not trading could be taught.  They conducted an experiment by placing an ad in the Wall Street Journal and giving a group of traders some accounts and a system.  The turtles ended up making more than $150 million.    Richard Dennis came up with the idea after visiting a turtle farm in Singapore and thought that traders could be "grown" like turtles.

The Turtle Trading System's entry and exit parameters are quite simple.  They are based on the Donchian channel breakout indicator.  The Donchian channel merely measures the highest high and the lowest low in a series of bars.  For example, if you set the lookback to 10 bars, then the system will give you the highest high/lowest low in the last 10 bars.

The turtle trading system had two different entry systems that could be used. Each one based is based on the Donchian Channel breakout.  The one we'll consider goes long/short at the highest high/lowest low of the last 20 days.   It exits when when there is a new high/low 10 bars ago.   Here is an example:

https://i.imgur.com/SEIUCFq.png

If we backtest the indicator, then we will get some information about how it performed historically on BTC prices.  This information can be valuable when thinking about designing a system.   However, my primary purpose in this post is to distinguish between an indicator and a strategy.

Let's take the Turtle System which is given for fee by one of the former turtles:

http://bigpicture.typepad.com/comments/files/turtlerules.pdf

Notice that there are at least six things that go into the complete strategy:

1. Markets - What to buy or sell

2. Position Sizing - How much to buy or sell

3. Entries - When to buy or sell

4.  Stops - When to get out of a losing position

5. Exits - When to get out of a winning position

6. Tactics - How to buy or sell

Its important to notice that the entry/exit is only a small part of the entire strategy.  I will illustrate by running two  backtests and compare this with buy/hold.

The first will be a backtest of the Donchian indicator itself as determined by the Turtle trading strategy.  The second will be a backtest that includes the position sizing parameters of the Turtle system.  Each backtest will assume an account of $10,000 with profits reinvested and trade from 1/1/2012 thru 6/1/2014.

Here are the results for trading the Donchian channel indicator itself.  (Note: all profits are reinvested every trade because there is no position sizing strategy.)  The results include a 22 trades- 50% profitable- a profit factor of 1- a drawdown of 98%- and a net profit of $3,482.

https://i.imgur.com/5REjbmN.png

Here is a youtube playback of the system.  (You will notice the large drawdown occurs in April 2013).

http://youtu.be/aTlpsat74dY

As is obvious, buy/sell signals generated from the indicator severely underperformed buy/hold.    Buying $10,000 worth of Bitcoin would've netted you a nice profit of 1.38 mill.  As you can see below:

https://i.imgur.com/gqFHam9.png

Now, let's consider what happens when we include the position sizing algorithm that the Turtle's used.  We will follow their rules of only entering 4 units per trade.  Where a unit is defined by (0.01 x AccountSize)/ Dollar per point x the Average True Range).  This position sizing algorithm leads them to enter less money when markets are volatile and more when they are not.   Here are the results using the same time period.

There were 22 trades-50% profitable-a profit factor of 5.27- a drawdown of 98% and a net profit of 3.61 mill.  See below:

https://i.imgur.com/4efSiUa.png

I will also add the Youtube playback so you that you can see how the position sizing affected the trading.  Notice the awful trade that occurs in April 2013 doesn't  destroy all the equity because the trade size is risk adjusted for larger volatility.

http://youtu.be/cT5wu_ODap0

 

Here are some takeaways:

1. An indicator is not a system.  It should not be used as a complete system when trading.  It should not be evaluated as a complete system when backtesting.
2. An indicator that, by itself, underperforms buy/hold can outperform buy/hold when placed in a good strategy.

3.You can learn a lot from a Turtle.  Personally, the 34 pages providedis one of the most concise and helpful pieces on trading that I've read.  http://bigpicture.typepad.com/comments/files/turtlerules.pdf

As always, the code is given at my blog

http://www.signalstrengthfinance.com/distinguishing-a-strategy-from-an-indicator-a-lesson-from-the-turtles/

Historical Bitcoin Data can be downloaded for free in NinjaTrader format  at the link below:

http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

Trade Well and Prosper

excellent post!  after trading for a while  I find that you can't rely on indicators too much.  A lot of trading is mental and risk management is a big part of that

[let_me_backtest_that]

How About the Parabolic SAR

The Parabolic SAR (hereafter SAR)  is a trend indicator.   It was one of the indicators published by Welles Wilder in New Concepts in Technical Trading Systems.  It plots a dot above or below the price action to indicate the existence of a trend.  If the dot is plotted above the price, then a bearish trend is indicated.  A dot below the price action indicates a bullish trend.

The SAR indicates a trend reversal when the high/low move below the value of the dot plotted.  These trend changes are designed to detect reversals.    Hence the name, "Stop and Reverse."     Here is an example:

https://i.imgur.com/ewFiQwg.png

Before we go deeper into the data, I will just state that this indicator performs poorly at various points on the backtest.  Perhaps that is all you may want to know.  If it is, then it still shows the importance of backtesting an idea before trading with it.  Sometimes the best information you can get from a backtest is <em>not</em> to use it.

Additionally, as I've said before, an indicator is not a stand alone system.  So, if people have some interesting ideas on using the SAR in other ways, then I will do my best to integrate some new ideas.

Now that we have a bit of an introduction to the SAR let's see how it does on historical BTC prices.  We will start with a simple backtest that  assumes 1 BTC  per trade (no slippage and commissions) and merely enter when the SAR indicates a reversal long or short.  The backtest will run from 1/1/2012 until 7/1/2014.  The indicator gives 52 trades, a $472 net profit, a 59% peak-to-valley drawdown and a profit factor of 1.61.

https://i.imgur.com/HKPnpka.png

One thing to point out is that all the losses came from the short side of the signals.   The long part of the strategy had a net profit of $564.00  and a drawdown of 46%.

We can contrast can see that this underperforms a buy/hold strategy.  A buy/hold strategy with 1 BTC has a profit of $670 and a peak-to-valley drawdown of 73%.

https://i.imgur.com/zS8IUZ7.png

Much of this underperformance is due to whipsaws and early exits on long trends.  Each of these features is undesirable from the perspective of a trend following indicator.   Here is a replay of the results so you can get a feel for the SAR signals.

https://www.youtube.com/watch?v=MIaRYYoU6W4&feature=youtu.be

Here are some observations from the backtest:

1. The indicator has many whipsaws when prices are moving sideways.  This suggests that its usage in a system would benefit from a trend filter.  In fairness, Wilder suggested that it be used with the ADX to indicate trend following.

2. The indicator exits a trend early.  Generally, "letting your profits run" is the motto of trend following, so this is undesirable from that perspective.  (You can see from the backtest that the exit at $600 was far from the end of the trend).

3. Could it be used as a trailing stop (i.e., a stop that continually moves up to capture profit before the trend reverses)?  Perhaps, IMO there are better methods for finding a trailing stop for each particular strategy.

The backtests were performed using NinjaTrader which can be downloaded for free.  We provide historical BTC data that is updated daily and formatted for NinjaTrader  for free.

https://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

Also, the code is available for download and can be imported directly into NinjaTrader

https://www.signalstrengthfinance.com/how-does-the-parabolic-sar-work/

Trade Well and Prosper

[let_me_backtest_that]

I got a suggestion to use the SAR on the weekly bars.  It does indicate the previous two bubbles pretty early.  The net profit doesn't change much, but i am assuming the person who used it didn't rely on the SAR as an exit signal. 

see below:



https://i.imgur.com/6K8GMmC.png

https://i.imgur.com/6K8GMmC.png

[efreeti]

Past performance is not a predictor of future results.

Back testing can be useful if you can control your risk.

[Wary]

Thanks, good work, very informative. Let God bless you with success in trading and with crowds of customers.  

(BTW, <em></em> doesn't work here. You can use [ b ] [ /b ] or  [ i ]  [ /i ] or [ color=red ][ /color ] instead)

EDIT: A suggestion: since most of BTC traders trade to increase their BTC holdings, it would make sense to calculate profits and losses in BTC, rather than in dollars.

[let_me_backtest_that]

Past performance is not a predictor of future results.

Back testing can be useful if you can control your risk.

Excellent point and worth repeating.  Everyone engaged in backtesting strategies needs to be aware of any and all pitfalls.   

[let_me_backtest_that]

Thanks, good work, very informative. Let God bless you with success in trading and with crowds of customers.  

(BTW, <em></em> doesn't work here. You can use [ b ] [ /b ] or  [ i ]  [ /i ] or [ color=red ][ /color ] instead)

EDIT: A suggestion: since most of BTC traders trade to increase their BTC holdings, it would make sense to calculate profits and losses in BTC, rather than in dollars.

Thanks for the suggestions!  Placing the data in terms of accumulation of bitcoins is an interesting goal.  I like it, but I wonder the best way to proceed.   If we proceed this way, then I would need to include a position sizing strategy in my backtests.  Position sizing strategies should differ from one strategy to another.  So, I would have to somehow choose one of them.    However, I don't want to be give any misleading results by unintentionally curve fitting my results. 

If other posters here have some suggestions.  I would be very happy to hear them.

[oda.krell]

re: SAR

I tried using the SAR myself, but find it alternatively too lagging and too "trigger happy" i.e. yielding too many, often unprofitable, signals. There's one positive observation I made however: since it is a momentum indicator with a "built in" price target, it can be useful for exiting a trade on a rebound, i.e. say I sell based on some indicator combination, buy back short term because I determine the "bottom is in", and then exit the trade = sell when price hits the SAR target. Usually relatively low time resolution, between 1h and 12h.

EDIT: A suggestion: since most of BTC traders trade to increase their BTC holdings, it would make sense to calculate profits and losses in BTC, rather than in dollars.

I thought the same at first, but then I came to the conclusion (please correct me if I'm wrong): a USD maximized holding and a BTC maximized holding are equivalent, as long as you just tack on one final 'buy' order to the end (i.e. calculate how many coins the final USD value buys you). Correct?

To see why: imagine you don't take a trade that is suggested by the tested system, and that turns out profitable. Your BTC value will be equal, but your USD to buy coins is higher at the next signal, so being USD optimal is equivalent to being BTC optimal. The only difference is that the final value of the holding is given in USD, so convert it to BTC according to current price.

@l_m_b_t Not entirely sure I understand your point about needing to include position sizing and curve fitting because of calculating BTC holdings. Then again, maybe I misunderstood wary's question.

[Wary]

re: SAR

I tried using the SAR myself, but find it alternatively too lagging and too "trigger happy" i.e. yielding too many, often unprofitable, signals. There's one positive observation I made however: since it is a momentum indicator with a "built in" price target, it can be useful for exiting a trade on a rebound, i.e. say I sell based on some indicator combination, buy back short term because I determine the "bottom is in", and then exit the trade = sell when price hits the SAR target. Usually relatively low time resolution, between 1h and 12h.

EDIT: A suggestion: since most of BTC traders trade to increase their BTC holdings, it would make sense to calculate profits and losses in BTC, rather than in dollars.

I thought the same at first, but then I came to the conclusion (please correct me if I'm wrong): a USD maximized holding and a BTC maximized holding are equivalent, as long as you just tack on one final 'buy' order to the end (i.e. calculate how many coins the final USD value buys you). Correct?

To see why: imagine you don't take a trade that is suggested by the tested system, and that turns out profitable. Your BTC value will be equal, but your USD to buy coins is higher at the next signal, so being USD optimal is equivalent to being BTC optimal. The only difference is that the final value of the holding is given in USD, so convert it to BTC according to current price.

@l_m_b_t Not entirely sure I understand your point about needing to include position sizing and curve fitting because of calculating BTC holdings. Then again, maybe I misunderstood wary's question.

About SAR rebound - we may have a chance to test it big-scale when/if the bubble come and burst.

About measuring in BTC - basically agree. What I mean is just a slight change to presentation of results,  so the reader won't have to calculate how his BTC holding have changed and won't have to translate "strategy made a profit, but performed worse than B&H" to "strategy made a loss".

[let_me_backtest_that]

re: SAR

I tried using the SAR myself, but find it alternatively too lagging and too "trigger happy" i.e. yielding too many, often unprofitable, signals. There's one positive observation I made however: since it is a momentum indicator with a "built in" price target, it can be useful for exiting a trade on a rebound, i.e. say I sell based on some indicator combination, buy back short term because I determine the "bottom is in", and then exit the trade = sell when price hits the SAR target. Usually relatively low time resolution, between 1h and 12h.

EDIT: A suggestion: since most of BTC traders trade to increase their BTC holdings, it would make sense to calculate profits and losses in BTC, rather than in dollars.

I thought the same at first, but then I came to the conclusion (please correct me if I'm wrong): a USD maximized holding and a BTC maximized holding are equivalent, as long as you just tack on one final 'buy' order to the end (i.e. calculate how many coins the final USD value buys you). Correct?

To see why: imagine you don't take a trade that is suggested by the tested system, and that turns out profitable. Your BTC value will be equal, but your USD to buy coins is higher at the next signal, so being USD optimal is equivalent to being BTC optimal. The only difference is that the final value of the holding is given in USD, so convert it to BTC according to current price.

@l_m_b_t Not entirely sure I understand your point about needing to include position sizing and curve fitting because of calculating BTC holdings. Then again, maybe I misunderstood wary's question.

About SAR rebound - we may have a chance to test it big-scale when/if the bubble come and burst.

About measuring in BTC - basically agree. What I mean is just a slight change to presentation of results,  so the reader won't have to calculate how his BTC holding have changed and won't have to translate "strategy made a profit, but performed worse than B&H" to "strategy made a loss".

Thanks for the suggestions.  It would be easiest to give a final calculation that shows what the net profit would be in BTC at the end of the test period.   In that way, no additional position sizing would be necessary.

As far as using the SAR as a profit target on a bounce from recent lows...that is an interesting idea.  I would have to find a way to determining those entry points systematically and then use the SAR as a (stable or dynamic) profit target.  This gives me something to work on and then share with those interested.  Thanks for the suggestions!

[BombaUcigasa]

https://i.imgur.com/0kLTgHV.png

Very nice buut... you gained 800% in 2 years? Buy and hold gives you that in less than a year. Maybe you can consider "tweaking" your backtest a bit more for more consistent results.

[bananaControl]

https://i.imgur.com/0kLTgHV.png

Very nice buut... you gained 800% in 2 years? Buy and hold gives you that in less than a year. Maybe you can consider "tweaking" your backtest a bit more for more consistent results.

So those who bought half a year ago should expect an 800% return before 2015? Yeah

[BombaUcigasa]

https://i.imgur.com/0kLTgHV.png

Very nice buut... you gained 800% in 2 years? Buy and hold gives you that in less than a year. Maybe you can consider "tweaking" your backtest a bit more for more consistent results.

So those who bought half a year ago should expect an 800% return before 2015? Yeah

Show me your time machine...

Please look at the data again, this is a backtest, not an estimation. Any backtest should be ran against the very simple and basic strategy of buy and hold which requires no TA, no bots, no effort.

[bananaControl]

Show me your time machine...

Please look at the data again, this is a backtest, not an estimation. Any backtest should be ran against the very simple and basic strategy of buy and hold which requires no TA, no bots, no effort.

Whoosh! ... Thanks for the explanation.

[Alisho]

DOES MACD BEAT BUY AND HOLD?

The MACD is a fairly straightforward momentum indicator.  It consists of the difference between a 12 and 26 bar EMA.  The difference between these two moving averages is the MACD.  It is smoothed by a 9 Day EMA that is called the "signal line."

The MACD  is usually plotted with a histogram that represents the difference between the MACD and the signal line.  The histogram can give either positive or negative values.  On the traditional interpretation, a buy signal occurs when the signal line crosses the MACD and a sell signal occurs when the signal line crosses below the MACD.   See the example below:

https://i.imgur.com/Llda1qf.png

The MACD is a popular indicator on BTC forums.  How does it hold up historically? Does it beat  buy and hold?  How does a long and short strategy perform?  We will perform two backtests.  The first will be a MACD long and short strategy.  The second will be a long only strategy.  Finally, we will compare this to buy and hold.

We will make the following assumptions: (1) The backtest will cover 1/'1/2012-6/1/2012.  (2) We will reinvest profits beginning with a $1,000 account.  (3) We will keep the default parameters for the MACD.

EDIT: The backtest is performed over 1/1/2012-6/1/2014

How does the long/short strategy perform?

The strategy entered 41 total trades - 59% were profitable- net profit was $9,820- there was a 98% a drawdown and a profit factor of 6.42.  See below

https://i.imgur.com/UcJD6aH.png

I am also attaching a youtube replay of each trade in the system to get a better feel for how the system would've worked.

https://www.youtube.com/watch?v=3mwCZOr-6fc&feature=youtu.be

You will notice that the system trades a different number of BTC depending on the account size.  Once it hit a large drawdown at the end of April 2013, then the system did not have the equity to trade larger quantities of BTC.

https://i.imgur.com/vwQTIqI.png

Now let's backtest the long only system.

The strategy entered a total of 21 trades - 76.19% were profitable - net profit was $135,000,  there was a 19% drawdown and a profit factor of 60.41.

https://i.imgur.com/ZFZLGI5.png

Here is the youtube replay.

https://www.youtube.com/watch?v=jE_eDDAe10w&feature=youtu.be

Ultimately, MACD outperforms a buy/hold strategy as seen below.  The Buy/Hold had a net profit of $126,600 and a drawdown of 73%.  (For Buy/Hold I entered a buy at the open of everyday with the same amount of BTC 202 on each trade to make the performance equivalent. )

https://i.imgur.com/kz70wD9.png

Here are a few takeaways:

1. Ultimately, MACD long only does beat buy and hold on both a net profit and drawdown.   It caught every major uptrend even though it lagged the market.

2. However, MACD long only underperformed buy and hold during the large price increases.  At one point, someone would've seen their BTC holdings drop from 202 to 58 due to price increases, lag and some losing trades.

3.  The drawdown  in USD on the MACD strategy is superior to buy/hold.  Returns should always be related to risk.  Someone trading the MACD would've had a $1,982 drawdown in USD, while the buy/hold drawdown equaled $159,000.

You can download updated BTC historical data formatted for NinjaTrader here:

http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

You can download the code for your own analysis on my blog;

http://www.signalstrengthfinance.com/bitcoinchartsmacd/


Trade Well and Prosper

Hi, thanks very much for this amazing work.. Could you share the code for the MACD indicator is this is possible.. I have not knowledge in programming.. I would really thankful. I would like to test it with other parameters.

[xcapator]

With this type of indicator you could reasonably predict when the next long term spike is about to occur. Using 10 to 1 leverage you could turn 10 BTC into 12 pretty easily. (minus the fees)