let_me_backtest_that (OP)
Newbie
Offline
Activity: 32
Merit: 0
|
|
June 23, 2014, 07:16:59 PM |
|
I will be posting a series of backtests for various indicators on BTC historical data. I posted on the Aroon last week, but thought I would begin a larger thread to see if anyone is interested in this research project. This time I am backtesting the ADX indicator The ADX is one of the more complicated indicators in a trader's arsenal. The reason why the ADX is complex is because it is a compilation of many indicators created by J. Welles Wilder in his book New Concepts in Technical Trading. The book is worth the read because his ideas have been so influential on previous and contemporary traders. (I also heard from a little bird that it is easy to find a pdf of the book from a basic google search.) I am going to give a basic overview of the indicator then backtest it on historical BTC data. The ADX is designed to indicate changes in momentum and the direction and strength of a trend. The indicator does this by first calculating the Directional Movement Index (DMI). The DMI generally consists of the difference between two different values the DM+(blue or green) and the DM-(red). The DM+ and DM- consist of the net difference between the most recent highs and lows. Changes in momentum are generally detected when the DM+ crosses the DM- and vice versa. Additionally, the direction of the trend is determined by whether or not the DM+ or DM- has a higher value. The creation of the ADX is based on the DMI and various smoothing techniques. This involves taking the difference between the absolute value of the DM+ and DM- and dividing it by the sum of the DM+ and DM-. The calculation is somewhat complex and can be found in more detail at stockcharts.com http://stockcharts.com/school/doku.php?id=chart_school:technical_indicators:average_directional_index_adxThe traditional interpretation of the indicator involves using the ADX line (green or blue) to indicate a trend. When the ADX is below 20, then the markets are moving sideways. If it is above 20, then the markets are trending. Trading signals are determined when there is a crossover of the DM+ and DM-. Here is an example: https://i.imgur.com/QC1vLGn.pngNow that we have the basics of the ADX down let's do some historical research. I am going to do two backtests using the default parameters for the ADX and a basic non-optimized position sizing strategy that trades approx $1,000 (or less) on each trade. The position sizing strategy helps to focus on the performance of the indicator itself. Since this is only research on an indicator and not a fully developed system, then our focus consists on how it works on various individual trades which I give below. I will create two separate strategies. The first will be to trade when the DM+ crosses above the DM- and sell with the reverse. Also, we will use the ADX line above 20 as a filter. Call this ADX Research. The second will merely use the DMI without the ADX filter (DMI Research). As usual, I will use NinjaTrader for the backtest and analysis. I will post images of the results and provide links for the code below. Here is a summary of the results of the ADX results: https://i.imgur.com/0kLTgHV.pngHere is a summary of each trade https://i.imgur.com/LwmtVkg.pngHere is short simulation of the strategy on Youtube to give you a picture of each trade. https://www.youtube.com/watch?v=OCWIXg-xxGg&feature=youtu.beHere is a summary of the DMI results https://i.imgur.com/Zkif51J.pngHere is a summary of each trade https://i.imgur.com/m3gH4sR.pngHere is a short simulation on Youtube to give you a picture of each trade. https://www.youtube.com/watch?v=9dR97Uq6Gww&feature=youtu.beHere are some things to notice: 1. The ADX system does succeed at filtering out many whipsaw trades when the market is flat. However, this increase in % profitable also comes with the cost of less total profit and a lower profit fact. 2. The total drawdown of each system is the same. Importantly, the ADX filter does not decrease drawdown risk. The ADX filter does not perform better on a return to risk basis. 3. One reason that the DMI indicator does well is because BTC prices trend and have low volatility when moving sideways. If these features were to change, then the DMI indicator would do less well. I am merely pointing out one of the many limitations that occur through backtesting. You can download the code for on my blog http://www.signalstrengthfinance.com/better-know-an-indicator-adx/You can also find a free daily updating link to upload historical BTC data into NinjaTrader here http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/Trade Well and Prosper
|
|
|
|
wobber
Legendary
Offline
Activity: 1064
Merit: 1001
|
|
June 23, 2014, 07:32:10 PM |
|
Absolutely brilliant! Waiting for more! Thanks!
|
If you hate me, you can spam me here: 19wdQNKjnATkgXvpzmSrkSYhJtuJWb8mKs
|
|
|
HeliKopterBen
|
|
June 23, 2014, 08:28:58 PM |
|
Nice! You should charge for your services.
|
Counterfeit: made in imitation of something else with intent to deceive: merriam-webster
|
|
|
let_me_backtest_that (OP)
Newbie
Offline
Activity: 32
Merit: 0
|
|
June 30, 2014, 04:47:48 PM Last edit: June 30, 2014, 10:48:15 PM by let_me_backtest_that |
|
DOES MACD BEAT BUY AND HOLD? The MACD is a fairly straightforward momentum indicator. It consists of the difference between a 12 and 26 bar EMA. The difference between these two moving averages is the MACD. It is smoothed by a 9 Day EMA that is called the "signal line." The MACD is usually plotted with a histogram that represents the difference between the MACD and the signal line. The histogram can give either positive or negative values. On the traditional interpretation, a buy signal occurs when the signal line crosses the MACD and a sell signal occurs when the signal line crosses below the MACD. See the example below: https://i.imgur.com/Llda1qf.pngThe MACD is a popular indicator on BTC forums. How does it hold up historically? Does it beat buy and hold? How does a long and short strategy perform? We will perform two backtests. The first will be a MACD long and short strategy. The second will be a long only strategy. Finally, we will compare this to buy and hold. We will make the following assumptions: (1) The backtest will cover 1/'1/2012-6/1/2012. (2) We will reinvest profits beginning with a $1,000 account. (3) We will keep the default parameters for the MACD. EDIT: The backtest is performed over 1/1/2012-6/1/2014 How does the long/short strategy perform? The strategy entered 41 total trades - 59% were profitable- net profit was $9,820- there was a 98% a drawdown and a profit factor of 6.42. See below https://i.imgur.com/UcJD6aH.pngI am also attaching a youtube replay of each trade in the system to get a better feel for how the system would've worked. https://www.youtube.com/watch?v=3mwCZOr-6fc&feature=youtu.beYou will notice that the system trades a different number of BTC depending on the account size. Once it hit a large drawdown at the end of April 2013, then the system did not have the equity to trade larger quantities of BTC. https://i.imgur.com/vwQTIqI.pngNow let's backtest the long only system. The strategy entered a total of 21 trades - 76.19% were profitable - net profit was $135,000, there was a 19% drawdown and a profit factor of 60.41. https://i.imgur.com/ZFZLGI5.pngHere is the youtube replay. https://www.youtube.com/watch?v=jE_eDDAe10w&feature=youtu.beUltimately, MACD outperforms a buy/hold strategy as seen below. The Buy/Hold had a net profit of $126,600 and a drawdown of 73%. (For Buy/Hold I entered a buy at the open of everyday with the same amount of BTC 202 on each trade to make the performance equivalent. ) https://i.imgur.com/kz70wD9.pngHere are a few takeaways: 1. Ultimately, MACD long only does beat buy and hold on both a net profit and drawdown. It caught every major uptrend even though it lagged the market. 2. However, MACD long only underperformed buy and hold during the large price increases. At one point, someone would've seen their BTC holdings drop from 202 to 58 due to price increases, lag and some losing trades. 3. The drawdown in USD on the MACD strategy is superior to buy/hold. Returns should always be related to risk. Someone trading the MACD would've had a $1,982 drawdown in USD, while the buy/hold drawdown equaled $159,000. You can download updated BTC historical data formatted for NinjaTrader here: http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/You can download the code for your own analysis on my blog; http://www.signalstrengthfinance.com/bitcoinchartsmacd/Trade Well and Prosper
|
|
|
|
oda.krell
Legendary
Offline
Activity: 1470
Merit: 1007
|
|
June 30, 2014, 04:57:22 PM |
|
Very interesting, as before. Thanks!
"long/short" MACD strategy means on a buy signal, the system buys in with the entire account value, and on a sell signal, it sells and actively shorts? While "long" only means the system only buys and sells, without entering a margin position?
|
Not sure which Bitcoin wallet you should use? Get Electrum!Electrum is an open-source lightweight client: fast, user friendly, and 100% secure. Download the source or executables for Windows/OSX/Linux/Android from, and only from, the official Electrum homepage.
|
|
|
let_me_backtest_that (OP)
Newbie
Offline
Activity: 32
Merit: 0
|
|
June 30, 2014, 05:04:45 PM |
|
Very interesting, as before. Thanks!
"long/short" MACD strategy means on a buy signal, the system buys in with the entire account value, and on a sell signal, it sells and actively shorts? While "long" only means the system only buys and sells, without entering a margin position?
Yes, thanks for clarifying that. I think its important to show how a going long and actively selling short with one's full account value can have some pretty horrendous consequences. EDIT: I had the order backwards in the post. It now makes more sense.
|
|
|
|
oda.krell
Legendary
Offline
Activity: 1470
Merit: 1007
|
|
June 30, 2014, 05:18:31 PM |
|
I keep being surprised by how well daily MACD still seems to work for this market, even after it being mentioned and discussed over and over again. In fact, I suspect during the 2013/2014 bear market it would have performed even better than in your 2012 data backtest.
|
Not sure which Bitcoin wallet you should use? Get Electrum!Electrum is an open-source lightweight client: fast, user friendly, and 100% secure. Download the source or executables for Windows/OSX/Linux/Android from, and only from, the official Electrum homepage.
|
|
|
fallinglantern
|
|
June 30, 2014, 05:20:52 PM |
|
Interesting to read that trading MACD beats buy hold. Does that factor in the tax savings (long term capital gains vs. short term capital gains) inherent in the buy+hold strategy that MACD trading doesn't have?
|
|
|
|
BitchicksHusband
|
|
June 30, 2014, 05:23:21 PM |
|
Outperformed "buy and hold DAILY". That's not a valid measure. Nobody buys and holds daily.
Also: "The backtest will cover 1/'1/2012-6/1/2012", the cherry-picked deadest period possible for buy and hold.
|
1BitcHiCK1iRa6YVY6qDqC6M594RBYLNPo
|
|
|
oda.krell
Legendary
Offline
Activity: 1470
Merit: 1007
|
|
June 30, 2014, 05:29:50 PM |
|
Outperformed "buy and hold DAILY". That's not a valid measure. Nobody buys and holds daily.
AFAIK, he doesn't include slippage and fees in his calculations, so assuming no gaps between daily candles, I believe selling at candle close and buying at candle open is just his way to simulate a buy&hold strategy. It is (for the reasons mentioned above) equivalent to "real" b&h.
|
Not sure which Bitcoin wallet you should use? Get Electrum!Electrum is an open-source lightweight client: fast, user friendly, and 100% secure. Download the source or executables for Windows/OSX/Linux/Android from, and only from, the official Electrum homepage.
|
|
|
let_me_backtest_that (OP)
Newbie
Offline
Activity: 32
Merit: 0
|
|
June 30, 2014, 07:59:32 PM |
|
Outperformed "buy and hold DAILY". That's not a valid measure. Nobody buys and holds daily.
AFAIK, he doesn't include slippage and fees in his calculations, so assuming no gaps between daily candles, I believe selling at candle close and buying at candle open is just his way to simulate a buy&hold strategy. It is (for the reasons mentioned above) equivalent to "real" b&h. Yes, that's right. I am buy on the open and selling on the close without slippage and commissions to simulate both the gains and losses of buy/hold. Also, I don't see how the data is cherry picked, not intentionally anyway, I am using the same data set as the other indicator backtests. The bitstamp data is pretty shoddy before 1/1/2012. In fact, starting at the beginning of the most recent move to 1,200USD and decline would've given the best results because the MACD has had a very good record of shorting during the recent bear market.
|
|
|
|
let_me_backtest_that (OP)
Newbie
Offline
Activity: 32
Merit: 0
|
|
June 30, 2014, 08:01:41 PM |
|
Interesting to read that trading MACD beats buy hold. Does that factor in the tax savings (long term capital gains vs. short term capital gains) inherent in the buy+hold strategy that MACD trading doesn't have?
Good point. I am not factoring in slippage, commissions or taxes which should be thoroughly taken into account before using this as a stand alone strategy. I want to emphasize that this thread is just for testing indicators not fully developed strategies.
|
|
|
|
BitchicksHusband
|
|
June 30, 2014, 11:14:10 PM |
|
Outperformed "buy and hold DAILY". That's not a valid measure. Nobody buys and holds daily.
AFAIK, he doesn't include slippage and fees in his calculations, so assuming no gaps between daily candles, I believe selling at candle close and buying at candle open is just his way to simulate a buy&hold strategy. It is (for the reasons mentioned above) equivalent to "real" b&h. Yes, that's right. I am buy on the open and selling on the close without slippage and commissions to simulate both the gains and losses of buy/hold. Also, I don't see how the data is cherry picked, not intentionally anyway, I am using the same data set as the other indicator backtests. The bitstamp data is pretty shoddy before 1/1/2012. In fact, starting at the beginning of the most recent move to 1,200USD and decline would've given the best results because the MACD has had a very good record of shorting during the recent bear market. Change it to first six months of 2013 and see if it still beats it.
|
1BitcHiCK1iRa6YVY6qDqC6M594RBYLNPo
|
|
|
oda.krell
Legendary
Offline
Activity: 1470
Merit: 1007
|
|
June 30, 2014, 11:30:26 PM Last edit: June 30, 2014, 11:46:59 PM by oda.krell |
|
@BitchicksHusband Let it go, man. Just by eyeballing MACD, I can already tell that during the first 6 months of 2013, b&h would perform better. Then skip forward to 2013/2014, and MACD signal is better again.
What matters are overall profits, and this entire thread is basically an invitation to backtest your own ideas. Instead, you come in here, with your "only b&h makes sense" attitude, and take cheap shots. Why not _download_ the motherf'ing program and test it for yourself?
EDIT: that said, any reason why you don't run the test over the entire trading history, let_me_backtest_that?
EDIT 2: sounded more aggressive than intended. apologies.
|
Not sure which Bitcoin wallet you should use? Get Electrum!Electrum is an open-source lightweight client: fast, user friendly, and 100% secure. Download the source or executables for Windows/OSX/Linux/Android from, and only from, the official Electrum homepage.
|
|
|
let_me_backtest_that (OP)
Newbie
Offline
Activity: 32
Merit: 0
|
|
July 01, 2014, 12:38:32 AM |
|
@BitchicksHusband Let it go, man. Just by eyeballing MACD, I can already tell that during the first 6 months of 2013, b&h would perform better. Then skip forward to 2013/2014, and MACD signal is better again.
What matters are overall profits, and this entire thread is basically an invitation to backtest your own ideas. Instead, you come in here, with your "only b&h makes sense" attitude, and take cheap shots. Why not _download_ the motherf'ing program and test it for yourself?
EDIT: that said, any reason why you don't run the test over the entire trading history, let_me_backtest_that?
EDIT 2: sounded more aggressive than intended. apologies.
Thanks oda.krell- You are right, this post is a an invitation to test people's own ideas with a some data and code to get started with. I am responsible for some of the misunderstanding because the backtest length was a typo. I am including the MACD backtest info up till 6-29-14. People can see that it is similar to the other results. https://i.imgur.com/bFcicsE.png
|
|
|
|
Wary
|
|
July 02, 2014, 05:34:41 AM |
|
Thanks for the tests! I've looked at the MACD video. The indicator seems to be good for quiet periods, but not for the rally time. At April's rally it 1 Completely missed the rally's top 2 Sold exactly at the following bottom 3 Bought exactly at the following top At October's rally it was planning to replicated April's disaster , but was saved by an accident: it was going to sell at the bottom again, but since the rally had two overlapping peaks, it sold around the second peak. Ergo: 1d MACD is good at quiet periods, but at rally time, because of the lag, it's signals becoming worse than useless (namely harmful). (Sorry for harsh words. My anger is directed not to participants of this discussion, but to the indicator itself. I guess it wouldn't mind ) EDIT: grammar.
|
Fairplay medal of dnaleor's trading simulator.
|
|
|
let_me_backtest_that (OP)
Newbie
Offline
Activity: 32
Merit: 0
|
|
July 02, 2014, 02:06:42 PM |
|
Thanks for the tests! I've looked at the MACD video. The indicator seems to be good for quiet periods, but not for the rally time. At April's rally it 1 Completely missed the rally's top 2 Sold exactly at the following bottom 3 Bought exactly at the following top At October's rally it was planning to replicated April's disaster , but was saved by an accident: it was going to sell at the bottom again, but since the rally had two overlapping peaks, it sold around the second peak. Ergo: 1d MACD is good at quiet periods, but at rally time, because of the lag, it's signals becoming worse than useless (namely harmful). (Sorry for harsh words. My anger is directed not to participants of this discussion, but to the indicator itself. I guess it wouldn't mind ) EDIT: grammar. My purpose here is to give the backtested information so people can determine the usefulness of the indicator for themselves. One reason I provide the videos is so that people can see the exact issues that you point out. I will say this about the MACD, as with any lagging momentum indicator, it will not catch the exact top. Also, any price that decreases precipitously in one or two bars will give back a lot of gains. What I am less comfortable with is the following inference: If the MACD lags (and thereby doesn't sell at the top of a rally), then it is only useful in non-trending periods. Also, I want to point out that my purpose is not to give full blown strategy. I think a person could find various exit parameters alongside the MACD indicator that would be useful. Just something to think about.
|
|
|
|
Wary
|
|
July 02, 2014, 08:27:05 PM |
|
Thanks for the tests! I've looked at the MACD video. The indicator seems to be good for quiet periods, but not for the rally time. At April's rally it 1 Completely missed the rally's top 2 Sold exactly at the following bottom 3 Bought exactly at the following top At October's rally it was planning to replicated April's disaster , but was saved by an accident: it was going to sell at the bottom again, but since the rally had two overlapping peaks, it sold around the second peak. Ergo: 1d MACD is good at quiet periods, but at rally time, because of the lag, it's signals becoming worse than useless (namely harmful). (Sorry for harsh words. My anger is directed not to participants of this discussion, but to the indicator itself. I guess it wouldn't mind ) EDIT: grammar. My purpose here is to give the back-tested information so people can determine the usefulness of the indicator for themselves. One reason I provide the videos is so that people can see the exact issues that you point out. I will say this about the MACD, as with any lagging momentum indicator, it will not catch the exact top. Also, any price that decreases precipitously in one or two bars will give back a lot of gains. What I am less comfortable with is the following inference: If the MACD lags (and thereby doesn't sell at the top of a rally), then it is only useful in non-trending periods. Also, I want to point out that my purpose is not to give full blown strategy. I think a person could find various exit parameters alongside the MACD indicator that would be useful. Just something to think about. Your back-tests gave us very useful information and it is much appreciated. Especially appreciated is the videos, which gives much more clear picture for table-challenged persons like myself I realize that no single indicator can give perfect signals. I also realize that catching the exact tops&bottoms is Holy Grail that can be searched for but hardly can be found. What I'm trying to figure out about 1d MACD is when it should be used and when avoided. It seems to be it is more useful for quiet time, i.e. not during the parabolic rise or immediately after. It is too lagging for it. The quiet time can be trending, just things should be moving slow enough for the indicator to catch up. Especially good it is for "buy" signals. For violent time and "sell" signals it should be used more cautiously, with more confirmations or not at all. Does it make any sense?
|
Fairplay medal of dnaleor's trading simulator.
|
|
|
BitchicksHusband
|
|
July 02, 2014, 08:58:03 PM |
|
First off, Oda, he said it beats buy and hold and I intend to make him prove it more than his simple statements. He brought it up, he should have expected criticism.
My point: Buy and Hold is easier to perform perfectly than ADX. So I have the following questions:
How do you perform all these trades?
How do you perform all these trades perfectly?
Do you ever sleep?
Do you program a bot to do them for you?
Do you leave your money on an exchange such as Mt. Gox? If so, your carefully traded stash just went to zero. If you say Bitstamp, how do you know Bitstamp isn't the next Mt. Gox?
All of these questions must be answered adequately in order to beat buy and hold in a cold storage wallet, which takes zero effort and gives 93% of the same result (by the numbers given).
Mistakes can easily lose you that 7%.
|
1BitcHiCK1iRa6YVY6qDqC6M594RBYLNPo
|
|
|
oda.krell
Legendary
Offline
Activity: 1470
Merit: 1007
|
|
July 02, 2014, 09:17:39 PM |
|
@BitchicksHusband
Okay, I'll jump to let_me_backtest_that's defense one more time... I'm starting to feel I'm whiteknighting here :D
Anyway. Your points are all absolutely valid questions.
However, let_me_backtest_that said clearly, that he doesn't provide a fully developed strategy. Call it "a hypothetical strategy based on a single signal", if you want.
This is not just a semantic difference. Understanding the difference between a single indicators's signal and a strategy is the difference between making profitable trades and incurring losses in the long run. I learned that lesson the last year.
The backtest is just the evaluation of a single indicator signal, and it potentially invites the interested trader to continue investigating it. Since the signal all by itself slightly outperforms b&h (in this very theoretical, non-practical setting), it is a somewhat interesting invitation.
|
Not sure which Bitcoin wallet you should use? Get Electrum!Electrum is an open-source lightweight client: fast, user friendly, and 100% secure. Download the source or executables for Windows/OSX/Linux/Android from, and only from, the official Electrum homepage.
|
|
|
|