Goomboo's Journal

[Goomboo]

Some things we've discussed:

Here's a page of stats on the moving average crossover strategy.  We've covered a few aspects of this in a previous post.  The thing I want to draw your attention to is the max consecutive losers row.  Can you handle trading a strategy in which you lose up to 7 (or more) trades in a row?  Now look at the average loss row.  On average, you lose 1.58% (in the normal currency market) on a losing trade.  Multiply that by 7 - do you have the discipline to sit there and trade again when you are down 11% and you have lost 7 trades in a row?  This is in the "normal" currency market...a move of 2% in the currency market in a given day is a _massive_ move in most pairs.

Can you see how BTC is a very difficult market to trade, emotionally and financially?

One last thing before I sign out,

This thread is directed at individuals who are trying to trade bitcoin for a profit (myself included) - not the long term investors / developers / enthusiasts.  My hope is twofold - 1.  Give a reality check to people risking their money and 2.  Give you a way of actively, impartially, and rationally approaching the markets.  It's definitely not the "Holy Grail", but it's a system and if you stick with it, you are miles ahead of most people on the other side of your trades.  You still have a high probability of losing it all - commissions + spread + volatility are really, really rough in this market, let alone the fact that the market can't be explained by technical analysis as well as other, more liquid markets.

Sure thing...day trading BTC can really be brutal with the 1-2% spread + 1% round-trip commission.  Basically this requires price to move 3% for you to break even on a trade.  Trade 1-2 times a day and if the market doesn't go in your favor, your account will be down 20-30% in a week simply from the costs of transacting (assuming no added slippage).

[Seal]

Especially at the moment with an indecisive market and low volumes.

Does anyone have any theories on why this is happening?

Everybody has run out of secret rockets and the secret submersibles have all sunk. We have to wait for more to be manufactured.

[Goomboo]

so many people on here have been using the EMA crossover and yet the guru already changed his strategy.  Next time before trading, people should do their own backtesting and not choose a trading strategy based on a few trades of an anonymous person.

I'll help them out and show them the difference as well as explain my reasoning for trading the SMA.

Here's a summary of the data below.  This simple comparison shows the difference in the two strategies.  As you can see, the EMA is almost superior in every way when compared to the SMA.



Not for me, however:

- At this stage of life, I only have the time to trade the daily timeframe in BTC.  
- However, I want to frequently trade.  
- Additionally, I prefer to only trade strategies in which I can do the math of the indicators I am using quickly and easily.
- After all, if you don't know what goes into the colorful things on the screen, what makes you think that you will trust where they are pointing?

Which of the two strategies matches me best?


The EMA and SMA tell the same thing - which way the immediate trend is

- The EMA keeps you in a trade longer, which means that you will have a lower success ratio but a higher payout on trades that really move.

- The SMA gets you in a move earlier, which means that if price doesn't take off, you will more than likely have a profit rather than a loss.


Here is the SMA 10/21 crossover on hourly timeframes:

[Goomboo]

so many people on here have been using the EMA crossover and yet the guru already changed his strategy.  Next time before trading, people should do their own backtesting and not choose a trading strategy based on a few trades of an anonymous person.

I will say one more thing about this.  I absolutely agree with your conclusion.  I stand by the advice I gave one month ago.  Nowhere in this advice do I suggest anything but owning up to your financial decisions.  

If you have been blindly following the system without going through the below steps, shame on you, you missed the whole point of this thread and you are robbing yourself of a chance at successful trading.  Furthermore, you never learned the most important lessons of trading - consistency, confidence, discipline.  Why are you trusting some random person on the internet with your finances?  If you really cared about consistent profit and winning in the financial markets, you would put in the work necessary to trust your edge.  Stop outsourcing your decision making and take responsibility for the money that you worked hard to earn.

I reiterate; what is more important: making a few hundred dollars trading a system you learned on the internet or developing the discipline to be a wise steward of whatever money is entrusted to you?

Here is my tangible advice on how you can learn to be a trader:

1.  Develop a system
--Learn about technical analysis until you have a conceptual framework of when to participate in market action
--When will you enter the markets?
--How much money will you risk per trade?  Is this scalable and dynamic?
--When will you exit the markets?  Both profitable exit and unprofitable exit.
--What hours of the day will you trade?

2.  Backtest the system
--You must acquire or create some method of exploring if your system worked in the past.
--My suggestion is a demo feed connected to NinjaTrader and MB Trading.  When you open a chart in NinjaTrader, you can click the arrow keys to move a candle onto and off the chart allowing you to see the price action as though it is happening live.
--Drag a chart of historical prices one candle at a time, recording your trades as though you actually were taking them in a spreadsheet.  Chart your profit and loss.  Analyze, scrutinize, and try to disprove your edge.
--I suggest you do this for 200-400 trades across as much data as you can get your hands on.

3.  Forward test the system
--Once you have confidence that your strategy worked in the past, you must ensure that it works in the future.
--Open a demo account and trade fake money for at least two months to prove that your edge still exists

4. Live trade the system
--Start with tiny amounts of money and adhere to your system perfectly.  The amount of capital at risk should be so small that if you lost it all, you wouldn't be financially disadvantaged.
--If you can continue to make profit for at least 30-50 more trades, you should be able to gradually add more and more capital to your system.


Of course this isn't easy and more than likely the majority of people won't do it.  My purpose in saying this is just to let you know that this is what it takes if you wish to have a sustainable edge in the market.

[Otoh]

Romans $5.8-$10

FTFY 

[Goomboo]

There are two questions that have been asked or alluded to that I will end the night by answering:

1.  What happens if the market goes sideways and I'm getting chopped up?

2.  What happens if we have many people trading our strategy?

1.  Let's answer the first one.  What happens if the market is going sideways and we are getting chopped up?  Simple answer: we stick with the system.  All of our testing assumed that we stuck with our system perfectly.  However, we must periodically re-evaluate our system and see if the parameters we chose initially are holding up.  After all, we chose our parameters based upon an optimization on the daily timeframe and we are trading the hourly.  Below is a chart of this optimization.  I ask every individual who has been learning from this thread and is serious about learning to trade and analyze markets to not move forward from this post until you have observed and understood the below chart.  I have put a border on the 10 / 21 crossover, showing you what we are currently trading.

Look first at the number of trades - as some individuals are noting - through commissions, the spread, and slippage - the more trades, the more money spent!  Therefore, we really should try and minimize the number of trades that we do in order to maximize our bottom line.  As we can see from the below chart, the longer the period of our moving averages, the fewer times we trade.  This hopefully should be intuitive by now.

We have to balance our desire to trade infrequently with what the market says is the most profitable area to trade in.  If you notice, they do not perfectly overlap.  Finding the best place to be - where we aren't chopped up but at the same time capture trends - does not have a cut-and-dry solution.  I can point you to a region, but I can't and won't point you to the answer.  You have to look at the charts below and find the right trade-off for you.  Essentially, you have to find what works for you and stick with it.

2.  What happens if we have too many people trading our strategy?  Look at the answer to the first question and think it through. We're not even trading the best strategy.  The serious individuals will be departing from "10 / 21" and finding a happy medium soon, leading to hopefully dozens of variants of the strategy, each of which is based upon the simple concept of following the trend.

[realnowhereman]

Fascinating charts.  (excuse me if I use the wrong technical words in the following, my background is signal processing, not finance).

Quick questions: is that table showing results for daily or hourly quantised price data?  Does your profit factor take account of exchange fees?  If fees were (say) 20%, then obviously the better optimisation is to go for lower numbers of trades; if fees were 0%, then obviously we would want to optimise profit.

Longer questions/observations:

It seems that we're only seeing part of the picture in these charts, the green area (especially in the EMA table) is clustered around the bottom right.  The implication then is that there is more to see of this pattern.  For example, is this a long thin island horizontally (i.e. it is the fast parameter that is sensitive)?  Is it long and thing vertically (i.e. it is the slow parameter that is sensitive)?  Similarly, are both insensitive?

How hard would it be to continue the plot to higher slow/fast parameter values?

Presumably we want to be aiming at the centre of a green pool, since that would only be an inferred-from-the-past selection of parameters we would want to choose such that large variations in parameter cause small variations in profit. i.e. if we're wrong in selection, we are still likely to end up in a profitable region.

[LD-ZOGY8]

Goomboo, can I ask what software you are using? I would like to try and run some figures of my own.

[Goomboo]

Quick questions: is that table showing results for daily or hourly quantised price data?  Does your profit factor take account of exchange fees?  If fees were (say) 20%, then obviously the better optimisation is to go for lower numbers of trades; if fees were 0%, then obviously we would want to optimise profit.

The table is showing results for hourly data.  Basically NinjaTrader did 600 backtests, spit the results out to me, and I put them in a colored table showing their relative value compared to the others.

Profit factor = gross gains / gross losses.  Any number above 1 means that the system was profitable while anything below means that it lost money.  It is based purely on price change in this backtest and commissions are not factored in.  You are absolutely right - if commissions are low, the higher frequency strategies can be profitably applied to the markets.

Longer questions/observations:

It seems that we're only seeing part of the picture in these charts, the green area (especially in the EMA table) is clustered around the bottom right.  The implication then is that there is more to see of this pattern.  For example, is this a long thin island horizontally (i.e. it is the fast parameter that is sensitive)?  Is it long and thing vertically (i.e. it is the slow parameter that is sensitive)?  Similarly, are both insensitive?

I'll show you that pattern for more data and then tell you why I didn't show this much data - the colors are changed since it's conditional formatting, but the areas are still differentiated.  The first set of charts I showed you were for 600 moving average combinations - here is is a chart of 10,000 unique combinations of both the EMA and SMA crossovers.

SMA Profit Factor
SMA Trades
EMA Profit Factor
EMA Trades

The reason I didn't include this much is because of a concept called curve-fitting.  Basically if you let the optimizer have its way, it will find the moving average combination that will buy at the bottom and sell at the top in the past perfectly, but will reduce your effectiveness going forward because the predictive power is minimal.  There are a few methods of determining if you are curve fitting, one of which is the number of trades.  If you have 30 trades or less, you more than likely have curve-fit your data or have too little information to make a good decision.

The second reason I didn't show this much data is the nature of what we're after here - the purpose of this is to teach short-term trend following trading systems.  In my opinion, any indicator which uses more than 50 periods is intermediate to long-term.  If you're trading in this region, you need to ask yourself why you aren't looking at a higher timeframe.

For example, is this a long thin island horizontally (i.e. it is the fast parameter that is sensitive)?  Is it long and thing vertically (i.e. it is the slow parameter that is sensitive)?  Similarly, are both insensitive?

Since we're dealing with a relatively few amount of trades (less than 500), I'd be wary to attach too much significance to contours in the data.  The reason these contours appear is because that region of specific averages captures an explosive move very well (curve fitting).  What we're after is a clump of above-average performance with a range of averages that "makes sense".  This increases our probability of staying alive when the markets change (which they constantly do).

Presumably we want to be aiming at the centre of a green pool, since that would only be an inferred-from-the-past selection of parameters we would want to choose such that large variations in parameter cause small variations in profit. i.e. if we're wrong in selection, we are still likely to end up in a profitable region.

Boom!  Exactly...markets always are changing.  What is best today won't be the best tomorrow.  We're trying to land in the region of above average profitability so that maybe we can survive the future.

[Goomboo]

Goomboo, can I ask what software you are using? I would like to try and run some figures of my own.

I use:
-Ninjatrader - backtesting / optimizing
-Excel - taking Ninjatrader's output and applying formatting to "pretty" it up


Steps I followed:
1.  Download Ninjatrader and learn how to import data and create custom instruments
2.  Download the data from www.bitcoincharts.com and modify it in Excel such that it can imported (they have some mistakes where opens are greater than highs, etc.)
3.  Import the data into Ninjatrader
4.  Create a strategy
5.  Backtest and optimize the strategy
6.  Export the data to excel
7.  Put it in a pivot table, color the range, format it
8.  Put it in a picture and upload it

For anyone who is interested in stock, futures, forex, or commodities trading, I highly recommend Ninjatrader - it is a solid, solid platform.

[notme]

Just a point of clarification... curve-fitting it the process regardless of success.  The term you are looking for is overfitting.

http://en.wikipedia.org/wiki/Overfitting

[Goomboo]

Just a point of clarification... curve-fitting it the process regardless of success.  The term you are looking for is overfitting.

http://en.wikipedia.org/wiki/Overfitting

In the finance and trading world, we use the term curve-fitting to describe the process of making your trading strategy fit the price curve.  But yes, that's what it is - overfitting.

Edit - what I mean to say is that we literally say "curve-fitting" when describing our optimization and write "curve-fitting" in textbooks about trading systems.  Interesting that the technical term doesn't actually "fit" our usage of it :p ... thanks for pointing that out.

[notme]

Just a point of clarification... curve-fitting it the process regardless of success.  The term you are looking for is overfitting.

http://en.wikipedia.org/wiki/Overfitting

In the finance and trading world, we use the term curve-fitting to describe the process of making your trading strategy fit the price curve.  But yes, that's what it is - overfitting.

Edit - what I mean to say is that we literally say "curve-fitting" when describing our optimization and write "curve-fitting" in textbooks about trading systems.  Interesting that the technical term doesn't actually "fit" our usage of it :p ... thanks for pointing that out.

NP... pointing out nearly irrelevant discrepancies is what I do.  That, and inflaming trolls .

[Timbo925]

So did you guys catch the drop. I personally was sleeping 

@ goomboo. I was just wondering if you could do a fast backtrack of your system (so the 21/10 crossover) but with a 1 hour delay after every crossover.
This because sometimes the crossover happens just before a huge sell/buy but most of the time you will miss it when your not glued to your computer.

[adamstgBit]

most of the time you will miss it when your not glued to your computer.

this might help

http://www.bitcoin-tools.de/

[Timbo925]

most of the time you will miss it when your not glued to your computer.

this might help

http://www.bitcoin-tools.de/

Thx for the link.
Here just a pictures of my resent trades. I now this strategy is expected to make a loss with most of your trades and if you make a profit it is a big one.
Still have the feeling i'm bleeding a little bit much lately..

Ledgend:
Green = Profit
Red = Loss
Yellow = intrest from bitcoinica

[piotr_n]

most of the time you will miss it when your not glued to your computer.

this might help

http://www.bitcoin-tools.de/

Why bother waking up?
It's actually quite easy to create an own bot. Just don't run it on a Linode VPS - better use your own PC at home

1) Every hour fetch the last 100 or so candles from the bitcoincharts: http://bitcoincharts.com/charts/chart.json?m=mtgoxUSD&i=H1&c=1&s=2012-02-20
2) Calculate EMA(long) and EMA(short) from the recent data: http://www.iexplain.org/ema-how-to-calculate/
3) If the lines have just crossed: execute the mtgox or bitcoinica API, according to the changing trend.
As simple as that.

You can basically do it in one relatively simple script; use php, python, bash, javascript, anything that suits you... I personally prefer Go - it's a great language.
Then run the 3 steps every hour (crontab, windows scheduler or whetever) - and you have no need to wake up in a middle of a night. Unless only to check if your bot is doing well...

[Otoh]

most of the time you will miss it when your not glued to your computer.

this might help

http://www.bitcoin-tools.de/

Why bother waking up?
It's actually quite easy to create an own bot. Just don't run it on a Linode VPS - better use your own PC at home

1) Every hour fetch the last 100 or so candles from the bitcoincharts: http://bitcoincharts.com/charts/chart.json?m=mtgoxUSD&i=H1&c=1&s=2012-02-20
2) Calculate EMA(long) and EMA(short) from the recent data: http://www.iexplain.org/ema-how-to-calculate/
3) If the lines have just crossed: execute the mtgox or bitcoinica API, according to the changing trend.
As simple as that.

You can basically do it in one relatively simple script; use php, python, bash, javascript, anything that suits you... I personally prefer Go - it's a great language.
Then run the 3 steps every hour (crontab, windows scheduler or whetever) - and you have no need to wake up in a middle of a night. Unless only to check if your bot is doing well...

I expect that if you could make this available for non programers then there would be some paying interest demand for it, I'd be interested if it was idiot proof or nearly & not needing a PHD in geekism to set up with different parameters

[piotr_n]

I expect that if you could make this available for non programers then there would be some paying interest demand for it, I'd be interested if it was idiot proof or nearly & not needing a PHD in geekism to set up with different parameters

I could have, but you should not be trusting strangers in internet to take care of such stuff.
It's a piece of software, which you run on your PC, that deals with your money - you don't know me, so you shouldn't be taking it from me if you cannot understand what is inside.
Otherwise someone will screw you over one day - and I'm not being cynical here, it's just life that we all need to deal with

What I can do to help you..
You tell me what do you actually need and I can post some simple scripts here, so others could verify if I'm not screwing you over with them.
And then you can run them on your pc, if you want. I can eventually tell you what you need to install first - a software from some trusted source, like google, oracle or microsoft

But basically, as when it comes to bitcoincharts+bitcoinica, I believe all you need is a continuously running web browser - you can afford it, cant you?
For the new mtgox api, which i think is much better, they have a crypto with some weird hmac stuff - I dont know how easy it would be to do it in javascript. Probably php would be best here, since their example actually works - and there is not much more one needs to add over that.

[conspirosphere.tk]

I expect that if you could make this available for non programers then there would be some paying interest demand for it, I'd be interested if it was idiot proof or nearly & not needing a PHD in geekism to set up with different parameters

I could have, but you should not be trusting strangers in internet to take care of such stuff.

If the 2 crossing EMAs tech really works more often than not (anyone tested it? results?) I would let a bot run by some forum member with rep to play with a few of my BTC. A good bot would even be more than enough for a speculative fund quoted on the GLBSE.