Trading Bots

[ohforf]

I think you're better off finding the right algorithm first, backtesting/bootstrapping, and then learning about the MtGox API. At least then you won't be tempted to turn it on at the first evolved GA you get that seems to work.

BTW, if you use RGP to create the algorithm you get a fair degree of flexibility since it allows boolean operations. So you end up with "if then" type conditions. It's reasonably quick to plateau as well.

I generated a good GA that was basically a moving average crossover which did well in backtesting and has done well so far. It only trades once every two or three weeks though.

Let me look at that Document... function call to "geneticProgramming"
Parameters:

Code:

geneticProgramming(fitnessFunction,stopCondition=makeTimeStopCondition(5),population,
populationSize=100,eliteSize=ceiling(0.1*populationSize),
elite=list(),functionSet=mathFunctionSet,
inputVariables=inputVariableSet("x"),
constantSet=numericConstantSet,
selectionFunction=makeTournamentSelection(),
crossoverFunction=crossover,mutationFunction,
restartCondition=makeEmptyRestartCondition(),
restartStrategy=makeLocalRestartStrategy(),
breedingFitness=function(individual)TRUE,breedingTries=50,
extinctionPrevention=FALSE,archive=FALSE,genealogy=FALSE,
progressMonitor,verbose=TRUE)
typedGeneticProgramming(fitnessFunction,type,stopCondition=makeTimeStopCondition(5),
population,populationSize=100,eliteSize=ceiling(0.1*
populationSize),elite=list(),functionSet,inputVariables,
constantSet,selectionFunction=makeTournamentSelection(),
crossoverFunction=crossoverTyped,mutationFunction,
restartCondition=makeEmptyRestartCondition(),
restartStrategy=makeLocalRestartStrategy(populationType=type),
breedingFitness=function(individual)TRUE,breedingTries=50,
extinctionPrevention=FALSE,archive=FALSE,genealogy=FALSE,
progressMonitor,verbose=TRUE)

WTF ?  
I have never seen a Function call like this.

[1714796375]

1714796375

[organofcorti]

You have a list of all possibilities there. Only a few are needed. Have you never used R before?

You'd have to be familiar with R and then learn to use the RGP package. The last bit is not a steep learning curve.

[Kettenmonster]

WTF ?  
I have never seen a Function call like this.

I consider those function calls broken.

Nevertheless could be it works. Still it then might be a good idea to look a bit deeper into the idea of GA.
What are your Populations? Can you reproduce trades? Will those be as good again as they were before?
What is your fitness function? Is the function meant to change or just its values?

[MatthewLM]

What's low and what's high?

Mine trie to figure out a strategy itself an d it found really good strategies... that only worked for old data. It was good at hindsight, rubbish at foresight. Maybe it needed tweaking.

This problem is known as overfitting.  The solution is to add noise to the data.  Create 10 copies of the data with random permutations to train it on.

So your solution would be to copy the data and add various noise... how would this noise be added, random changes in the prices? What should the spread and distribution in the randomness be?

I guess using more data and perhaps testing for consistency would help also.

[notme]

What's low and what's high?

Mine trie to figure out a strategy itself an d it found really good strategies... that only worked for old data. It was good at hindsight, rubbish at foresight. Maybe it needed tweaking.

This problem is known as overfitting.  The solution is to add noise to the data.  Create 10 copies of the data with random permutations to train it on.

So your solution would be to copy the data and add various noise... how would this noise be added, random changes in the prices? What should the spread and distribution in the randomness be?

I guess using more data and perhaps testing for consistency would help also.

That's the art of it.  I might try something along the lines of scaling similar movements from different timescales to replace the move that actually happened.  This will give the solution space more robustness to handle similar, but slightly different situations.

[MatthewLM]

Though mixing bits up might create an inaccurate representation of the actual market do you think?

I could experiment a little with different algorithms and test them side by side. I might come back to it but I've got other things I should perhaps be working on.

[notme]

Though mixing bits up might create an inaccurate representation of the actual market do you think?

I could experiment a little with different algorithms and test them side by side. I might come back to it but I've got other things I should perhaps be working on.

Sure... ideally you want to maintain valid wave counts.  I wish I had the time to build such a wave validity checker and mutator.

[guruvan]

Though mixing bits up might create an inaccurate representation of the actual market do you think?

I could experiment a little with different algorithms and test them side by side. I might come back to it but I've got other things I should perhaps be working on.

Sure... ideally you want to maintain valid wave counts.  I wish I had the time to build such a wave validity checker and mutator.

a wave counting bot would be soooo cool.

[Seal]

Are there any open source classes where you could add a data feed and it will spit out calcs on various technical indicators?

The charting functions on Bitcoincharts is similar to what I'm after but I want to be able to programatically receive the chart outputs. The website says the charts are under creative commons licence, where can I find the source code?

[organofcorti]

Are there any open source classes where you could add a data feed and it will spit out calcs on various technical indicators?

The charting functions on Bitcoincharts is similar to what I'm after but I want to be able to programatically receive the chart outputs. The website says the charts are under creative commons licence, where can I find the source code?

R has plenty, and writing a script to plot them all is quite easy. You just have to have the data from mtgox, and you can quickly split it into hi-lo-close and then have functions operate on the data.

You want library(xts),library(quantmod) and library(TTR). TTR has about 35 or 40 technical trading rules, for example easy to vary SMA/EMA.

Then if you want to do bootstrapping you can use (backtesting with variation) you can use library(quantstrat) (i think, this is all from memory).

All of these can be applied within rgp to give you huge flexibility when it comes to evolving GAs. If you do bootstrapping right, you don't have problems with overfitting.

[Seal]

Are there any open source classes where you could add a data feed and it will spit out calcs on various technical indicators?

The charting functions on Bitcoincharts is similar to what I'm after but I want to be able to programatically receive the chart outputs. The website says the charts are under creative commons licence, where can I find the source code?

R has plenty, and writing a script to plot them all is quite easy. You just have to have the data from mtgox, and you can quickly split it into hi-lo-close and then have functions operate on the data.

You want library(xts),library(quantmod) and library(TTR). TTR has about 35 or 40 technical trading rules, for example easy to vary SMA/EMA.

Then if you want to do bootstrapping you can use (backtesting with variation) you can use library(quantstrat) (i think, this is all from memory).

All of these can be applied within rgp to give you huge flexibility when it comes to evolving GAs. If you do bootstrapping right, you don't have problems with overfitting.

Thanks organofcorti, I'm reading up about it now. I've been programming for a little while but have never heard of R, even for statistical analysis. How come it isnt more mainstream? I've heard of languages such as K and Q which seems to be used fairly widely for banks processing tick by tick data however trading strategies tend to be predominantly done in C.

Do you have an overview of the pros/cons of R, and how it compares to the others. I wanna get a bit more of an idea what the language is like before investing serious time into learning it.

Can it interface easily with PHP and/or the existing API's to automatically place trades?

[organofcorti]

Thanks organofcorti, I'm reading up about it now. I've been programming for a little while but have never heard of R, even for statistical analysis. How come it isnt more mainstream? I've heard of languages such as K and Q which seems to be used fairly widely for banks processing tick by tick data however trading strategies tend to be predominantly done in C.

Do you have an overview of the pros/cons of R, and how it compares to the others. I wanna get a bit more of an idea what the language is like before investing serious time into learning it.

Can it interface easily with PHP and/or the existing API's to automatically place trades?

I'm not a good person to ask. Apart from R I've done a little C, python, and basic and that's about it (I can feel my kudos disappear as I write shudder). But I like R, it's extensible, flexible, has error messages that actually help you find bugs, and can do everything I need it to. I've scraped webpages with it, and stochastic wrote some code that interfaces with bitcoinica (posted here.

Another good upside is that it's been around in one form or another for decades. There's not one question I've had that couldn't be answered by google.

The downside is that it is very specialised. It's an object oriented math language, like matlab (only better) or mathematica (but open source and not quite as good). How much that will affect you I can't tell since I've never written an active tradebot, just the algorithms (they've done well, but I didn't power bots with them since they only traded on a week or month scale). Stochastic shows how it's done, though.

[MatthewLM]

I updated my algorithm in attempt to get it to test for consistency over different time periods and also use use more data but unfortunately it has a weird problem. http://stackoverflow.com/questions/10036739/line-of-c-code-causes-a-segmentation-fault-on-a-completely-irrelevant-line-of-co 

[MelMan2002]

I updated my algorithm in attempt to get it to test for consistency over different time periods and also use use more data but unfortunately it has a weird problem. http://stackoverflow.com/questions/10036739/line-of-c-code-causes-a-segmentation-fault-on-a-completely-irrelevant-line-of-co  

Don't you just need to define

FILE * file;

first and then do the

file = fopen(....

on a separate line?

[MatthewLM]

I'm currently unable to test that but that should not be the case. It works when I return from the other function before the suspected bad line occurs.

[Kettenmonster]

What is a CombinationResult? ... possibly a pointer?

[MatthewLM]

Forgot about that, it's a structure with an int and float.

[Kettenmonster]

Code:

(*PPO_results)[x]

is a valid object?
Make sure all objects used are initialized plus not yet released.

Is it possibly an

Code:

extern "C" {

issue?

p.s.

Code:

float usd,btc,bid,ask,order_price,possible_price;

Bad style I´d say.
Not so much of an issue on basic types, but still.
My Grandma used to say: "Wokeen Grössn ni eert ward n Daaler ni weert!" meaning don´t be sloppy even not on little things.

[MatthewLM]

Code:

(*PPO_results)[x]

This is is not a C++ object. I'm not even using C++, I'm using C. The type is CombinationResult[PPO_COMBINATIONS * 11].

Also I don't know why declaring several variables on one line is bad style...

[Kettenmonster]

I'm using C.

... so check your pointers. Make sure nothing is pointing to the stack and so on.