Gekko - a javascript trading bot and backtesting platform

[kenshin23]

I'm currently using Gekko to help me automate trading and follow the system that Goomboo posted. I don't know much about trading and there's still so much to learn, but I've noticed something during backtesting that I think is related to the latest posts.

When doing tests, does anyone change the number of initial candles to store? I took the H1 data from the last 5 months at BitcoinCharts (3570 candles), changed the format to what the backtester expects and tried this:

EMA 10/21, +-0.25 threshold, 60 minutes interval:

	Simulated profit	Simulated yearly profit	# trades
100 candles:	115.715%	286.074%	61
200 candles:	112.729%	286.653%	59
300 candles:	160.956%	421.325%	56
400 candles:	174.353%	470.224%	55

...and so on.

I'm wondering if there's really a need to change that setting or not. Note the huge change from 200 to 300 candles.

[davider]

Hi whydifficult,

what about running gekko on a Raspberry Pi?
It would be great to support an external LCD like they did with this Raspberry USB Miner project:

http://learn.adafruit.com/piminer-raspberry-pi-bitcoin-miner/
https://github.com/adafruit/piminer

Maybe not an high priority feature but it could be cool! 

[TradesLikeAPotato]

Awesome bot! I love the backtesting feature. I'm getting an error with setting up email though, here's the console:

2013-08-13 23:44:50 (INFO):     I'm gonna make you rich, Bud Fox.
2013-08-13 23:44:50 (INFO):     Let me show you some Exponential Moving Averages
.


2013-08-13 23:44:50 (INFO):     Using normal settings to monitor the live market

2013-08-13 23:44:50 (INFO):     NOT trading with real money
2013-08-13 23:44:51 (INFO):     Profit reporter active on simulated balance
2013-08-13 23:44:51 (INFO):     Calculating EMA on historical data...
2013-08-13 23:44:58 (INFO):     ADVICE is to BUY @ 96.709 (0.454)
2013-08-13 23:44:58 (WARN):     ERROR SENDING MAIL { [Error: authorization.faile
d]
  code: 3,
  previous:
   { [Error: bad response on command 'AUTH']
     code: 2,
     smtp: '334 eyJzdGF0dXMiOiI0MDAiLCJzY2hlbWVzIjoiQmVhcmVyIiwic2NvcGUiOiJodHRw
czovL21haWwuZ29vZ2xlLmNvbS8ifQ==\n' },
  smtp: undefined }

[dplusf]

Awesome bot! I love the backtesting feature. I'm getting an error with setting up email though, here's the console:

2013-08-13 23:44:50 (INFO):     I'm gonna make you rich, Bud Fox.
2013-08-13 23:44:50 (INFO):     Let me show you some Exponential Moving Averages
.


2013-08-13 23:44:50 (INFO):     Using normal settings to monitor the live market

2013-08-13 23:44:50 (INFO):     NOT trading with real money
2013-08-13 23:44:51 (INFO):     Profit reporter active on simulated balance
2013-08-13 23:44:51 (INFO):     Calculating EMA on historical data...
2013-08-13 23:44:58 (INFO):     ADVICE is to BUY @ 96.709 (0.454)
2013-08-13 23:44:58 (WARN):     ERROR SENDING MAIL { [Error: authorization.faile
d]
  code: 3,
  previous:
   { [Error: bad response on command 'AUTH']
     code: 2,
     smtp: '334 eyJzdGF0dXMiOiI0MDAiLCJzY2hlbWVzIjoiQmVhcmVyIiwic2NvcGUiOiJodHRw
czovL21haWwuZ29vZ2xlLmNvbS8ifQ==\n' },
  smtp: undefined }

Hi,

i got exactly the same error after updating to the latest version.

[whydifficult]

I'm currently using Gekko to help me automate trading and follow the system that Goomboo posted. I don't know much about trading and there's still so much to learn, but I've noticed something during backtesting that I think is related to the latest posts.

When doing tests, does anyone change the number of initial candles to store? I took the H1 data from the last 5 months at BitcoinCharts (3570 candles), changed the format to what the backtester expects and tried this:

[...]

I'm wondering if there's really a need to change that setting or not. Note the huge change from 200 to 300 candles.

I am looking in to the data as to what kind of difference this makes, but do note:

The EMA method is supposed to start out with a history, the ideal case would be the complete history but since this is not practicle (and the difference decreases exponentially as you remove old history). If you don't start with a big history you aren't using pure EMA as you are not able to calculate the real EMA (a number which is built up using the complete history). Diabolicus showed the EMA difference in EMA calculation between a big and a small history.

If you are looking at the profit as a result of different histories you are just comparing different kind of EMAs (different levels of pure EMA so to say) which each other. It may the case that an EMA without an history would be the most profitable in a backtest, but it pretty hard to turn that into conclusions.

Hope that this explanation makes sense.

Awesome bot! I love the backtesting feature. I'm getting an error with setting up email though, here's the console:

2013-08-13 23:44:50 (INFO):     I'm gonna make you rich, Bud Fox.
2013-08-13 23:44:50 (INFO):     Let me show you some Exponential Moving Averages
.


2013-08-13 23:44:50 (INFO):     Using normal settings to monitor the live market

2013-08-13 23:44:50 (INFO):     NOT trading with real money
2013-08-13 23:44:51 (INFO):     Profit reporter active on simulated balance
2013-08-13 23:44:51 (INFO):     Calculating EMA on historical data...
2013-08-13 23:44:58 (INFO):     ADVICE is to BUY @ 96.709 (0.454)
2013-08-13 23:44:58 (WARN):     ERROR SENDING MAIL { [Error: authorization.faile
d]
  code: 3,
  previous:
   { [Error: bad response on command 'AUTH']
     code: 2,
     smtp: '334 eyJzdGF0dXMiOiI0MDAiLCJzY2hlbWVzIjoiQmVhcmVyIiwic2NvcGUiOiJodHRw
czovL21haWwuZ29vZ2xlLmNvbS8ifQ==\n' },
  smtp: undefined }

Hi,

i got exactly the same error after updating to the latest version.

Darn APIs that keep changing, working on it!

[whydifficult]

Hi whydifficult,

what about running gekko on a Raspberry Pi?
It would be great to support an external LCD like they did with this Raspberry USB Miner project:

http://learn.adafruit.com/piminer-raspberry-pi-bitcoin-miner/
https://github.com/adafruit/piminer

Maybe not an high priority feature but it could be cool! 

There are people running Gekko on a raspberry PI, however I am not familar with such a LED display on a PI (I am on an Arduino). In the long term I want to offer an easy API so that people (who own such cool stuff) can route the output to wherever they want: on their raspberry PI LED, on their website, to their Google glass, etc.

---


As I've said before I am working on my own solution for the exchanges API problem. Here you can see a sneak preview

Wizbit

[itod]

As I've said before I am working on my own solution for the exchanges API problem. Here you can see a sneak preview

Wizbit

Nice, can you give some more details?

[whydifficult]

As I've said before I am working on my own solution for the exchanges API problem. Here you can see a sneak preview

Wizbit

Nice, can you give some more details?

Sure

It started out with me working on Gekko and getting tired of all the API changes and how difficult it is to get historical trade data. Bitcoincharts has a lot of nice data, but they also changed their API silently twice now since I started Gekko.

So basically I am trying to setup a service where I offer a lot of exchange trade data (in charts as well as through an API), similar to bitcoincharts but also supporting alt coins, a stable API and more asian (chinese) markets.

I have started watching a bunch of different exchanges and I am working on exposing candles for all these markets through a beta version of Wizbit soon. Here is the list of markets I am currently watching, as well as some DB info:

http://wizb.it/watcher.txt

* Note that I do this all in my free time, so a solid platform may never come, I also have a couple of bugs in Gekko I still need to fix so it may take a while to get somewhere.

[telemaco]

Thanks a lot for your bot. It is awesome !

One question. To minimize risk of exchange shutdown and also to greately reduce slippage but still have different configurations i would like to run different ema and percent values on different exchanges. Can i just copy the gekko folder and run different instances with different configurations?

I have copied the folder to ...\Desktop\gekko-master\gekko-master2\ and run node gekko (i have already one running on ...\Desktop\gekko-master\gekko-master\ . Both gekko bots have different EMA configurations. Why? i have very different trading rates and backtesting that seems to heavily change the benefit i get depending on EMAs and percent thresholds.

The first one with btc-e is running fine but with the second one (gekko-master2) i am just getting a "Failed to load historical trades from Bitstamp". I don't know if there has been a change in their api or it is a problem of my configuration. I have api activated and email verified on bitstamp. It seems to be connecting right because it displays the correct amount of BTC and USD but fails getting historical data. I have tried lowering the number of candles from 300 to even 5 and it does the same.

What do you think might be happening? I have latest gekko version.

Thanks!

[whydifficult]

Thanks a lot for your bot. It is awesome !

One question. To minimize risk of exchange shutdown and also to greately reduce slippage but still have different configurations i would like to run different ema and percent values on different exchanges. Can i just copy the gekko folder and run different instances with different configurations?

I have copied the folder to ...\Desktop\gekko-master\gekko-master2\ and run node gekko (i have already one running on ...\Desktop\gekko-master\gekko-master\ . Both gekko bots have different EMA configurations. Why? i have very different trading rates and backtesting that seems to heavily change the benefit i get depending on EMAs and percent thresholds.

The first one with btc-e is running fine but with the second one (gekko-master2) i am just getting a "Failed to load historical trades from Bitstamp". I don't know if there has been a change in their api or it is a problem of my configuration. I have api activated and email verified on bitstamp. It seems to be connecting right because it displays the correct amount of BTC and USD but fails getting historical data. I have tried lowering the number of candles from 300 to even 5 and it does the same.

What do you think might be happening? I have latest gekko version.

Thanks!

The first question about running multiple Gekkos with different settings: You can also copy the config file and run a second Gekko and tell it to use the other config file, you can read more information in the docs. But you can also copy the whole folder.

The problem with Bitstamp is has to do with them changing the API. I am working on distrubiting candles using my own platform, but in the meantime you can try this solution.

[kenshin23]

I'm currently using Gekko to help me automate trading and follow the system that Goomboo posted. I don't know much about trading and there's still so much to learn, but I've noticed something during backtesting that I think is related to the latest posts.

When doing tests, does anyone change the number of initial candles to store? I took the H1 data from the last 5 months at BitcoinCharts (3570 candles), changed the format to what the backtester expects and tried this:

[...]

I'm wondering if there's really a need to change that setting or not. Note the huge change from 200 to 300 candles.

I am looking in to the data as to what kind of difference this makes, but do note:

The EMA method is supposed to start out with a history, the ideal case would be the complete history but since this is not practicle (and the difference decreases exponentially as you remove old history). If you don't start with a big history you aren't using pure EMA as you are not able to calculate the real EMA (a number which is built up using the complete history). Diabolicus showed the EMA difference in EMA calculation between a big and a small history.

If you are looking at the profit as a result of different histories you are just comparing different kind of EMAs (different levels of pure EMA so to say) which each other. It may the case that an EMA without an history would be the most profitable in a backtest, but it pretty hard to turn that into conclusions.

Hope that this explanation makes sense.

Hi, thanks for taking the time to reply! Let's see if I'm getting everything right. When you say "a different history" you mean to say that when I change the number of candles to retain for calculation, I'm basically evaluating a different timeframe = different history, correct?

On another note, I was looking through the source code to see if I could write a tester that would automatically find the optimum EMA values for a given timeframe, but I haven't had much time to do so.

[telemaco]

Thanks a lot for your bot. It is awesome !

One question. To minimize risk of exchange shutdown and also to greately reduce slippage but still have different configurations i would like to run different ema and percent values on different exchanges. Can i just copy the gekko folder and run different instances with different configurations?

I have copied the folder to ...\Desktop\gekko-master\gekko-master2\ and run node gekko (i have already one running on ...\Desktop\gekko-master\gekko-master\ . Both gekko bots have different EMA configurations. Why? i have very different trading rates and backtesting that seems to heavily change the benefit i get depending on EMAs and percent thresholds.

The first one with btc-e is running fine but with the second one (gekko-master2) i am just getting a "Failed to load historical trades from Bitstamp". I don't know if there has been a change in their api or it is a problem of my configuration. I have api activated and email verified on bitstamp. It seems to be connecting right because it displays the correct amount of BTC and USD but fails getting historical data. I have tried lowering the number of candles from 300 to even 5 and it does the same.

What do you think might be happening? I have latest gekko version.

Thanks!

The first question about running multiple Gekkos with different settings: You can also copy the config file and run a second Gekko and tell it to use the other config file, you can read more information in the docs. But you can also copy the whole folder.

The problem with Bitstamp is has to do with them changing the API. I am working on distrubiting candles using my own platform, but in the meantime you can try this solution.

Thanks a million for your answer.

I am trying with really small ema (1/4) periods and 720 minutes but i only manage to get it a really small number of candles With only one it gives an error. With two it just stays there waiting for 12 hours until it gets the minimum to calculate an ema crossover then gives you an advice. With 3 it just says that it could not download the historical data. I have tried also with the standard values but still it gives the failed to load historical trades from bitstamp error.

Thanks again,

[whydifficult]

Hi, thanks for taking the time to reply! Let's see if I'm getting everything right. When you say "a different history" you mean to say that when I change the number of candles to retain for calculation, I'm basically evaluating a different timeframe = different history, correct?

On another note, I was looking through the source code to see if I could write a tester that would automatically find the optimum EMA values for a given timeframe, but I haven't had much time to do so.

the number of candles means two things right now:

- the amount of candles to grab from before `now` to base the EMA history on.
- the amount of candles at any time to keep in memory (this doesn't have to be here at all since you only need the last once you started).

So it influences how pure the EMA calculation will be based on the grabbed history. But because the EMA is recalculated every interval, after running Gekko for a long time this difference will be almost gone. This is also where things are currently broken in Bitstamp & BTC-e: the APIs have changed on how to get older trades.

Thanks a million for your answer.

I am trying with really small ema (1/4) periods and 720 minutes but i only manage to get it a really small number of candles With only one it gives an error. With two it just stays there waiting for 12 hours until it gets the minimum to calculate an ema crossover then gives you an advice. With 3 it just says that it could not download the historical data. I have tried also with the standard values but still it gives the failed to load historical trades from bitstamp error.

Thanks again,

That's a pretty big interval, if you set the amount of candles to 100 Gekko would need to fetch all trades from the last 12 hours * 100. It can't ask for candles so it needs to fetch all those trades and calculate the candles. You could try to turn on debug in the config to get more information on what works and what doesn't. The only other option you have left is Mt. Gox right now.

Note that the new website I am building offers a solution this problem by offering candles (very light weight) instead of all the trades needed to calculate those candles.

[TradesLikeAPotato]

whydifficult, do you have an ETA on the email fix?

[voxelot]

whydifficult.. thank you!.. just started playing with your code today. i will of course inform you of anything interesting I find while testing!

keep up the good work guys!

[whydifficult]

whydifficult, do you have an ETA on the email fix?

It's fixed! The mail dependency I was using broke the authentication with the gmail, luckily someone submitted a fix for it.

How to apply fix:

• git pull
• browse to `gekko/node_modules/` and delete the whole folder `emailjs`
• npm install

whydifficult.. thank you!.. just started playing with your code today. i will of course inform you of anything interesting I find while testing!

keep up the good work guys!

Thanks If you need any help sifting through the code just let me know. Unfortunately I have very little spare time as of late so I don't have as much time as I had hoped to work on Gekko the last couple of weeks.

[TradesLikeAPotato]

whydifficult, do you have an ETA on the email fix?

It's fixed! The mail dependency I was using broke the authentication with the gmail, luckily someone submitted a fix for it.

How to apply fix:

• git pull
• browse to `gekko/node_modules/` and delete the whole folder `emailjs`
• npm install

Thanks a ton!

[Tarion]

Gekko is a good reason to start with nodeJS - I always wanted to learn it. I just started an little bot that also has a web frontend (express & jade) based on / inspired by gekko. Maybe I will be able to contribute something to gekko some day.

For now I don't like (or am not familiar enough with)  some of the coding styles in gekko like these emit("someMethod") calls that are quiet hard to track down with the IDE. There are quiet a lot of frameworks / DSLs to learn in the JS world But thats what I want to understand. So let's see where it ends.

[whydifficult]

Gekko is a good reason to start with nodeJS - I always wanted to learn it. I just started an little bot that also has a web frontend (express & jade) based on / inspired by gekko. Maybe I will be able to contribute something to gekko some day.

For now I don't like (or am not familiar enough with)  some of the coding styles in gekko like these emit("someMethod") calls that are quiet hard to track down with the IDE. There are quiet a lot of frameworks / DSLs to learn in the JS world But thats what I want to understand. So let's see where it ends.

That sounds awesome. If you're on to something I would love to see it, and it would be awesome if you would one day contribute to Gekko.

About the coding style:

I'm not a programmer with XX years of experience or anything, I am just a student with a focus on web technologies. In my spare time I am discovering the trading world with Gekko. It could be possible that the coding style is strange / unfamiliar and I am very willing to see how this can be improved to something more understandable for other people then me.

--

The Emit methods (events) are an easy way to glue separate pieces of code together (once you get the hang of it), they are extremely handy for a lot of different (async) things you are doing with node (except for doing stuff like serving jade templates). It's hard to track down what happens when an event gets triggered but that is also the advantage: you can just plug functions in and out without altering the original function. So for example the piece of code responsible for doing trades doesn't have anything to do with whether or not an email needs to be send, it just emits a 'doing new trade' event and the email function is listening to it based on the cofiguration, etc.

They are also pretty popular on the frontend, with jQuery for example (the difference is that you are almost always listening to events going through the DOM instead of creating them). You can take a look at a pretty popular frontend framework called Backbone, which does not deal with the DOM but is handy when you are creating more complicated web apps. Backbone also uses events everywhere.

If you need any help with understanding it in Gekko, let me know.

[TradesLikeAPotato]

whydifficult I realize I may be getting a little obnoxious about this, but I think I've found another bug in the backtester. When putting a unix timecode in the "from:" category the program misinterprets it (I think). The timecode is 1367989200 which should translate to 2013-05-08 05:00:00 but instead is read as 2013-08-16 17:00:00. Doing the same on the "to:" time works completely fine. Obviously this isn't a pressing issue, since I can still just edit the CSV files, it's just one I figured you should know about.