Gekko - a javascript trading bot and backtesting platform

[arne]

Hi, this might be a bug:

Code:

2014-01-02 19:46:57 (INFO):     ADVICE is to BUY @ 768.000 (0.406)

/home/pi/arne/gekko/portfolioManager.js:153
  if(this.minimalOrder.unit === 'currency')
                      ^
TypeError: Cannot read property 'unit' of undefined
    at [object Object].<anonymous> (/home/pi/arne/gekko/portfolioManager.js:153:23)
    at [object Object].bound [as getMinimum] (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at [object Object].<anonymous> (/home/pi/arne/gekko/portfolioManager.js:169:22)
    at [object Object].bound [as buy] (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at [object Object].<anonymous> (/home/pi/arne/gekko/portfolioManager.js:126:12)
    at bound (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:229:13
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:139:25
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:226:17
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:553:34

Trying to buy @ BTC-e.
Where should the minimalOrder be set?
Thanks for the good work (y)

[Scorpius]

This is definitely a great bot but it's kinda too agressive.  The "agressiveness" should be an option.  

What I mean is, this bot will always advice to sell even if it bought for a higher price before.  Even though in my testing is better to let the bot lose money, I think some of us prefer to take that decision manually.  So I modified it a little:

1.- Never start with a SELL.  Always start with a BUY.

2.- Never sell for a price lower than the last buy operation.  Hold until year 2140 (when the last bitcoin will be mined) if it's necessary.

Right now I'm trying to configure the thresholds to make the bot work well with 5 minute candles since one hour is way too much:  Many things happen in one hour in the bitcoin world.

Anyway, this is great work though.  Congrats.

[whydifficult]

Hi, this might be a bug:

Code:

2014-01-02 19:46:57 (INFO):     ADVICE is to BUY @ 768.000 (0.406)

/home/pi/arne/gekko/portfolioManager.js:153
  if(this.minimalOrder.unit === 'currency')
                      ^
TypeError: Cannot read property 'unit' of undefined
    at [object Object].<anonymous> (/home/pi/arne/gekko/portfolioManager.js:153:23)
    at [object Object].bound [as getMinimum] (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at [object Object].<anonymous> (/home/pi/arne/gekko/portfolioManager.js:169:22)
    at [object Object].bound [as buy] (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at [object Object].<anonymous> (/home/pi/arne/gekko/portfolioManager.js:126:12)
    at bound (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:229:13
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:139:25
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:226:17
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:553:34

Trying to buy @ BTC-e.
Where should the minimalOrder be set?
Thanks for the good work (y)

Ah :/ will look at this

This is definitely a great bot but it's kinda too agressive.  The "agressiveness" should be an option. 

What I mean is, this bot will always advice to sell even if it bought for a higher price before.  Even though in my testing is better to let the bot lose money, I think some of us prefer to take that decision manually.  So I modified it a little:

1.- Never start with a SELL.  Always start with a BUY.

2.- Never sell for a price lower than the last buy operation.  Hold until year 2140 (when the last bitcoin will be mined) if it's necessary.

Right now I'm trying to configure the thresholds to make the bot work well with 5 minute candles since one hour is way too much:  Many things happen in one hour in the bitcoin world.

Anyway, this is great work though.  Congrats.

Sounds great, would be awesome if you could offer it back as an alternative trading method. Just a couple of things to keep in mind with your suggestion:

1. If you never start with a sell and Gekko stops & restarts (crash, user turned off) after a buy if can't do anything until you manually sell.
2. While this sounds great in theory it can be dangerous: if the bot bought at ~1200 during the last peak it won't do anything ever anymore. Even though there is money to be made even though the price is < 1200.

> Right now I'm trying to configure the thresholds to make the bot work well with 5 minute candles since one hour is way too much.

If you set the interval in the config to 5, Gekko will calculate 5 minute candles instead of 60 minute. It wil also recheck every 5 minutes and make a trade decision.

[Scorpius]

Sounds great, would be awesome if you could offer it back as an alternative trading method. Just a couple of things to keep in mind with your suggestion:

1. If you never start with a sell and Gekko stops & restarts (crash, user turned off) after a buy if can't do anything until you manually sell.
2. While this sounds great in theory it can be dangerous: if the bot bought at ~1200 during the last peak it won't do anything ever anymore. Even though there is money to be made even though the price is < 1200.

> Right now I'm trying to configure the thresholds to make the bot work well with 5 minute candles since one hour is way too much.

If you set the interval in the config to 5, Gekko will calculate 5 minute candles instead of 60 minute. It will also recheck every 5 minutes and make a trade decision.

That's my point: No bots should be left unattended.  The "aggressive" option off is meant so you can decide for yourself, but the bot should still alert you (via email) that the downtrend has come so you can take a decision.

I insist: In my back tracking simulations the bot have made more profit if I let it lose money, but there are some simulated trades that I frankly would have panicked.  This makes me think about a thousand of alternatives, including letting the bot decide how much you can lose like configuring a percentage or something.  Good traders always lose money in some bad trades, but they always make good money in good trades, so it's part of the business.

Bitcoin trading using 5 minute candles based EMA is VERY TRICKY (so the ocasional trader reading this: please stick to the 60 minutes intervals), and looking for crossovers between 10 and 21 exponential moving averages in 5 minutes intervals just don't work; but there are some numbers that do

I love this bot! though I use it as an advisor that alerts me, I'm not using it like an automatic trader bot.

[tom_o]

If anyone is interested in running Gekko, could you weigh in on the discussion with:

- On what machine do you want to run Gekko (desktop, laptop, raspberry PI or VPS/cloud instance)
- How big can the history be that Gekko stores on disk (in MB).
- How much memory can Gekko consume per market you want to watch (1-10MB, 10-100MB or more?)

I run on an amazon AWS free instance - single core windows VPS with 615MB RAM. Only have 13GB diskspace so under 2GB per currency pair would be fine.
Barely any RAM free though so it would have to be efficient swapping to disk or running off of 100MB.

[whydifficult]

I run on an amazon AWS free instance - single core windows VPS with 615MB RAM. Only have 13GB diskspace so under 2GB per currency pair would be fine.
Barely any RAM free though so it would have to be efficient swapping to disk or running off of 100MB.

Ok awesome that sounds reasonable. Do you have any preferences of how many markets you want to watch (in the case Gekko can trade every market on BTC-e at the same time in one algo for example)? Because a single market does not drain that much recourses.

So this is what Gekko is probably going to look like right now:

- persists about 100k / market data to disk (less than 40MB per year) - in 1 minute candles
- around ~30 MB of memory usage per market.

[tom_o]

I run on an amazon AWS free instance - single core windows VPS with 615MB RAM. Only have 13GB diskspace so under 2GB per currency pair would be fine.
Barely any RAM free though so it would have to be efficient swapping to disk or running off of 100MB.

Ok awesome that sounds reasonable. Do you have any preferences of how many markets you want to watch (in the case Gekko can trade every market on BTC-e at the same time in one algo for example)? Because a single market does not drain that much recourses.

So this is what Gekko is probably going to look like right now:

- persists about 100k / market data to disk (less than 40MB per year) - in 1 minute candles
- around ~30 MB of memory usage per market.

Yeah sounds about perfect, I'd want to to watch LTC/BTC, USD/BTC and USD/LTC.

I know it's a long shot but would a GUI be possible at any point? I can use it ok but there are a couple of things I'm unsure of how to do such as the historical testing.

[whydifficult]

Yeah sounds about perfect, I'd want to to watch LTC/BTC, USD/BTC and USD/LTC.

I know it's a long shot but would a GUI be possible at any point? I can use it ok but there are a couple of things I'm unsure of how to do such as the historical testing.

Well those would be perfectly possible I think.

So this is the roadmap for Gekko:

0.1.0:

TODO before we can merge the localDB branch into Master (Gekko 0.1.0)

*(We only merge stable stuff into master).*

- Fix remaining bugs in the datastore (#106).
- Move filestore from neDB to our own.
- Finalize [actor API](https://github.com/askmike/gekko/blob/localDB/docs/internals/Actors.md)
- Add test cases for all vital parts of core
  - ~~Trade fetcher~~
  - ~~candle store~~
  - candle calculator
- Add test cases for all trading methods and indicators:
  - ~~EMA~~
  - EMAC
  - MACD
- Verify that all tests are even valid
- Add a trader actor (hook up to portfolioManager)
- make the Trade fetcher able to handle exchanges that do provide historical data (either `since` or `tid`).
- Add basic backtesting capabilities.

More info here and here.

After that all works we are planning some exciting stuff like a webbased GUI, sneak peak of my early early prototype (more of a proof of concept really):



The GUI is going to be the main interface in that you can backtest through here, manage histories, manage traders, manage profits, etc. Though this is going to take a while if I keep on developing this whole project on my own.

[flower1024]

i've made a small web interface for it.

screenshot: https://www.dropbox.com/s/od0ac7djvvdyrps/gekkoweb.png
source: https://www.dropbox.com/s/54y2crazfnyghnd/gekko.zip

i am not familiar with npm or node. so you have to install a few dependencies by hand (eg npm install express).

there are currently some drawbacks:
 - portfolio is only updated after a successfull buy or sell
 - web interface does work as soon as a the first advice is made.
 - only ema does work as the web if does show some internal data from it

i use it in "advanced mode" using data from mtgox and trading on mtgox and btce (but only with small amounts)

i did not test it with other configurations, so use it at your own risk (i am not responsiple for any losses etc).

as this is a pure personal project i dont think i'll make other extensions, but i am always open for suggestions

regards
flower

[whydifficult]

i've made a small web interface for it.

screenshot: https://www.dropbox.com/s/od0ac7djvvdyrps/gekkoweb.png
source: https://www.dropbox.com/s/54y2crazfnyghnd/gekko.zip

i am not familiar with npm or node. so you have to install a few dependencies by hand (eg npm install express).

there are currently some drawbacks:
 - portfolio is only updated after a successfull buy or sell
 - web interface does work as soon as a the first advice is made.
 - only ema does work as the web if does show some internal data from it

i use it in "advanced mode" using data from mtgox and trading on mtgox and btce (but only with small amounts)

i did not test it with other configurations, so use it at your own risk (i am not responsiple for any losses etc).

as this is a pure personal project i dont think i'll make other extensions, but i am always open for suggestions

regards
flower

Woow this looks great! I am pretty sure we can use a lot of your code and put it into Gekko (if you don't mind).

So we are working on a new version for Gekko that has a plugin system which makes it easy to build stuff on top of Gekko. The demo server I built uses this system already.

It also uses websockets in favor of a rest style API. Though I am pretty sure we can leverage your code quite a bit (especially the frontend stuff)

Awesome stuff!

[flower1024]

i've made a small web interface for it.

screenshot: https://www.dropbox.com/s/od0ac7djvvdyrps/gekkoweb.png
source: https://www.dropbox.com/s/54y2crazfnyghnd/gekko.zip

i am not familiar with npm or node. so you have to install a few dependencies by hand (eg npm install express).

there are currently some drawbacks:
 - portfolio is only updated after a successfull buy or sell
 - web interface does work as soon as a the first advice is made.
 - only ema does work as the web if does show some internal data from it

i use it in "advanced mode" using data from mtgox and trading on mtgox and btce (but only with small amounts)

i did not test it with other configurations, so use it at your own risk (i am not responsiple for any losses etc).

as this is a pure personal project i dont think i'll make other extensions, but i am always open for suggestions

regards
flower

Woow this looks great! I am pretty sure we can use a lot of your code and put it into Gekko (if you don't mind).

So we are working on a new version for Gekko that has a plugin system which makes it easy to build stuff on top of Gekko. The demo server I built uses this system already.

It also uses websockets in favor of a rest style API. Though I am pretty sure we can leverage your code quite a bit (especially the frontend stuff)

Awesome stuff!

just take what you need..

one suggestion for graphs and multiple indicators: let the indicator export the jqplot options and data (i am planning to do this myself to play with it).

i would really like to see a bitfinex exchange plugin. and ways to decide a risk strategy for margin trading (but the exchange part of bitfinex would be a nice first step)

btw great work. i really enjoyed working with it!

[helluvaname]

whydifficult, thanks for sharing your bot!
I was desperatly trying to backtest my EA on btc-e's MT4, but I fail due to strategy tester wrong commission calculation on every data older then 24 november 2013.
So, right now I am looking towards using your bot and I have several question I hope someone will answer

1) does Gekko counts broker commission while backtesting?

2) does it have any kind of a trailing stop?

3) in all the programming world I am slightly familiar with C++, nothing more, how long will it take me to implement those features listed above in the Gekko bot if they are not already on board? (of course I am not seeking for a precise answer, just an estimation like - a day, a week, several months?)

[Quora]

Gekko saved my life, thanks again for sharing. Amazing new developments, and great suggestions to include Kraken (really the most professional exchange so far) and Bitfinex too

EDIT: Now I got many errors and warnings when installing Gekko on one laptop (after several entirely unsuccessful attempts) but after this most recent attempt it seems like it successfully installed, and it started calculating EMA on historical data. Regardless of that, should I try again until it's a totally clean install? I'm new to almost any type of coding, please help!

[whydifficult]

whydifficult, thanks for sharing your bot!
I was desperatly trying to backtest my EA on btc-e's MT4, but I fail due to strategy tester wrong commission calculation on every data older then 24 november 2013.
So, right now I am looking towards using your bot and I have several question I hope someone will answer

1) does Gekko counts broker commission while backtesting?

2) does it have any kind of a trailing stop?

3) in all the programming world I am slightly familiar with C++, nothing more, how long will it take me to implement those features listed above in the Gekko bot if they are not already on board? (of course I am not seeking for a precise answer, just an estimation like - a day, a week, several months?)

1. Gekko connects straights to exchanges, no brokers involved. Though exchanges also charge fees (which most of the time depends on the amount of volume). The backtest simulates buying / selling directly at the exchange. With Gekko you can configure this fee and the backtester will take it into account, the default is 0.6% (the default fee on mt. gox).

2. At this moment not, the version you are looking at only support the method EMA as described here. We are working on a new (but not yet stable) version of Gekko which currently supports DEMA, MACD and PPO and we are trying to include more asap.

Gekko saved my life, thanks again for sharing. Amazing new developments, and great suggestions to include Kraken (really the most professional exchange so far) and Bitfinex too

EDIT: Now I got many errors and warnings when installing Gekko on one laptop (after several entirely unsuccessful attempts) but after this most recent attempt it seems like it successfully installed, and it started calculating EMA on historical data. Regardless of that, should I try again until it's a totally clean install? I'm new to almost any type of coding, please help!

Awesome thanks!


--

I have spawned a mini chart website where you can watch the following markets in semi realtime (updates automatically about every minute):

- bitstamp-USD-BTC
- mtgox-USD-BTC
- cexio-BTC-GHS
- btce-USD-BTC
- btce-USD-LTC
- btce-BTC-LTC
- btce-RUR-BTC
- btce-BTC-NVC

data.wizb.it.

All the data is delivered by 8 gekkos (of the new version) running in parallel across all those markets.

[helluvaname]

2. At this moment not, the version you are looking at only support the method EMA as described here. We are working on a new (but not yet stable) version of Gekko which currently supports DEMA, MACD and PPO and we are trying to include more asap.

Thanks for the reply!
I didn't get it though, will trailing stop be included along with new stuff or it isn't actually on your to-do list?
I couldn't find it here https://github.com/askmike/gekko/issues/114
In my backtests EMA strategies with a trailing stop outperformed strategies that don't use one by at least 20% in profit.

Is there any estimated time of arrival for a new stable version?

[whydifficult]

Thanks for the reply!
I didn't get it though, will trailing stop be included along with new stuff or it isn't actually on your to-do list?
I couldn't find it here https://github.com/askmike/gekko/issues/114
In my backtests EMA strategies with a trailing stop outperformed strategies that don't use one by at least 20% in profit.

Is there any estimated time of arrival for a new stable version?

So after stabilizing and verifying this new version and adding the real trader (0.1.0) I'll start working on some things already planned for 0.1.1, a trailing stop is not part of this yet.

Though I have a couple of things planned for after that, these include:

- More indicators (RSI, Ichimoku, Heikin Ashi, trailing stop losses, etc).
- More exchanges (bitfinex, kraken, etc)
- webbased interface

--

I don't have any ETA as this is an open source project unfortunately. Though you can always support the project by adding the algo you want yourself.

[Thom]

currently enjoying the localdb branch, cos i dig cex.io and gekko is, as far as I know, the only free backtesting game in town for cex since cryptotrader went paid-only. Plus i can actually run gekko in macosX without wrapping it in a VM.

Interestingly, I'm still running the cryptrade opensource bot alongside gekko with identical EMA settings, but they're not giving identical advice. In fact, it's barely only similar. But though they differ, gekko hasn't given any bad advice. It's just a bit different to how the cryptrade bot behaves. I'll try the 'traditional' 10/21 and see if they agree then. Hmm.

ANYHOO. I keep gekko open in a terminal, but in the usual procession of

Code:

2014-01-21 00:44:43 (INFO):	Processed trades, sleeping for a minute...

messages, it would be nice if every line, the last buy advice was repeated. It's sometimes a marathon of scrolling to see what the last advice was, and it's often scrolled out of buffer.

This would be easily visible also if COLO(u)RS got involved, too. Imagine:

2014-01-21 00:44:43 (INFO):   Trading advice: Long - BUY   | Processed trades, sleeping for a minute...
2014-01-21 00:44:43 (INFO):   Trading advice: None - WAIT | Processed trades, sleeping for a minute...
2014-01-21 00:52:34 (INFO):   Trading advice: Short - SELL | Processed trades, sleeping for a minute...

Then even one line would be enough to be advised with a quick glance.

It would also be pretty sweet to have an option to have a verbose line of the current algo. Cryptrade's EMA algo rocks it like this, if enabled:

Code:

2014-01-21T00:55:04.253Z - verbose: EMA difference: 0.01209289 price: 0.05 at Tue Jan 21 2014 01:50:00 GMT+0100 (CET)

A bit of that action would be good for judging those "maybe it's time for human trading intervention" moments.

And much as i'm on the localdb branch, so it's not trading for me, it would be most beneficial to be able to easily toggle trading/watching with a keystroke. When a market lurches oddly and you have to jump in to do something, got to be able to disable a bot real quick!

[ThomasTheTrain]

So as a med student with a weak technical background in this script, I am having some trouble configuring everything. Honestly, I am struggling immediately beyond the stage of [node gekko]. Is there somewhere you can point me for further (simple) reading, that'll hopefully clarify the process of getting the trading bot up and running for me?

[whydifficult]

The new version (localDB branch) is nearing a stable release. Real trading now works (except Bitstamp, they banned my main IP so will come in an hour or so).

To sum it up:

There is a new version of Gekko which has trading methods for EMA (called DEMA), MACD and PPO on all markets on BTC-e, Mt. Gox, CEX.io. This includes all altcoin markets for BTC-e! So you can now trade LTC and other altcoins with Gekko.

currently enjoying the localdb branch, cos i dig cex.io and gekko is, as far as I know, the only free backtesting game in town for cex since cryptotrader went paid-only. Plus i can actually run gekko in macosX without wrapping it in a VM.

And also the only one where you can verify yourself that nobody tinkered with any data.

Interestingly, I'm still running the cryptrade opensource bot alongside gekko with identical EMA settings, but they're not giving identical advice. In fact, it's barely only similar. But though they differ, gekko hasn't given any bad advice. It's just a bit different to how the cryptrade bot behaves. I'll try the 'traditional' 10/21 and see if they agree then. Hmm.

So I know of a implementation difference: if you set Gekko to use hourly candles it won't start at full hours, but as soon as it got the history it needs (can be at :41 for example). This way the candles / data / triggers can all be off. If you find any other differences let me know.

ANYHOO. I keep gekko open in a terminal, but in the usual procession of

Code:

2014-01-21 00:44:43 (INFO):	Processed trades, sleeping for a minute...

messages, it would be nice if every line, the last buy advice was repeated. It's sometimes a marathon of scrolling to see what the last advice was, and it's often scrolled out of buffer.

Those processed trades messages are only logged in debug mode since a couple of days now. Let me know if that helps.

This would be easily visible also if COLO(u)RS got involved, too. Imagine:

2014-01-21 00:44:43 (INFO):   Trading advice: Long - BUY   | Processed trades, sleeping for a minute...
2014-01-21 00:44:43 (INFO):   Trading advice: None - WAIT | Processed trades, sleeping for a minute...
2014-01-21 00:52:34 (INFO):   Trading advice: Short - SELL | Processed trades, sleeping for a minute...

Then even one line would be enough to be advised with a quick glance.

It would also be pretty sweet to have an option to have a verbose line of the current algo. Cryptrade's EMA algo rocks it like this, if enabled:

Code:

2014-01-21T00:55:04.253Z - verbose: EMA difference: 0.01209289 price: 0.05 at Tue Jan 21 2014 01:50:00 GMT+0100 (CET)

A bit of that action would be good for judging those "maybe it's time for human trading intervention" moments.

Good idea, will improve this.

And much as i'm on the localdb branch, so it's not trading for me, it would be most beneficial to be able to easily toggle trading/watching with a keystroke. When a market lurches oddly and you have to jump in to do something, got to be able to disable a bot real quick!

It is auto trading now, read above!

So as a med student with a weak technical background in this script, I am having some trouble configuring everything. Honestly, I am struggling immediately beyond the stage of [node gekko]. Is there somewhere you can point me for further (simple) reading, that'll hopefully clarify the process of getting the trading bot up and running for me?

We have a new version which is almost stable now. This one has way more clear documentation:

https://github.com/askmike/gekko/blob/localDB/docs/Configuring_gekko.md

The new one is almost considered stable (we need to make sure it doesn't make mistakes, etc, which we almost are). You can download that one by running:

Code:

git clone https://github.com/askmike/gekko.git --single-branch localDB

For both versions it works like this: you edit everything you want in `config.js` and you type `node gekko` to launch it, `ctrl c` to stop it.

[Thom]

new localdb version gitpulled. And very nice it is too. Once cex flatlines again I'll let it trade. One more thought:

Profit reports could be cut to two lines, and would be nice if expressed in currency and asset, with the item you're currently holding in bold/coloured. like so?

2014-01-22 12:12:02 (INFO):   (PROFIT REPORT) original / current simulated balance: 0.58000 BTC / 1.00000 BTC
2014-01-22 12:12:02 (INFO):   (PROFIT REPORT) simulated profit: 1.00000 BTC / 4.00000 GHS (1.00000%)