Gekko - a javascript trading bot and backtesting platform

[kuzetsa]

Testing a new market data (candle) processing engine inspired by "Heikin-Ashi candles"

http://www.investopedia.com/articles/technical/04/092204.asp

So far, I've only incorporated this feature into 3 new experimental MACD-based indicators:

nikiehihsa is Heikin-Ashi spelled backwards(ish ... close enough)

• nikiehihsa -- MACD based on regular EMAs, but with Heikin-Ashi xClose instead of simple Close price
• x3nikiehihsa -- same as above MACD, except using TEMA moving averages
• ZERO -- Heikin-Ashi with zero-lag EMAs (DEMA for long, TEMA for short), and vanilla 1x EMA signal line...
  Additionally, the ZERO method has a few tweaks to more intelligently use more features from the MACD:
  Mostly I just mean it also uses the difference between the EMAs instead of just the momentum oscillator.
  (there is also some other simple logic changes in this method to reduce false "buy signals")

The new ZERO method (as well as the other Heikin-Ashi-based MACD methods) are in the new "prerelease" / beta named "zerolag:

https://github.com/kuzetsa/gekko/releases

Also, the fee rounding code has passed all the testing during the past several days, so the current "latest" release has been bumped to the one where the cex.io fees are actually at the correct 0.2%

https://github.com/kuzetsa/gekko/releases/latest


Edited to add:

I decided that at some point in a future, upcoming release... and only once I finish auditing the relevant code (part of why I'm working on the new "zerolag" method is that it gives me a chance to debug a method I made myself without breaking older code) I'll be re-adding the option for sell orders.

For now, the only working logic is "buy and hold", since that's pretty harmless if it ever acts buggy for something like cex.io GHS cloud mining -- even a bad price is still a reinvestment

Perfect is the enemy of good.

Anything worth doing well is worth doing poorly.

[wizzardTim]

Also, the fee rounding code has passed all the testing during the past several days, so the current "latest" release has been bumped to the one where the cex.io fees are actually at the correct 0.2%

https://github.com/kuzetsa/gekko/releases/latest

Great! I'm gonna install this version then.

I decided that at some point in a future, upcoming release... and only once I finish auditing the relevant code (part of why I'm working on the new "zerolag" method is that it gives me a chance to debug a method I made myself without breaking older code) I'll be re-adding the option for sell orders.

Woohoo! 

[ernie-]

How did the bots go with the large GHS price crash earlier today?

[kuzetsa]

How did the bots go with the large GHS price crash earlier today?

There were several ups and downs with the price, and the zerolag MACD method I was using (with 43-83-7 EMA weights) did a good job for me, and worked almost exactly as I had expected (typically bought at better prices than the manually reinvested buy orders I usually do with my namecoin balance).

Every time the price was in a decline, the zerolag MACD method (43-83-7 weights) would delay the reinvestment "buy" action until the downward price trend started to reverse. Or worded another way, every time it seemed like the price was "as low as it is going to get" the bot reinvested whatever newly mined BTC was available in my portfolio.

Here are my logs (a raw log, and an "cooked" log) for the randomly selected (short/long/signal) 43-83-7 zerolag EMA weights:

RAW) https://gist.github.com/kuzetsa/10883a7bb7c6aaec27f1

COOKED) https://gist.github.com/kuzetsa/bc038a4072fbc653770a


After 2014-07-01 19:05:16 (local NY time) I decided to change my EMA weights.

The data wouldn't have been consistent, so I backed up the log, manually labeling it 43-83-7.RAW.log after I ran my "data cooker" script on it:

https://gist.github.com/kuzetsa/56e951ce821d0e69d96e

I'll be doing much testing to find the EMA weights which work best for my own needs. As should anyone who uses TA indicators to make [re]investment or trading decisions and/or automate them. There is no "one size fits all" trading strategy or method, and that's why the bot will continue to have the ability to do things I don't personally consider to be useful, or wish to use as a basis for automated actions using my API key(s), etc. etc. etc.

[wizzardTim]

So far the bot only buys GHS when I deposit BTC or get BTC from mining. It never sells GHS to buy them later at a cheaper price. Is this normal?

[ernie-]

So far the bot only buys GHS when I deposit BTC or get BTC from mining. It never sells GHS to buy them later at a cheaper price. Is this normal?

It's not meant to sell them. Use the normal gekko if you want that.

[wizzardTim]

So far the bot only buys GHS when I deposit BTC or get BTC from mining. It never sells GHS to buy them later at a cheaper price. Is this normal?

It's not meant to sell them. Use the normal gekko if you want that.

Is the normal gekko working? I thought it is no longer maintained

[kuzetsa]

So far the bot only buys GHS when I deposit BTC or get BTC from mining. It never sells GHS to buy them later at a cheaper price. Is this normal?

It's not meant to sell them. Use the normal gekko if you want that.

Is the normal gekko working? I thought it is no longer maintained

There are some other people maintaining a fork of gekko which still works, but I haven't found any which support all exchanges, as I believe most of the forks which are in a ^[semi-]working state appear to only support and test on 1-2 exchanges.

Speaking of which...

I found one with support for the cryptsy exchange, and I pulled in the code to enable cryptsy support
(I have not tested it yet though, so it's nothing I'm promising if/when it will actually work)

Yesterday's prerelease / 2014 July 2nd (initial cryptsy support)

^ The new cryptsy support is based on this fork:
https://github.com/dowjames/gekko

[ernie-]

So far the bot only buys GHS when I deposit BTC or get BTC from mining. It never sells GHS to buy them later at a cheaper price. Is this normal?

It's not meant to sell them. Use the normal gekko if you want that.

Is the normal gekko working? I thought it is no longer maintained

There are some other people maintaining a fork of gekko which still works, but I haven't found any which support all exchanges, as I believe most of the forks which are in a ^[semi-]working state appear to only support and test on 1-2 exchanges.

Speaking of which...

I found one with support for the cryptsy exchange, and I pulled in the code to enable cryptsy support
(I have not tested it yet though, so it's nothing I'm promising if/when it will actually work)

Yesterday's prerelease / 2014 July 2nd (initial cryptsy support)

^ The new cryptsy support is based on this fork:
https://github.com/dowjames/gekko

Did you merge the crypsty code into your cex.io fork of gekko or are you keeping them separate?

[kuzetsa]

/// (( ... snip ... )) ///
Speaking of which...

I found one with support for the cryptsy exchange, and I pulled in the code to enable cryptsy support
(I have not tested it yet though, so it's nothing I'm promising if/when it will actually work)

Yesterday's prerelease / 2014 July 2nd (initial cryptsy support)

/// (( ... snip ... )) ///

Did you merge the crypsty code into your cex.io fork of gekko or are you keeping them separate?

Yes. The cryptsy code is in my cex.io fork, and you can download it from that link.

Note: sorry about this bad news, but the cryptsy support will probably not be very useful on my fork currently while I'm still fixing the bugs with cex.io reinvest support -- keep in mind that while my fork is still set up for testing on cex.io it has to have the logic for "sell" completely disabled, so you'd only ever see "buy" signals until "sell" is re-enabled.

I don't have a timeframe yet for when I'll be re-enabling the sell logic. Most of the trading methods are still based on old code which is largely untested with my fork, and might still be incompatible or cause issues.


... But you might just want to directly use this one though since it doesn't have "sell" orders disabled:

https://github.com/dowjames/gekko

Sorry, I can't really debug that one since it's too different than my own fork.

Good luck.

[Najska]

Hi!

I don't want to wait for 20 days to have a reliable signal from RSI indicator, so how can I add the historical data on top of the data collected by the bot in its runtime?

Bitcoincharts.com has the historical data, and it's easy to put them in the form of {"s":1239,"o":628.372,"h":628.372,"l":628.372,"c":628.372,"v":0.033,"p":628.372,"_id":"plNRczdJWMI7ds5V"}, but what's "s", can it be negative? And what's _id? And I guess p is the adjusted price. Is it vwap?

Thank you very much in advance

[kuzetsa]

Hi!

I don't want to wait for 20 days to have a reliable signal from RSI indicator, so how can I add the historical data on top of the data collected by the bot in its runtime?

Bitcoincharts.com has the historical data, and it's easy to put them in the form of {"s":1239,"o":628.372,"h":628.372,"l":628.372,"c":628.372,"v":0.033,"p":628.372,"_id":"plNRczdJWMI7ds5V"}, but what's "s", can it be negative? And what's _id? And I guess p is the adjusted price. Is it vwap?

Thank you very much in advance

Code:

{
  start: [moment object],
  o: [open of candle],
  h: [high of candle],
  l: [low of candle],
  c: [close of candle],
  p: [average weighted price of candle],
  v: [volume]
}

That's the exact spec... as for the "_id" it's related to the format used for the database...

The "s" is the "start" time for when that candle opened... It's in "moment" format:

http://momentjs.com/docs/

... and the _id is a fancy key identifier specific to nedb... but I have no clue what format that _id is in.

not sure if that helps you to know gekko stores its data in a "nedb" database:

https://github.com/louischatriot/nedb

(it's an open-source database for node.js, the language gekko is written in)

Changing gekko over to using the nedb database for storing historical data was one of the last things to happen before the original author stopped supporting gekko completely


But I'm just gonna warn you in advance, I think there's a bug which prevents gekko from reliably loading data more than 24 hours old... it doesn't happen every time, but if you ever have to restart gekko there's a chance the history could be invalidated due to corruption and you'll lose all but the most recent 24 hours or so worth od data.

As I understand, the way RSI is traditionally works requires long-term (generally more than 2 days) of data, so you I really hope you're using a fork of gekko which doesn't have issues with database correction...

I really don't test or debug the historical datastore "local DB" system since there's other bugs I'm working on fixing and the database rewrite itself originally took several months to complete, so there's a lot of code involved in figuring out what's wrong

[Najska]

kuzetsa,

Thank you very much! I don't use softwares with lack of developmental support. I didn't know the original author stopped the support. So that means the end of Gekko for me, until maybe one day I want to involve in supporting it. I already have a code running, collecting data and executing orders, but I thought immigrating to Gekko might serve better for future interests.

I use 2 hourly intervals, and RSI(14) needs somewhat 250 candles, which makes about 20 days for a reliable signal.

[jeezy]

Have it running in testmode for the last couple of days, your implemantation of buy&hold on the best spots seems to work fine.

[kuzetsa]

kuzetsa,

Thank you very much! I don't use softwares with *lack* of developmental support. I didn't know the original author stopped the support. So that means the end of Gekko

((... snip ...))

In all fairness, when is the last time satoshi nakamoto did work on bitcoin

Just because the original author left doesn't mean there's a "[total] lack of support"

• https://github.com/kuzetsa/gekko/graphs/commit-activity
• https://github.com/kuzetsa/gekko/graphs/contributors

As I work on it, I try to keep detailed, well-documented commits live on the "master" branch. I rarely make a change which is poorly documented (I include code comments) and the commit messages and comments on each commit has plenty of info as well.

[kuzetsa]

Have it running in testmode for the last couple of days, your implemantation of buy&hold on the best spots seems to work fine.

This morning I've pushed a new prerelease version of the zerolag method which has fewer signals
(tries to only buy once per day, and has better signal filtering to better reject short-term noise)

Now has a "crystalball" parameter because I didn't have a better idea on what else to call it.

Download link: The new prerelease "zerolag" method  ~ ^{today's update [2014 July 5th]}

exact changes to the code
(the "crystal ball" feature itself just lets the signal occur before the EMA crossover itself if momentum is positive)

[kuzetsa]

I really don't test or debug the historical datastore "local DB" system since there's other bugs I'm working on fixing and the database rewrite itself originally took several months to complete, so there's a lot of code involved in figuring out what's wrong

CRITICAL BUGFIX!!!

I took the time to audit the code today since the zerolag method benefits from being initialized with more than one day worth of historical data...

• /commit/4ef18a95f3d508f8e230f7a70520413c5de23369 (the code changes for the actual fix)
  (for those who read code)
  .
• /kuzetsa/gekko/releases/critical.bugfix (download link for the critical bugfix)
• /kuzetsa/gekko/releases/latest ("latest" currently is the same build
  ...and all future releases will still have this bug fixed)

Bug has been fixed since a little before 2014 July 5th, 15:00 (3 PM) Local NY time, but I've been taking time to update documentation and release notes and whatnot.


Edited to add:

In case it wasn't clear why I posted this:

History database has been fixed, and older data now works reliably & no longer gets thrown out because of the bug.
(technically it wasn't the database that was the issue, but the code which loads it at startup)

Easy to fix / easy to spot "off by one" bug which nobody was looking at or trying to fix for the past several months up until I started doing audits recently

[tofm2]

Hi,

I have Gekko running on my linux box. Unfortunately, it is only giving console text output.

I guess I have missed something for enabling its true puwer.

Do I have to coulpe it with a web server or something ??

thanks for your info.

[kuzetsa]

Hi,

I have Gekko running on my linux box. Unfortunately, it is only giving console text output.

I guess I have missed something for enabling its true puwer.

Do I have to coulpe it with a web server or something ??

thanks for your info.

It's NOT a TA tool made for manual, human-powered trading where you look at charts and whatnot...

If you really wanted to you could convert the console output into graphs, or even enable the builtin web module
(it has a web server built-in, but I've never tested it and it's not very well documented.)

I'd recommend using something like ninjatrader if you need a graphical interface with charts.


It was never made for that, and here's a quote from the TL;DR "pretty long"
very first (original) post from page 1 of this thread:

((...snip...))

So Gekko is not

• A fully automated trading bot that you turn on and will generate profit withouth you having to do anything.
• A trading platform for human day traders with a GUI and charts.
• A High frequency trading bot designed to operate on < minute resolution.
• An exchange.

[Hermann1337]

Hi guys!

i installed latest gekko from https://github.com/kuzetsa/gekko on my archlinux on raspberry pi

Working fine so far.

I configured it to catch short term data and show me some advice.

But only output i get is:

2014-07-08 15:31:31 (INFO):     We have all required history, going to calculate
 advice from this point.
2014-07-08 15:32:19 (INFO):     processing 14 trade(s)
2014-07-08 15:33:08 (INFO):     processing 9 trade(s)
2014-07-08 15:33:56 (INFO):     processing 1 trade(s)
2014-07-08 15:34:45 (INFO):     processing 4 trade(s)
2014-07-08 15:35:33 (INFO):     processing 4 trade(s)
2014-07-08 15:36:21 (INFO):     processing 1 trade(s)
2014-07-08 15:37:10 (INFO):     processing 4 trade(s)
2014-07-08 15:37:58 (INFO):     processing 2 trade(s)
2014-07-08 15:38:46 (INFO):     processing 4 trade(s)
2014-07-08 15:39:35 (INFO):     processing 1 trade(s)
2014-07-08 15:42:00 (INFO):     processing 1 trade(s)
2014-07-08 15:42:48 (INFO):     processing 10 trade(s)
2014-07-08 15:43:37 (INFO):     processing 4 trade(s)
2014-07-08 15:44:25 (INFO):     processing 2 trade(s)
2014-07-08 15:45:13 (INFO):     processing 2 trade(s)
2014-07-08 15:46:02 (INFO):     processing 6 trade(s)
2014-07-08 15:46:50 (INFO):     processing 1 trade(s)
2014-07-08 15:48:27 (INFO):     processing 2 trade(s)
2014-07-08 15:49:15 (INFO):     processing 6 trade(s)
2014-07-08 15:50:04 (INFO):     processing 5 trade(s)
2014-07-08 15:50:52 (INFO):     processing 1 trade(s)
2014-07-08 15:54:05 (INFO):     processing 3 trade(s)
2014-07-08 15:54:54 (INFO):     processing 1 trade(s)
2014-07-08 15:55:42 (INFO):     processing 3 trade(s)
2014-07-08 15:56:31 (INFO):     processing 22 trade(s)
2014-07-08 15:57:19 (INFO):     processing 3 trade(s)
2014-07-08 15:58:07 (INFO):     processing 2 trade(s)
2014-07-08 15:58:56 (INFO):     processing 6 trade(s)
2014-07-08 16:00:33 (INFO):     processing 5 trade(s)
2014-07-08 16:01:21 (INFO):     processing 5 trade(s)
2014-07-08 16:02:10 (INFO):     processing 3 trade(s)
2014-07-08 16:02:58 (INFO):     processing 3 trade(s)
2014-07-08 16:03:46 (INFO):     processing 1 trade(s)
2014-07-08 16:04:35 (INFO):     processing 5 trade(s)
2014-07-08 16:05:23 (INFO):     processing 5 trade(s)
2014-07-08 16:06:12 (INFO):     processing 7 trade(s)
2014-07-08 16:07:00 (INFO):     processing 6 trade(s)
2014-07-08 16:07:48 (INFO):     processing 12 trade(s)
2014-07-08 16:08:37 (INFO):     processing 9 trade(s)
2014-07-08 16:09:25 (INFO):     processing 2 trade(s)
2014-07-08 16:10:13 (INFO):     processing 4 trade(s)
2014-07-08 16:11:50 (INFO):     processing 3 trade(s)
2014-07-08 16:12:38 (INFO):     processing 1 trade(s)
2014-07-08 16:13:27 (INFO):     processing 2 trade(s)
2014-07-08 16:15:03 (INFO):     processing 5 trade(s)
2014-07-08 16:16:40 (INFO):     processing 3 trade(s)
2014-07-08 16:17:28 (INFO):     processing 6 trade(s)
2014-07-08 16:18:17 (INFO):     processing 1 trade(s)

shouldnt there be something like short or long advices?


EDIT:

Okay now i have some results

2014-07-08 16:48:56 (INFO):     (PROFIT REPORT) original simulated balance:
 1.16275 BTC
2014-07-08 16:48:56 (INFO):     (PROFIT REPORT) current simulated balance:
 1.16254 BTC
2014-07-08 16:48:56 (INFO):     (PROFIT REPORT) simulated profit:
 -0.00021 BTC (-0.01840%)
2014-07-08 16:49:45 (INFO):     processing 6 trade(s)
2014-07-08 16:51:21 (INFO):     processing 1 trade(s)