Gekko - a javascript trading bot and backtesting platform

[ErebusBat]

whydifficult I realize I may be getting a little obnoxious about this, but I think I've found another bug in the backtester. When putting a unix timecode in the "from:" category the program misinterprets it (I think). The timecode is 1367989200 which should translate to 2013-05-08 05:00:00 but instead is read as 2013-08-16 17:00:00. Doing the same on the "to:" time works completely fine. Obviously this isn't a pressing issue, since I can still just edit the CSV files, it's just one I figured you should know about.

I noticed this too and assumed it had to do with UTC/GMT offsets.

[TradesLikeAPotato]

Strange, I get the same issue by editing the CSV files to only the dates I want, but only when using daily candles... Hourly candles in the same date range work just fine. Here is the candles file.

[alexxy]

Any plans to adding possibility to trade with non btc pairs as well as altcoin exchanges like cryptsy etc?

PS trade data can be downloaded from http://www.cryptocoincharts.info

[whydifficult]

whydifficult I realize I may be getting a little obnoxious about this, but I think I've found another bug in the backtester. When putting a unix timecode in the "from:" category the program misinterprets it (I think). The timecode is 1367989200 which should translate to 2013-05-08 05:00:00 but instead is read as 2013-08-16 17:00:00. Doing the same on the "to:" time works completely fine. Obviously this isn't a pressing issue, since I can still just edit the CSV files, it's just one I figured you should know about.

I noticed this too and assumed it had to do with UTC/GMT offsets.

Hmm that's very strange, might be the offset but I'll try to verify this. I don't mind at all, please let me know everything you find. I am keeping a list of stuff I need todo for Gekko.

Unfortunatly I have very little spare time right now, as soon as I have some I'll work on some issues with Gekko!

Strange, I get the same issue by editing the CSV files to only the dates I want, but only when using daily candles... Hourly candles in the same date range work just fine. Here is the candles file.

noted, will look into this.

Any plans to adding possibility to trade with non btc pairs as well as altcoin exchanges like cryptsy etc?

Yep, I am working on offering all required trade data for this soon.

---

My current situation offers me very little spare time to work on Gekko, as soon as I have some I will.

I have noted all issues and when I have some spare time I'll look into everything. In the meantime Gekko is open to pull requests, and when you find a bug it would be great if you could describe it as clear as possible, thanks!

I am working on a platform that provides market data for a lot of different exchanges, the reason I have some time to work on this and less on Gekko is because Wizbit is going to be part of my graduation project, which means I can do this in university hours so to say (but I don't have a lot of time to work on this either).

[TradesLikeAPotato]

It is indeed the offset. Subtracting 100 days from the date solved the problem, I just missed it on hourly because 4 days is less noticeable than 100. I've come across something else as well. Watching the market with BTCe EMAs 19/45 with an offset of 100 leads to a "Failed to load historical trades from bitcoincharts." Reducing the offset to 70 solves the problem, but causes inaccurate EMAs. It may be due to to the missing candles on bitcoincharts on the 26th of August.

[TradesLikeAPotato]

I now get the error using any offset on BTCe.

EDIT: Read the other parts of the thread, it is a known issue.

[ewibit]

I now get the error using any offset on BTCe.

here the same

Code:

gekko_BTC-e/methods/realtime-candle-fetcher.js:98
    throw 'Failed to load historical trades from ' + this.watcher.name;
                                                   ^
Failed to load historical trades from bitcoincharts

[limit]

Hi there,
trying to install, when i exeecute "nmp install" I get the following error:

http 404 https://github.com/titaniumlou/emailjs/tarball/patch-1


Thanks

[kenshin23]

Something odd happened today: I had Gekko running in the background for like a day or so (trading in MtGox), but suddenly I got a signal to sell, even though the chart on bitcoincharts didn't give any indication to do so. When I closed and reopened it, I got things like this:

Code:

2013-09-11 00:18:56 (INFO):     ADVICE is to HOLD @ 129.043 (NaN)

Ran it again with config.debug set:

Code:

...
2013-09-11 00:24:24 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:24 (DEBUG):             short ema 133.005
2013-09-11 00:24:24 (DEBUG):             long ema 132.115
2013-09-11 00:24:24 (DEBUG):             diff ema 0.671
2013-09-11 00:24:24 (DEBUG):    calculated candle: 11
2013-09-11 00:24:24 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:24 (DEBUG):             short ema NaN
2013-09-11 00:24:24 (DEBUG):             long ema NaN
2013-09-11 00:24:24 (DEBUG):             diff ema NaN
2013-09-11 00:24:24 (DEBUG):    calculated candle: 10
2013-09-11 00:24:24 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:24 (DEBUG):             short ema NaN
2013-09-11 00:24:24 (DEBUG):             long ema NaN
2013-09-11 00:24:24 (DEBUG):             diff ema NaN
2013-09-11 00:24:24 (DEBUG):    need new trades, refetching Mt. Gox
2013-09-11 00:24:26 (DEBUG):    calculated candle: 9
2013-09-11 00:24:26 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:26 (DEBUG):             short ema NaN
2013-09-11 00:24:26 (DEBUG):             long ema NaN
2013-09-11 00:24:26 (DEBUG):             diff ema NaN
2013-09-11 00:24:26 (DEBUG):    calculated candle: 8
2013-09-11 00:24:26 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:26 (DEBUG):             short ema NaN
2013-09-11 00:24:26 (DEBUG):             long ema NaN
2013-09-11 00:24:26 (DEBUG):             diff ema NaN
2013-09-11 00:24:26 (DEBUG):    calculated candle: 7
2013-09-11 00:24:26 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:26 (DEBUG):             short ema NaN
2013-09-11 00:24:26 (DEBUG):             long ema NaN
2013-09-11 00:24:26 (DEBUG):             diff ema NaN
2013-09-11 00:24:27 (DEBUG):    need new trades, refetching Mt. Gox
2013-09-11 00:24:31 (DEBUG):    calculated candle: 6
2013-09-11 00:24:31 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:31 (DEBUG):             short ema NaN
2013-09-11 00:24:31 (DEBUG):             long ema NaN
2013-09-11 00:24:31 (DEBUG):             diff ema NaN
2013-09-11 00:24:31 (DEBUG):    calculated candle: 5
2013-09-11 00:24:31 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:31 (DEBUG):             short ema NaN
2013-09-11 00:24:31 (DEBUG):             long ema NaN
2013-09-11 00:24:31 (DEBUG):             diff ema NaN
2013-09-11 00:24:31 (DEBUG):    calculated candle: 4
2013-09-11 00:24:31 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:31 (DEBUG):             short ema NaN
2013-09-11 00:24:31 (DEBUG):             long ema NaN
2013-09-11 00:24:31 (DEBUG):             diff ema NaN
2013-09-11 00:24:31 (DEBUG):    calculated candle: 3
2013-09-11 00:24:31 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:31 (DEBUG):             short ema NaN
2013-09-11 00:24:31 (DEBUG):             long ema NaN
2013-09-11 00:24:31 (DEBUG):             diff ema NaN
2013-09-11 00:24:31 (DEBUG):    calculated candle: 2
2013-09-11 00:24:31 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:31 (DEBUG):             short ema NaN
2013-09-11 00:24:31 (DEBUG):             long ema NaN
2013-09-11 00:24:31 (DEBUG):             diff ema NaN
2013-09-11 00:24:31 (DEBUG):    need new trades, refetching Mt. Gox
2013-09-11 00:24:35 (DEBUG):    calculated candle: 1
2013-09-11 00:24:35 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:24:35 (DEBUG):             short ema NaN
2013-09-11 00:24:35 (DEBUG):             long ema NaN
2013-09-11 00:24:35 (DEBUG):             diff ema NaN
2013-09-11 00:24:35 (DEBUG):    fetching new trades for new bucket at Mt. Gox
2013-09-11 00:25:20 (DEBUG):    calculated candle: 0
2013-09-11 00:25:20 (DEBUG):    calced EMA properties for new candle:
2013-09-11 00:25:20 (DEBUG):             short ema NaN
2013-09-11 00:25:20 (DEBUG):             long ema NaN
2013-09-11 00:25:20 (DEBUG):             diff ema NaN
2013-09-11 00:25:20 (DEBUG):    we are currently not in an up or down trend @ 128.933 (NaN)
2013-09-11 00:25:20 (INFO):     ADVICE is to HOLD @ 128.933 (NaN)

Anyone else having issues? I had been running Gekko for almost a month with no problems until today.

P.S.: My timezone is GMT -4:30, in case you're wondering.

[avert]

Hi there,
trying to install, when i exeecute "nmp install" I get the following error:

http 404 https://github.com/titaniumlou/emailjs/tarball/patch-1


Thanks

in package.json

change where emailjs is retrieved from

Code:

"emailjs": "git://github.com/eleith/emailjs.git#master", 

[avert]

Something odd happened today: I had Gekko running in the background for like a day or so (trading in MtGox), but suddenly I got a signal to sell, even though the chart on bitcoincharts didn't give any indication to do so. When I closed and reopened it, I got things like this:

Code:

2013-09-11 00:18:56 (INFO):     ADVICE is to HOLD @ 129.043 (NaN)

Ran it again with config.debug set:

...

gox was down for a bit, so it seems to be its trying to calculate EMA's on times where there is just no data. So number * NaN = NaN right... ( just like 234234*0=0). This will happen until the candle gets pushed out of the array i guess.

I think i have a fix which sorts this out, but saying this, i dont know gekko in its entirety at all. And these could have some adverse effects.

in methods/realtime-candle-fetcher.js
line 171: calculateCandle function

Code:

CandleCalculator.prototype.calculateCandle = function() {
  var bucket = this.buckets[this.currentBucket];
  // because buckets (and their contents) are chronologically reversed
  // the _last_ item is the open and the _first_ is the close

if(typeof _.last(bucket) == 'undefined')
        return false;
  if(typeof _.max(bucket) == 'undefined')
        return false;
  if(typeof _.min(bucket) == 'undefined')
       return false;
  if(typeof _.first(bucket) == 'undefined')
        return false;

  this.candles.open.push(_.last(bucket));
  this.candles.high.push(_.max(bucket));
  this.candles.low.push(_.min(bucket));
  this.candles.close.push(_.first(bucket));

  log.debug('calculated candle:', this.currentBucket);

  this.emit('calculated candle');

  if(this.currentBucket === 0)
    this.emit('calculated new candle');
}

I added the sexy if statements to skip over underfined values.
but as i said, these could have some bad effects. I've only been using gekko for about 4 days and am no way in a position to predict what this would do:P

[Mogumodz]

Something odd happened today: I had Gekko running in the background for like a day or so (trading in MtGox), but suddenly I got a signal to sell, even though the chart on bitcoincharts didn't give any indication to do so. When I closed and reopened it, I got things like this:

Code:

2013-09-11 00:18:56 (INFO):     ADVICE is to HOLD @ 129.043 (NaN)

Ran it again with config.debug set:

....

I had the same thing with NaN appearing after the data from MtGox was interrupted, after adding the if statements it appears I'm only missing two candles (20 & 21) when running with debug set to true. Better than waiting until the candles are out of the array as previously stated.



It's now back to calculating the EMA from the available data coming in.

[kenshin23]

Added the if statements to the code, and now it seems to be working again. Thanks!

I'll have to wait until the next sell signal to start trading again, though.

[ewibit]

I always get

Code:

methods/realtime-candle-fetcher.js:98
    throw 'Failed to load historical trades from ' + this.watcher.name;
                                                   ^
Failed to load historical trades from bitcoincharts

from BTC-e and Bitstamp

[play]

I always get

Code:

methods/realtime-candle-fetcher.js:98
    throw 'Failed to load historical trades from ' + this.watcher.name;
                                                   ^
Failed to load historical trades from bitcoincharts

from BTC-e and Bitstamp

Any updates on this?

[karmazynt]

in javascript you don't have to write:
if (typeof _.first(bucket) == 'undefined')

you just could write:
if (_.first(bucket))

but this will catch also null values and other falsy values, it can be good thing, it depends.
If want only check for undefinied you can write:
if (_.first(bucket) === undefined)

[MCT]

Hi gekko fans!

This program could be awesome. But recently i only get

Code:

Failed to load historical trades from bitcoincharts

.

I already tried to change the api link to the new one http://api.bitcoincharts.com/v1/ but it keeps failing. I spend some hours on debugging via node-inspector and i think it has something to do with the realtime candle fetcher and the following lines:

Code:

// fill buckets based on the trade data.
CandleCalculator.prototype.fillBuckets = function(trades) {
  var latestTradeDate;
  var startBucket = this.currentBucket;

  var emptyBucket = true;
  var nextBucketTime = util.intervalsAgo(this.currentBucket);
  _.every(trades, function(trade) {
    var time = moment.unix(trade.date);
    // as long as this trade is to recent
    // it should go to the next bucket
    while(time > nextBucketTime) {
      if(emptyBucket === true)
        return false;

Actually im neither good in developing nor familiar with javascript.

But i would really appreciate a fix on that.

Please excuse my rather bad english.

[MCT]

Well,

I think I got it.

First of all you have to change the source where email.js is retrieved:
/gekko-master/package.json

Code:

"emailjs": "0.3.x",

After this it will successfully install via npm.

Step 1 Change Bitcoincharts Api Link:
/gekko-master/node-modules/bitcoincharts/bitcoincharts.js

Code:

/**
 * The client for bitcoincharts.com API.
 *
 * @constructor
 */
var BitcoinCharts = function() {
  this.url = "http://api.bitcoincharts.com/v1/";
};

Step 2 Change reversal of trades.csv:
/gekko-master/exchanges/bitcoincharts.js

Code:

Watcher.prototype.getTrades = function(since, callback, descending) {
  var params = { symbol: this.symbol };
  if(since)
    // we don't want to hammer bitcoincharts,
    // this will fetch trades between start and now
    params.start = since.format('X');

  var args = _.toArray(arguments);
  this.bitcoinCharts.trades(params, _.bind(function(err, data) {
    if(err)
      return this.retry(this.getTrades, args);

    if(!data || !data.length)
      return this.retry(this.getTrades, args);

    // normalize the data
    var trades = [];
    _.each(data, function(array) {
      trades.push({
        date: array[0],
        price: array[1],
        amount: array[2]
      });
    });

   if(descending)
      callback(trades.reverse());
    else
      callback(trades);

  }, this));
}

Step 3 Change how the latest trades are fetched:
/gekko-master/methods/realtime-candle-fetcher.js

Code:

// this method fetches most recent trades and calculates a new candle based on the trades
CandleCalculator.prototype.getNewCandle = function() {

  if(this.fetchingHistorical)
    // we just fetched all historical buckets
    this.fetchingHistorical = false;
  else
    this.removeOldestBucket();

  this.removeOldestCandle();
  
  log.debug('fetching new trades for new bucket at', this.watcher.name);
  this.watcher.getTrades(util.intervalsAgo(1), this.fillNewBucket, true);
}

Its a quite simple workaround and of course not that beautiful but fetching trades from Bitcoincharts should be fixed then.
But it would be nice if the author or any other trusted programmer could test and validate those changes.

Have a nice weekend gekkos.

[whydifficult]

I am back guys.

I fixed the email & underscore errors: pull the latest version and `npm install` to get it working.

(Thanks avert for the email fix suggestion: implemented it the same way. For underscore: they removed a function I was relying on so I am now shifting towards lodash)

I'll check the code fixes posted asap and merge it in. Remember guys: Pull requests are always welcome!

[TradesLikeAPotato]

I am back guys.

I fixed the email & underscore errors: pull the latest version and `npm install` to get it working.

(Thanks avert for the email fix suggestion: implemented it the same way. For underscore: they removed a function I was relying on so I am now shifting towards lodash)

I'll check the code fixes posted asap and merge it in. Remember guys: Pull requests are always welcome!

'npm install' errors out on me.

C:\Users\Mark\Desktop\gekko-master>npm install
npm http GET https://registry.npmjs.org/moment
npm http GET https://registry.npmjs.org/bitcoincharts
npm http GET https://registry.npmjs.org/btc-e
npm http GET https://registry.npmjs.org/mtgox-apiv2
npm http GET https://registry.npmjs.org/prompt-lite
npm http GET https://registry.npmjs.org/lodash
npm http GET https://registry.npmjs.org/line-reader
npm http GET https://registry.npmjs.org/async
npm http GET https://registry.npmjs.org/bitstamp/0.0.4
npm ERR! not found: git
npm ERR!
npm ERR! Failed using git.
npm ERR! This is most likely not a problem with npm itself.
npm ERR! Please check if you have git installed and in your PATH.

npm ERR! System Windows_NT 6.2.9200
npm ERR! command "C:\\Program Files\\nodejs\\\\node.exe" "C:\\Program Files\\nod
ejs\\node_modules\\npm\\bin\\npm-cli.js" "install"
npm ERR! cwd C:\Users\Mark\Desktop\gekko-master
npm ERR! node -v v0.10.18
npm ERR! npm -v 1.3.8
npm ERR! code ENOGIT
npm http 304 https://registry.npmjs.org/mtgox-apiv2
npm http 304 https://registry.npmjs.org/bitcoincharts
npm http 304 https://registry.npmjs.org/prompt-lite
npm http 304 https://registry.npmjs.org/lodash
npm http 304 https://registry.npmjs.org/line-reader
npm http 304 https://registry.npmjs.org/btc-e
npm http 304 https://registry.npmjs.org/async
npm http 304 https://registry.npmjs.org/moment
npm http 304 https://registry.npmjs.org/bitstamp/0.0.4
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     C:\Users\Mark\Desktop\gekko-master\npm-debug.log
npm ERR! not ok code 0

EDIT: Fixed by installing git.