Gekko - a javascript trading bot and backtesting platform

[gambitv]

I think rikigst theory looks great. Be fascinating to get this working. I like the idea of a gradual buy or sell. The chrome bot also allows a 'reserve' to be kept safe. I think that could be useful to. I think a lot of us see bitcoin having greater value over time. So I aim to 'build' my reserve, and only trade a percentage.

Besides, you need bitcoins to buy things only for sale in bitcoins...like some of the shares available that give a great return compared to poor mans GPU mining on their gaming rig.

[whydifficult]

Very interesting project, I'm testing it now, my thoughts about future development:

- please take a look at TA-lib for node: https://npmjs.org/package/talib - using those libraries may speed up development of other useful methods
- before that, it would be good to decide how Gekko will deal with multiple methods and possibly contradictory suggestions. Some kind of voting mechanism with user-defined weights? Neural network? Other? Maybe suggestion should look like: 10% SELL, 70% KEEP, 20% BUY - SO KEEP.
- how about other input data? There's a lot of possible input data sources, other than exchange rate, eg. stock market indexes, google trends data for some keywords (related with bitcoin, world economy etc.) which can be correlated with btc future price trends. Will you propose some unified structure for such data to be shared possibly between different methods, or specific method will have to do it for it's own use?

I'd be very happy to contribute to the project, however I'm not a js programmer (but I'm trying to get a little bit more familliar with it). Currently I could share my debian scripts for running gekko daemon using node's "forever" module. Let me know if you're interested.

Thanks, let me know how the testing goes. Right now Gekko is just doing the really basic stuff (only look at price data of one single exchange, use a simple method like EMA and switch the full balance towards the detected trend). I am in the process of making Gekko more and more advanced.

I didn't know about TA-lib. It would indeed speed up Gekko's development by a lot of things if I can just plug it into Gekko! I've done a quick test and the installation fails on my laptop (OS X) and my vps (Linux), and I don't want to try windows. (I need to do some more research: not running latest node etc). So if I can get this working I think this is going to be optional in the settings.

I don't have a background in trading so I'm open in ideas about contradictory suggestions by different methods (for as soon as we have different methods). The user defined weights would be the most straightforward option. I would have to do more reading about neural networks before I can implement such a system into Gekko (if you know about it and can point me to some reading that would be great).

Your last point is something Gekko could definitely implement in the future, however this is much harder than applying basic math to current price numbers. So I first want to have a solid bot before I enter this kind of field. But we can of course already start thinking about what good indicators could be, I personally doubt a correlation between Bitcoin news coverage and the price (for example) at the moment, but this would all have to be backtested.

I'm getting better suggestions for new trade methods for Gekko to implement, so backtesting is a really big point on my list. I am currently figuring out if I can use the Mt. Gox data downloader provided by nitrous. This will download every trade from Mt. Gox and stuff it into a SQLite database, something Gekko could very easily use to extract data from.

I think rikigst theory looks great. Be fascinating to get this working. I like the idea of a gradual buy or sell. The chrome bot also allows a 'reserve' to be kept safe. I think that could be useful to. I think a lot of us see bitcoin having greater value over time. So I aim to 'build' my reserve, and only trade a percentage.

Besides, you need bitcoins to buy things only for sale in bitcoins...like some of the shares available that give a great return compared to poor mans GPU mining on their gaming rig.

The reserve part was pointed out to me yesterday as well and is on my current list of todos. Even though I don't want to force everyone into keeping / managing his bitcoin in a certain way, I do think it is best practise to not keep all of your BTC at an exchange.

My current todo list:

• Risk percentage (how much % can Gekko play with)
• Better profit logging
• Backtesting
• Solid way to implement trading methods (it is not possible to trade 10% of allowed balance for example)
• MACD trading method

[gambitv]

Out of curiosity, did anyone loose a bit on the recent drop in price. I had gekko running on default settings in 'simulation' mode for 12 hours before the drop. Got no advice except one emailed to me - Sell at bottom price. LoL.  The drop was so fast that by the time the bot got 3 confirmations it was all over. Well, not completely, it did drop another 5 dollars usd on the 120 price shortly after that, then climbed back up to 125.
Right this moment hovering around 118. I still believe it is only going to get stronger. It is picking up real momentum and the only way it is going to fail is something better comes along ( ie government publishes their own official crypto-world-currency ), or it is somehow taxed to oblivion at the exchanges.

[San1ty]

Out of curiosity, did anyone loose a bit on the recent drop in price. I had gekko running on default settings in 'simulation' mode for 12 hours before the drop. Got no advice except one emailed to me - Sell at bottom price. LoL.  The drop was so fast that by the time the bot got 3 confirmations it was all over. Well, not completely, it did drop another 5 dollars usd on the 120 price shortly after that, then climbed back up to 125.
Right this moment hovering around 118. I still believe it is only going to get stronger. It is picking up real momentum and the only way it is going to fail is something better comes along ( ie government publishes their own official crypto-world-currency ), or it is somehow taxed to oblivion at the exchanges.

That is pretty obvious as EMA crossovers are always lagging behind price. They are good for confirming trends, but they will never pick up a spike up or down.

[whydifficult]

Out of curiosity, did anyone loose a bit on the recent drop in price. I had gekko running on default settings in 'simulation' mode for 12 hours before the drop. Got no advice except one emailed to me - Sell at bottom price. LoL.  The drop was so fast that by the time the bot got 3 confirmations it was all over. Well, not completely, it did drop another 5 dollars usd on the 120 price shortly after that, then climbed back up to 125.
Right this moment hovering around 118. I still believe it is only going to get stronger. It is picking up real momentum and the only way it is going to fail is something better comes along ( ie government publishes their own official crypto-world-currency ), or it is somehow taxed to oblivion at the exchanges.

Gekko right now does not use the three thresholds to verify the trend. I think I should better clarify this in the readme.

Here is the data from my Gekko trading at Mt. Gox (price is in euros, time is in UTC: right now it's 19:10):

Code:

mike:~/nodeprojects/gekko-stuff$ cat gekko.log | grep SELL
2013-05-31 00:59:49 (INFO):	ADVICE is to SELL @ 99.605 (-0.288)
2013-05-31 01:00:20 (INFO):	SELL was succesfull

Code:

mike:~/nodeprojects/gekko-stuff$ tail gekko.log
2013-06-03 18:59:55 (INFO):	ADVICE is to HOLD @ 93.763 (-0.275)
2013-06-03 18:59:55 (INFO):	(PROFIT REPORT) -0.267% profit (in 1 trades)

EMA is not well suited for detecting this kind of sudden movements, because it takes time to verify that the trend is really happening. More trading methods will be added in the future!

[ewibit]

Code:

npm http 200 https://registry.npmjs.org/read
npm ERR! error installing prompt-lite@0.1.1

npm ERR! Error: No compatible version found: read@'>=1.0.0- <1.1.0-'
npm ERR! Valid install targets:
npm ERR! ["0.0.1","0.0.2","0.0.3","0.1.0","0.1.1"]
npm ERR!     at installTargetsError (/usr/share/npm/lib/cache.js:488:10)
npm ERR!     at next_ (/usr/share/npm/lib/cache.js:438:17)
npm ERR!     at next (/usr/share/npm/lib/cache.js:415:44)
npm ERR!     at /usr/share/npm/lib/cache.js:408:5
npm ERR!     at saved (/usr/share/npm/lib/utils/npm-registry-client/get.js:147:7)
npm ERR!     at Object.oncomplete (/usr/lib/nodejs/graceful-fs.js:230:7)
npm ERR! You may report this log at:
npm ERR!     <http://bugs.debian.org/npm>
npm ERR! or use
npm ERR!     reportbug --attach /home/xxx/gekko/npm-debug.log npm
npm ERR! 
npm ERR! System Linux 3.8.0-22-generic
npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "update"
npm ERR! cwd /home/xxx/gekko
npm ERR! node -v v0.6.19
npm ERR! npm -v 1.1.4
npm ERR! message No compatible version found: read@'>=1.0.0- <1.1.0-'
npm ERR! message Valid install targets:
npm ERR! message ["0.0.1","0.0.2","0.0.3","0.1.0","0.1.1"]
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/xxx/gekko/npm-debug.log
npm not ok

[whydifficult]

Code:

npm http 200 https://registry.npmjs.org/read
npm ERR! error installing prompt-lite@0.1.1

npm ERR! Error: No compatible version found: read@'>=1.0.0- <1.1.0-'
npm ERR! Valid install targets:
npm ERR! ["0.0.1","0.0.2","0.0.3","0.1.0","0.1.1"]
npm ERR!     at installTargetsError (/usr/share/npm/lib/cache.js:488:10)
npm ERR!     at next_ (/usr/share/npm/lib/cache.js:438:17)
npm ERR!     at next (/usr/share/npm/lib/cache.js:415:44)
npm ERR!     at /usr/share/npm/lib/cache.js:408:5
npm ERR!     at saved (/usr/share/npm/lib/utils/npm-registry-client/get.js:147:7)
npm ERR!     at Object.oncomplete (/usr/lib/nodejs/graceful-fs.js:230:7)
npm ERR! You may report this log at:
npm ERR!     <http://bugs.debian.org/npm>
npm ERR! or use
npm ERR!     reportbug --attach /home/xxx/gekko/npm-debug.log npm
npm ERR! 
npm ERR! System Linux 3.8.0-22-generic
npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "update"
npm ERR! cwd /home/xxx/gekko
npm ERR! node -v v0.6.19
npm ERR! npm -v 1.1.4
npm ERR! message No compatible version found: read@'>=1.0.0- <1.1.0-'
npm ERR! message Valid install targets:
npm ERR! message ["0.0.1","0.0.2","0.0.3","0.1.0","0.1.1"]
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/xxx/gekko/npm-debug.log
npm not ok

It is still complaining about the dependencies requiring a newer version of node. Your version is almost a year old (from june 2012). Did you catch my response last time?

Your node version is out of date, the Bitstamp module doesn't really require node 0.10 but other dependencies do. Could you try to update to 0.10.x?

You can check what node version is installed by running 'node -v' in a terminal. Here is a guide that will show you how to install node 0.10.x on a Raspberry PI.

EDIT: I have updated the profit reporting of Gekko, it now looks like this:

Code:

2013-06-04 00:02:30 (INFO):	ADVICE is to HOLD @ 120.750 (0.861)
2013-06-04 00:02:30 (INFO):	(PROFIT REPORT) original simulated balance:	 222.430 USD
2013-06-04 00:02:30 (INFO):	(PROFIT REPORT) current simulated balance:	 219.378 USD
2013-06-04 00:02:30 (INFO):	(PROFIT REPORT) simulated profit:		 -3.052 USD (-1.372%)

[Simonsays]

EDIT: I have updated the profit reporting of Gekko, it now looks like this:

Code:

2013-06-04 00:02:30 (INFO):	ADVICE is to HOLD @ 120.750 (0.861)
2013-06-04 00:02:30 (INFO):	(PROFIT REPORT) original simulated balance:	 222.430 USD
2013-06-04 00:02:30 (INFO):	(PROFIT REPORT) current simulated balance:	 219.378 USD
2013-06-04 00:02:30 (INFO):	(PROFIT REPORT) simulated profit:		 -3.052 USD (-1.372%)

Liking the new look! 

[whydifficult]

Thanks, note that is still doesn't look at real balance & profit but still simulates trades on every advice. This will get updated soon though.

---

Good news for all Bitstamp traders out there: Right now you need to hand over your login credentials because their API doesn't use API keys with restricted rights (like only acces to ask balance and create orders). I filled a ticket for this and just got a new response:

Dear Mike,

We are working on changing our API access at the moment.

This means that in the future you won't have to give a Bitstamp trade bot (Gekko or other) your full credentials.

[San1ty]

Thanks, note that is still doesn't look at real balance & profit but still simulates trades on every advice. This will get updated soon though.

---

Good news for all Bitstamp traders out there: Right now you need to hand over your login credentials because their API doesn't use API keys with restricted rights (like only acces to ask balance and create orders). I filled a ticket for this and just got a new response:

Dear Mike,

We are working on changing our API access at the moment.

This means that in the future you won't have to give a Bitstamp trade bot (Gekko or other) your full credentials.

Good Job! Also don't forget my remark on candlesticks, it would be good practice to save the open, high, low, close per interval instead of only the close you have now.

[oper128]

Thanks, let me know how the testing goes. Right now Gekko is just doing the really basic stuff (only look at price data of one single exchange, use a simple method like EMA and switch the full balance towards the detected trend). I am in the process of making Gekko more and more advanced.

Works stable, but log format changes too frequently ;-) BTW I've made some visualisations from log data for easier analysis. Currently after 1 day 6 hours time and 4 transactions I have -0.28% profit (-1.34 to 3.68 during that time) with market being ~ +1%. Not so bad ;-)

I didn't know about TA-lib. It would indeed speed up Gekko's development by a lot of things if I can just plug it into Gekko! I've done a quick test and the installation fails on my laptop (OS X) and my vps (Linux), and I don't want to try windows. (I need to do some more research: not running latest node etc). So if I can get this working I think this is going to be optional in the settings.

Installed flawlessly on my debian wheezy box with latest node. Let me know if I could help somehow.

I don't have a background in trading so I'm open in ideas about contradictory suggestions by different methods (for as soon as we have different methods). The user defined weights would be the most straightforward option. I would have to do more reading about neural networks before I can implement such a system into Gekko (if you know about it and can point me to some reading that would be great).

Simple voting should do the job for now. With neural networks (what amazes me there's again a ready to use node library https://npmjs.org/package/brain) you'll have to teach it with history as input (price and all methods) and deserved output (buy/sell) - this part may be the hardest job to do.

Your last point is something Gekko could definitely implement in the future, however this is much harder than applying basic math to current price numbers. So I first want to have a solid bot before I enter this kind of field. But we can of course already start thinking about what good indicators could be, I personally doubt a correlation between Bitcoin news coverage and the price (for example) at the moment, but this would all have to be backtested.

OK, keep it simple and working now, there'll be time for other improvements.

I'm getting better suggestions for new trade methods for Gekko to implement, so backtesting is a really big point on my list. I am currently figuring out if I can use the Mt. Gox data downloader provided by nitrous. This will download every trade from Mt. Gox and stuff it into a SQLite database, something Gekko could very easily use to extract data from.

My current todo list:

• Risk percentage (how much % can Gekko play with)
• Better profit logging
• Backtesting
• Solid way to implement trading methods (it is not possible to trade 10% of allowed balance for example)
• MACD trading method

+1 for backtesting and MACD, but use mysql or postgres as database engine if possible.

So it's time to pull latest version and see what happens

[whydifficult]

Good Job! Also don't forget my remark on candlesticks, it would be good practice to save the open, high, low, close per interval instead of only the close you have now.

Will do, if something like talib (read below) will not become the standard I think I will abstract out a candle calculator based on current data fetcher so that trading methods can just request candle data from exchanges, that way I can do stuff like that way more easily.

Works stable, but log format changes too frequently ;-) BTW I've made some visualisations from log data for easier analysis. Currently after 1 day 6 hours time and 4 transactions I have -0.28% profit (-1.34 to 3.68 during that time) with market being ~ +1%. Not so bad ;-)

Awesome! I'm curious about the visualisations.

Installed flawlessly on my debian wheezy box with latest node. Let me know if I could help somehow.

I still haven't had the time to look into it, but this is what I'm getting on my mac:

Code:

building talib functions...
  CXX(target) Release/obj.target/talib/src/talib.o
../src/talib.cpp:694:9: error: no matching function for call to 'uv_queue_work'
        uv_queue_work(uv_default_loop(), req, ExecuteWork, ExecuteWorkDone);
        ^~~~~~~~~~~~~
/Users/mike/.node-gyp/0.9.3/deps/uv/include/uv.h:1355:15: note: candidate function not viable:
      no known conversion from 'void (uv_work_t *, int)' to 'uv_after_work_cb'
      (aka 'void (*)(uv_work_t *)') for 4th argument;
UV_EXTERN int uv_queue_work(uv_loop_t* loop, uv_work_t* req,

(and I think I got the same on my ubuntu system, not 100% sure)

Simple voting should do the job for now. With neural networks (what amazes me there's again a ready to use node library https://npmjs.org/package/brain) you'll have to teach it with history as input (price and all methods) and deserved output (buy/sell) - this part may be the hardest job to do.

Woow that looks awesome, when I have the time I definitely will look into it!

+1 for backtesting and MACD, but use mysql or postgres as database engine if possible.

I prefer to keep Gekko as cross platform as possible and without a big dependency tree. Someone recommended levelDB as it's a lightweight datastore for node (without external dependencies). But I will have to make a list of what data I want to keep stored, etc before I'll choose a database.

[oper128]

whydifficult, your gcc version may be too old or too new 4.7.2 works for me.

Edit: google says it might be a problem with node v0.10.0, I use v0.10.8 now.

[whydifficult]

whydifficult, your gcc version may be too old or too new 4.7.2 works for me.

Edit: google says it might be a problem with node v0.10.0, I use v0.10.8 now.

Apparently it was caused by my old version of node, after updating everything works fine. I am now rewriting some things so that Gekko will internally calculate candles based on trade data so that implementing different methods (as well as Talib) will be easy as pie. I do think talib will become optional since it doesn't work on Windows.

Edit: when Talib makes it into Gekko, Gekko will support a lot of trading methods. These methods include:

• Exponential Moving Average
• Moving Average Convergence/Divergence
• Double Exponential Moving Average
• Triple Exponential Moving Average
• Bollinger Bands
• Relative Strength Index
• Directional Movement Index

The full list of all 138 supported methods can be found here.

[rikigst]

whydifficult, your gcc version may be too old or too new 4.7.2 works for me.

Edit: google says it might be a problem with node v0.10.0, I use v0.10.8 now.

Apparently it was caused by my old version of node, after updating everything works fine. I am now rewriting some things so that Gekko will internally calculate candles based on trade data so that implementing different methods (as well as Talib) will be easy as pie. I do think talib will become optional since it doesn't work on Windows.

Edit: when Talib makes it into Gekko, Gekko will support a lot of trading methods. These methods include:

• Exponential Moving Average
• Moving Average Convergence/Divergence
• Double Exponential Moving Average
• Triple Exponential Moving Average
• Bollinger Bands
• Relative Strength Index
• Directional Movement Index

The full list of all 138 supported methods can be found here.

Great! Do you plan to implement backtesting before TAlib? Since there are so many strategies, one could build his own, and test it on past data

[whydifficult]

Great! Do you plan to implement backtesting before TAlib? Since there are so many strategies, one could build his own, and test it on past data

I don't know the order at which I will implement the features (depends on a couple of things), but that's the main idea. Would be great if everyone could very easily backtest using all different algos with different settings. The next step would be to implement something like a neural network (or more simply multiple algo's at the same time with weight) as suggested by oper128.

At this moment Gekko's code is not really optimised for speed which is important when creating backtesting (though when using talib a lot of calculations are done by the low level library already).

---

I am changing some internals in Gekko so if someone is working on a new trading method watch the repo closely.

[whydifficult]

I shifted some things in the config. It should now be a lot easier to set Gekko up for a normal use case (set an exchange and fill in the keys if you want Gekko to trade).

If you want to do some experimental stuff like watching trends at Mt. Gox BTC/USD and acting (buying / selling) on BTC-e BTC/RUR than you have to check out the advanced section.

Everything in the config is now explained in the improved documentation.

[ewibit]

I have run it now ~1 week and now updated from git; - in this week:

Code:

2013-06-07 21:08:05 (INFO):	(PROFIT REPORT) profit:			 -4.698 USD (-2.149%)

[spartacus_]

I have run it now ~1 week and now updated from git; - in this week:

Code:

2013-06-07 21:08:05 (INFO):	(PROFIT REPORT) profit:			 -4.698 USD (-2.149%)

There is no magical program to win.

[whydifficult]

I have run it now ~1 week and now updated from git; - in this week:

Code:

2013-06-07 21:08:05 (INFO):	(PROFIT REPORT) profit:			 -4.698 USD (-2.149%)

Well the price just sank at the end of last week and over the weekend, not much any bot could have done I think. Keep in mind that a price drop is hurting everyone who is holding all their bitcoins. Most of  my Gekko instances (on Mt. Gox) suggested a buy a couple of hours ago.