ncurses based MtGox live monitor and trading-bot-framework

[hugolp]

can anyone help me with coding a strategy for this tool...

I need to calc the EMAs and want to use the differance in my stratagy...
I have no clue atm how to do this....

Im looking to create a macd stratagy on the dema from mtgox

Pm me pls, as I dont want to derail the thread, thx

MtQuid already showed how to include ta-lib in a strategy. Look early pages in this thread.

[1715594609]

1715594609

[1715594609]

1715594609

[1715594609]

1715594609

[evilscoop]

ahh thx ill look into it

[hugolp]

I hit a problem with my bot and its that the last days the number of trades has increased a lot at mtgox so downloading two days of data everytime I load the bot takes for ever. So Ive been programming a way to save the trades in a database. It obviously saves the trades in the database, but also on load my strategy checks the first received trade and the oldest trade in the database and downloads the trades missing between them so the history is always complete. The idea is to use it for backtesting as well. If anyone is interested I can share the code.

Also, with more datapoints the hourly ema formula gives almost exacts results to bitcoincharts (as suspected) so thats good.

[FoxMURDER]

I never liked how depth/tick messages keep flooding the console window. At first I was disabling those. But today I tried to hack it into a separate console window (I am using a large screen and half the console window was unused anyway).

So I got something like this: http://www.imagehosting.cz/thumbs/goxtool1.png

At the moment it is rather an ugly hack. And the question is: anyone else interested? Should I clean it up?

[prof7bit]

So I got something like this:

Did you change the character that is used for the depth bars or is that the way putty displays the ACS_CKBOARD character?

(Also from your screenshot it seems you are still using the socketio server, you should switch to websocket, much better experience, faster connect and no disconnects anymore)

[FoxMURDER]

That is just a mess from misconfigured (not using UTF8) putty. I use it just to check if I broke something before pushing to my server. The "real thing" is running in different window and bars are just fine. Just trying to figure out what protocol I'm using there.

[prof7bit]

Ok, I'm quickly inserting this post here before I forget it, this might be of interest for others trying to use goxtool with putty (there seem to be a lot), After seeing the screenshot I did some quick experiments with the Linux version of putty myself and line-drawing and other special characters looked totally screwed up here too. There seem to be the following few important settings that must be changed from the default, this will also solve the problem with F4 key not working:

Terminal > Keyboard > "Function keys and keypad" : Xterm R6
Wndow > Tranlation > "use font encoding"
Wndow > Tranlation > "[_] Override with UTF-8"  (remove the checkmark)

the equivalent options in the session settings file are:

LinuxFunctionKeys=2
LineCodePage=Use font encoding
UTF8Override=0

PS: It also seems that the UTF-8 problem will not occur when running goxtool through screen, screen seems to handle the character translation a different way but if not running through screen then these settings should help.

[supert]

Some beer money on the way to prof7bit from me.

[supert]

Having been mightily impressed by MtQuid's .js tomfoolery, perhaps someone is interested in using something like this to display the orderbook history:
http://code.google.com/p/javascript-surface-plot/

[MtQuid]

Having been mightily impressed by MtQuid's .js tomfoolery, perhaps someone is interested in using something like this to display the orderbook history:
http://code.google.com/p/javascript-surface-plot/

Ok I tried this out and got it working even though I think there must be better ways to represent the data.
See the files
https://github.com/MtQuid/goxtool/blob/master/strategies/dump_order_book.py
https://github.com/MtQuid/goxtool/blob/master/www-root/orderbook-3d.html

It can take some time to calculate the points within the browser window so I'm sure the solution can be optimised, though it is a lot faster when var fillPly = false;
Default settings are added to goxtool.ini on first load.  I have used this config solution in other strategies and I like it because all the configs are in one file.
If you want to keep the same folder layout then have a look at some of my other commits.

http://www.imagehosting.cz/thumbs/3dorderboo.png

See what you think and tell me if there are any bugs.

[c0inbuster]

Hello,

I don't see why using a 3D plot for a problem with only 2 variables (price, volume)...
unless third variable is time (but I'm not sure that it's what you are drawing)

In my mind it will be a much better idea to work on a Qt widget to display
candlesticks (like Highcharts Highstock) and also a Qt widget to display orderbook depth
Qwt could be help http://qwt.sourceforge.net/
or this project https://code.google.com/p/candelabro/
so it could be add to goxgui https://bitcointalk.org/index.php?topic=176489.0

On my side, I have now historical data M1, M5, M15, M30, H1, H2, H6, H12, D1, W1 data
for mtgoxBTCUSD.

I've done all the work to test if data are ok (no missing data, no overlapping data, constant timeframe)

I'm now working on a script to generate pseudo tick data (generate control points from candlestick).

I will release this soon...

Kind regards

[MtQuid]

Hello,

I don't see why using a 3D plot for a problem with only 2 variables (price, volume)...
unless third variable is time (but I'm not sure that it's what you are drawing)

Yep it's time.
Here is a graphic of an hours worth of change on the orderbook using rickshaw.
I just think these things look cool.

http://www.imagehosting.cz/thumbs/3dordebsb.png

I'll have a look at the Qt but I'm having fun messing around with python and js at the moment.
I hadn't looked at that project before - now exploring.

[c0inbuster]

Ok very well !!! in fact I prefer 2D version with colormap to display much recent orderbook depth darker
you should use opacity (if you can)

I hadn't looked at that project before - now exploring.

Which project are you talking of ?

• Qt
• Qwt
• Candelabro
• Goxgui

[supert]

MtQuid your work is awesome and I look forward to playing with it this weekend.

[MtQuid]

Which project are you talking of ?

goxgui
I had seen it before but when looking for a trading framework I skipped over it as I wasn't looking for a GUI.
In fact I still don't really need anything other than goxtool but I'll keep goxgui in mind when playing around.
What I really like about goxtool is that it's not a lot of source code and it's not hard to digest the whole lot.
If you plan to build a bot that plays dice with your own money then you really must look at all the code and have a feel for it.
Also goxtool is not compiled, or anything like that, and once you have examined the source you can safely use it and only
have to review each new commit.
There are many things I really like about goxtool.

I just tried goxgui but it was too big for my laptop screen and I couldn't shrink the dialog any more.
Not a great start....

Anyway I've been working on a buy/sell helper bot that could be useful to some people and I'm just finishing of the testing.
My plan is to have this advanced buying/selling being controlled by a strategy bot that takes inputs from other providers like
lumptrade/lag/ta-lib/...  Have it all modular.  I'll release it all when I'm happy it works as planned.

Would be nice to have a fake MtGox to test out the buying and selling on and simulate market events.  Less cost and less time.

And at the moment in that bot I'm using key-press with feedback in the log file but it would be nice to be able to pop up some ncursors
dialogs from inside a strategy.

Also we should think about what to do with multiple accounts or accounts trading in different quotes and bases
Should each account need another instance of goxgui running? If so at least a colour change would be useful for recognising an account.
Colours in goxtool.ini anyone??
Maybe an account selector via command line argument or start-up prompt.  With each account able to colourise the gui, along with inline switching.
I know I'm saying goxtool is lean but this does not add much weight.

Lots to play about with

Edit: Scrap the online account switching idea as this would be a nightmare for the bots...I have been drunk.
I think all that's needed is select config from command line argument and have some limited colour options in the config.

[c0inbuster]

I don't think having differents instances of goxtool will be a very good idea...
because you can design a strategy which need to know price of both mtgox|BTC/USD and mtgox|BTC/EUR.
for now, we can't (in my mind) have such a strategy implemented

[jbsnyder]

Been lurking for a bit, mainly because it took a while to get out of newbie jail.

I've been playing around with some strategies using the pandas python module and EMAs.  As I think prof7bit noted somewhere, things like this seem to work well in some cases but then fall apart in others resulting in somewhat of a wash.  The latest thing I've given a try adds thresholding based on volume on top of using EMA crossovers.  We'll see :-)

One thing that might be of interest to those on here developing such algorithms might be quantopian.com which is a Python-based stackexchange-ish online forum for backtesting and discussing algorithms (they'll run the algorithms online and generate some nice plots/stats).  Their main data sources do not include bitcoin, but they do provide an ability to pull in CSV data from anywhere and there's at least one database that will provide daily data for that that one can test against it:

https://www.quantopian.com/posts/anyway-to-import-bitcoin-data

It might be interesting to build an adapter within a template to match the API here, or perhaps one of the other backtesting frameworks out there to be able to test algorithms.

Edit: Also, regarding GUIs why not use something based on matplotlib or maybe Enthought's Chaco which is designed for interactive plotting:
http://code.enthought.com/chaco/

Edit2: quantopian's backend is built around the zipline module, so perhaps that could figure into offline backtesting:
https://github.com/quantopian/zipline

[limpbrains]

Been lurking for a bit, mainly because it took a while to get out of newbie jail.

I've been playing around with some strategies using the pandas python module and EMAs.  As I think prof7bit noted somewhere, things like this seem to work well in some cases but then fall apart in others resulting in somewhat of a wash.  The latest thing I've given a try adds thresholding based on volume on top of using EMA crossovers.  We'll see :-)

One thing that might be of interest to those on here developing such algorithms might be quantopian.com which is a Python-based stackexchange-ish online forum for backtesting and discussing algorithms (they'll run the algorithms online and generate some nice plots/stats).  Their main data sources do not include bitcoin, but they do provide an ability to pull in CSV data from anywhere and there's at least one database that will provide daily data for that that one can test against it:

https://www.quantopian.com/posts/anyway-to-import-bitcoin-data

It might be interesting to build an adapter within a template to match the API here, or perhaps one of the other backtesting frameworks out there to be able to test algorithms.

Edit: Also, regarding GUIs why not use something based on matplotlib or maybe Enthought's Chaco which is designed for interactive plotting:
http://code.enthought.com/chaco/

Edit2: quantopian's backend is built around the zipline module, so perhaps that could figure into offline backtesting:
https://github.com/quantopian/zipline

Thanks for great links

Good video to understand how to work with zipline and ipython
http://vimeo.com/53064082
If you want to try it, you should run ipython with pylab inline

Code:

ipython notebook --pylab inline

Have anybody succeed to load mtgox data into it?

[c0inbuster]

zipline seems to be exactly what I was looking for !!!
a python algo trading environment for backtesting (but event-driven !)

[MtQuid]

Great links with the zipline stuff

Here is my notebook with dual moving average on bitcoin data with 20/10 day windows.
http://nbviewer.ipython.org/5561936
I just don't know why I have to add 2 items to the moving average lists.

Though this stuff does not really belong in this goxtool thread..ooOOO  what to do?
In fact I only post in this thread so I don't care, the rest of the forum is the wild west to me.
And this stuff will belong in here when we make zipline work with a goxtool strategy.

Someone make a new thread..I'm going to bed...