ncurses based MtGox live monitor and trading-bot-framework

[prof7bit]

...written in python. Curses based UI, live orderbook, ascii-art chart of latest 24h, trading functions can be defined in separate module "strategy.py" which receives all events from the gox client and all keystrokes from the terminal. Strategy module can be edited and reloaded at runtime which is great for experimenting with bots. Works also without MtGox API-key to view only without trading.


(still experimantal, license: GPL3)
http://prof7bit.github.com/goxtool/
https://github.com/prof7bit/goxtool

screenshot


Usage:

• it needs python 2.7 (and maybe a few additional packages but nothing that would not be found in the standard ubuntu repositories). It will NOT work on Windows because there exists no ncurses (maybe it works with cygwin, not tested)
• Download and unzip (or better git clone the repo from github).
• ./goxtool.py

This should give you the orderbook on the left side and an M15 candlestick chart of the lat 24 hours (if your terminal is wide enough) on the right side and a log window at the bottom.

To use the trading features do the following:

./goxtool.py --add-secret

This will ask you for your MtGox API key and secret, you get them at the mtgox website, copy and paste key and secret, supply a passphrase which will be used to encrypt the secret. Then it will write the encrypted secret to goxtool.ini and exit. The next time you start ./goxtool.py it will ask for the passphrase to access the API secret and then all trading functions will be enabled, you will see your own orders in the orderbook and in the chart and your account balance at the top of the terminal.

To use it with other currencies, just edit the ini file. If it is one of these days when the socket.io just wont want to connect, edit the ini file and change use_plain_old_websockets to True. If you don't need history or full orderbook you can turn it off in the ini too.

press q to quit the program.
press F4 to buy
press F5 to sell
press F6 to view orders / cancel orders
press l (lowercase L) to reload the strategy module at runtime.
any other key will call the slot_keypress() in the Strategy class in strategy.py, see the examples. Here you can call methods like gox.buy() or .sell() or .cancel()  or peek around in the order book to decide what fancy stuff to do.

(this forum posting and the instructions might eventually come out of sync with the current version, see here for the official manual: http://prof7bit.github.com/goxtool/)

There are also methods that will be fired on signals from the gox object, you can use them to build fully automated trading bots (there exist even more signals to connect to, for example orderbook.signal_changed or gox.signal_wallet, etc. Look into the source of goxtool to find them all). You can also instantiate more Gox() objects from within your strategy for different currencies (you will also need separate GoxConfig() objects for each of them that use separate config files) to trade multiple currencies at once (useful for arbitrage strategies).

And finally you can rip out parts of this for your own programs, you can just import goxapi and use the Gox() class on its own in your own program, connect your slots to its signals and make your own UI for it (or no UI at all).

[molecular]

cool stuff man, just took a look at the code.

I couldn't find it so you probably haven't implemented keeping an up-to-date synced orderbook? Maybe I didn't look hard enough.

[prof7bit]

I couldn't find it so you probably haven't implemented keeping an up-to-date synced orderbook? Maybe I didn't look hard enough.

Please pull a recent version from github, I just made some changes an hour ago and fixed some bugs and also refactored some stuff (I'm still moving a lot of stuff around, thats why I called it "experimental" ;-).

The orderbook is updated automatically. The Gox() instance contains an OrderBook() instance, gox is generating signals for trade, depth, ticker, user_order, etc and orderbook will connect itself to all these signals. Look for signal_xxxx.connect() calls in the constructors to see what is connected to what and the slot_xxxx() functions to see how they all react to these signals. OrderBook will process the signals fulldepth, ticker, depth, trade and user_order and orderbook itself will then emit signal_changed for the UI to update.

[OpenYourEyes]

Fantastic work. I love curses based programs.

I'll chuck a donation you way soon. Any chance you could display the high/low/last prices at the top near the account info.
It will just make it easier to see the current BTC at a quick glance.

[prof7bit]

Today is again one of these days where MtGox's SocketIO server is almost unusable.

Open the .ini file and switch to plain old websocket protocol:

use_plain_old_websocket = True

Both protocols are implemented, you can compare how they behave and which one is better for you, default is socketio because they say the other one is deprecated but it turns out that plain old websocket is *much* more reliable on days like today while socketio won't connect at all or the messages lag behind websocket by 20 seconds or more!

(If you are going to experiment with a lot of restarts of the application then please temporarily turn off load_fulldepth and load_history because this is eating a lot of bandwidth)

[prof7bit]

Added:
* y-axis labels for chart
* better looking candle bodies in chart
* display current order-lag
* display last trade price, bid and ask in the xterm window title bar
* fixed some bugs and flaws (and probably also introduced new ones)

[Herodes]

Very cool project - I will look into this!

[prof7bit]

Implemented manual trading functions, now its a full featured trading client :-)



In the above screenshot you see the order cancel dialog (press F6 to open it). It is inspired by midnight commander: arrow keys to move up/down, insert to select/unselect an order, F8 to cancel all selected orders, F10 to exit the dialog (sorry, cannot use ESC without ugly hacks). F10 is the key to exit all dialogs.

F4: new buy order
F5: new sell order
F6: cancel order(s)
q quit the client

l (lowercase L) reload the strategy module (your trading bot)

other keys (a..z, except the two above) will emit the signal gox.signal_keypress, the default empty strategy class already has a slot connected to this signal, there you can trigger stuff to happen in your bot when you press certain keys.

Also added since my last posting: display total fiat on the orderbook bid side, total BTC on the orderbook ask side, ratio of these two numbers BTC/USD, everything updating in realtime (see screenshot)

Code:

$ ./goxtool.py -h
usage: goxtool.py [-h] [--add-secret] [--strategy STRATEGY]
                  [--protocol PROTOCOL] [--no-fulldepth] [--no-history]

MtGox live market data monitor and trading bot experimentation framework

optional arguments:
  -h, --help           show this help message and exit
  --add-secret         prompt for API secret, encrypt it and then exit
  --strategy STRATEGY  name of strategy module file, default=strategy.py
  --protocol PROTOCOL  force protocol (socketio or websocket), ignore setting
                       in .ini
  --no-fulldepth       do not download full depth (useful for debugging)
  --no-history         do not download full history (useful for debugging)

happy trading :-)

[K1773R]

is it possible to make it backward compatible to 2.6?

[Deafboy]

+1 for 2.6
Debian squeeze has 2.5, 2.6 and 3.0 in repo.

[prof7bit]

is it possible to make it backward compatible to 2.6?

Not sure, I don't have python 2.6 installed to test it, I relied on the fact that the 2.x branch ended at 2.7 many years ago (I consider it as old and mature and stable) and I assumed that over the time (until now?) from all the different 2.x versions there would eventually only 2.7 be left as the only relevant 2.x version. What Linux distribution are you using? Debian?

What exactly is missing/wrong/different? If it is something trivial it might be possible, but the main problem is I can't (and don't want to) test and maintain it on 2.6 but maybe someone else can temporarily maintain a 2.6 fork until there is no 2.6 anymore (hopefully soon).

[prof7bit]

+1 for 2.6
Debian squeeze has 2.5, 2.6 and 3.0 in repo.

Debian. Isn't the whole point of willingly choosing Debian instead of practically *any* other distribution to explicitly *avoid* any software that is less than 5 years old and work only with old and proven stuff that does not move a single millimeter anymore?

Don't misunderstand me. I mean I really try my best to avoid using any bleeding edge stuff when writing software (I once criticized bitcoin-wx for using a bleeding edge wx-widgets version that was *not* in widespread use), I consider something stable and available and usable for mainstream development if it is in the standard repositories of Ubuntu LTS. I had the same kind of problems when I started TorChat 5(!) years ago with python 2.6 (stable on Ubuntu Hardy at that time and widely used everywhere), soon after the Debian users started complaining that it would not work with their ancient python version.

There were even a few Windows-98 users (imagine! win98!), I even installed win98 in wmvare to see what strange problem they had with python-2.6 on win-98 and to see if I could fix it (and fixed it!) but eventually I decided that it is just not reasonably possible to care about all these exotic corner cases (and a Debian stable user (==conservative by definition) who wants to install the latest bleeding edge software is an exotic corner case). I don't have an army of programmers and testers working for me, At some point I must make a decision and then stick to it. In my case this is the software and libs and their versions that are available in Ubuntu.

[K1773R]

is it possible to make it backward compatible to 2.6?

Not sure, I don't have python 2.6 installed to test it, I relied on the fact that the 2.x branch ended at 2.7 many years ago (I consider it as old and mature and stable) and I assumed that over the time (until now?) from all the different 2.x versions there would eventually only 2.7 be left as the only relevant 2.x version. What Linux distribution are you using? Debian?

What exactly is missing/wrong/different? If it is something trivial it might be possible, but the main problem is I can't (and don't want to) test and maintain it on 2.6 but maybe someone else can temporarily maintain a 2.6 fork until there is no 2.6 anymore (hopefully soon).

python 2.6/2.7 are considered stable, therefore used by most ppls.
well it dosnt work:

Code:

$ python goxtool.py 
Traceback (most recent call last):
  File "goxtool.py", line 1031, in <module>
    main()
  File "goxtool.py", line 1023, in main
    curses.wrapper(curses_loop)
  File "/usr/lib/python2.6/curses/wrapper.py", line 44, in wrapper
    return func(stdscr, *args, **kwds)
  File "goxtool.py", line 947, in curses_loop
    gox = goxapi.Gox(secret, config)
  File "/home/k1773r/git/goxtool/goxapi.py", line 779, in __init__
    BaseObject.__init__(self)
  File "/home/k1773r/git/goxtool/goxapi.py", line 243, in __init__
    self.signal_debug = Signal()
  File "/home/k1773r/git/goxtool/goxapi.py", line 172, in __init__
    self._functions = weakref.WeakSet()
AttributeError: 'module' object has no attribute 'WeakSet'

[prof7bit]

Code:

AttributeError: 'module' object has no attribute 'WeakSet'

This is unfortunate because I am not going to refrain from using these weakref classes. Maybe it would be easier to port the entire program to python-3 than to backport weakref.WeakSet() to a deprecated old python-2.6 version, but then again the problem would be that I would then probably be tempted to use features from python 3.2 (Ubuntu) and Debian users with deprecated 3.0 would complain.

I'm doing this for *fun* and not for profit! If I see that there is there is python 2.7 installed on stable Ubuntu (and even was installed on the previous Ubuntu version too and probably even on the one before that) then I am going to use it. I choose python-2.7 because it is the largest stable island and common denominator in the entire python world that is most likely to be available *everywhere*.

There *must* exist a python-2.7 installer for your distribution, 2.7 is three(!) years stable already. Maybe somewhere in the backports repository. If not then there is something seriously wrong in Debian-land, sorry.

[K1773R]

yes, if backporting isnt possible i will run python2.7 coexistent, that should be fine

[prof7bit]

yes, if backporting isnt possible i will run python2.7 coexistent, that should be fine

I have just googled and it seems you could break your debian stable or bring it into some totally inconsistent state when installing 2.7 on it.

If you really want to experiment with new software then I think debian stable is not the right distribution for you. Its main "advantage" is that it consists only of stuff that is so old (old abandoned branches of software versions) that won't ever change anymore (not moving == stable (== dead?)). You can use it to run old server software that has existed for 20+ years and run them for another 20+ years, software that won't break or change APIs with new updates because the only updates they will ever receive at all will be security fixes. That the Debian definition of "stable": old, petrified, fossilized. Thats what Debian *stable* is meant for.

If you have a Desktop computer and want to be flexible and experiment with cool new stuff now and not in 3 years when its old, you should choose a different distribution. The simplest solution for you would probably to upgrade to Debian testing if you intend to test new (everything that is not really old) software and then stay on their testing release schedule from then on.

Bleeding edge and Debian-Stable just does not go together. Not even "pretty well stable" or "stable" or even "rock solid stable for 2 years now" and "Debian-stable". They have a different and very extreme definition of "stable" over there at Debian.

[K1773R]

yes, if backporting isnt possible i will run python2.7 coexistent, that should be fine

I have just googled and it seems you could break your debian stable or bring it into some totally inconsistent state when installing 2.7 on it.

If you really want to experiment with new software then I think debian stable is not the right distribution for you. Its main "advantage" is that it consists only of stuff that is so old (old abandoned branches of software versions) that won't ever change anymore (not moving == stable (== dead?)). You can use it to run old server software that has existed for 20+ years and run them for another 20+ years, software that won't break or change APIs with new updates because the only updates they will ever receive at all will be security fixes. That the Debian definition of "stable": old, petrified, fossilized. Thats what Debian *stable* is meant for.

If you have a Desktop computer and want to be flexible and experiment with cool new stuff now and not in 3 years when its old, you should choose a different distribution. The simplest solution for you would probably to upgrade to Debian testing if you intend to test new (everything that is not really old) software and then stay on their testing release schedule from then on.

Bleeding edge and Debian-Stable just does not go together. Not even "pretty well stable" or "stable" or even "rock solid stable for 2 years now" and "Debian-stable". They have a different and very extreme definition of "stable" over there at Debian.

you can install another python into a specific prefix, so u dont overwrite the systems versions. or you can do it with virtualenv too

[prof7bit]

I have just run my code through 2to3 and did some additional small adjustments and got it almost working (there are no fundamental problems, its just a few module names have changed, the () around the print (there are only few) and the change to the str class (strings in py3 are always unicode and for byte-strings i have to use bytes() because str() will not have a decode() method anymore, so a few things regarding the encryption/decryption of the api secret and and base64-encoding/decoding and signing stuff must be slightly changed to use bytes() where I currently (ab)use str() for byte strings, this should all be possible without too many problems. Its only few places that need to be changed.

The main problem was the websocket code from Hiroki Ohtani, I first tried to port it myself, it contains a lot of the same sort of string manipulation all over the place and after an hour or so I gave up. Then I saw that there already exists an experimental py3 version of this module from the same author already on github, so not all hope is lost, it can eventually be ported to python 3 if necessary.

I will try to make my own code more python3 friendly (in small steps to not break anything) so that it will be prepared for easy transition eventually but I won't switch to python 3 yet, not at this time.

[algorista]

nice work.

thank you.

[dutt]

Great stuff mate.