Bitcoin Forum
June 22, 2024, 09:03:26 AM *
News: Voting for pizza day contest
 
  Home Help Search Login Register More  
  Show Posts
Pages: [1] 2 »
1  Economy / Trading Discussion / Re: zipline / Quantopian - backtesting / trading framework on: May 15, 2013, 12:18:15 PM
New feature :
variable lot size (instead of fixed lot size)
self.trade_volume is now a function which returns volume to trade according
portfolio cash and BTCUSD price.

http://nbviewer.ipython.org/587a80f5e2eb9cf41d6d

alpha= 138.7%
max DD = 7.5%

should I order the rolls-royce?  Grin

Nice work.... if only we could go back in time Smiley
I think there were some missing rows in the method I used to pull from bitcoincharts so I've made some modifications and a cache that stays up to date.
I've also include frequency information but still 'minute' runs are not working correctly.  We might have to tamper with zipline internals to get them going.
You should update your work to at least use the better bitcoincharts methods.

Now in my testing I keep an up to date cache of minute data from bitcoincharts and resample that to how I want it.

Fractional trades are still an issue.  
A fix would be to use satoshi but then the prices will all have to be suitable scaled and the results will just look a mess.
Or bump the initial portfolio value up to a few billion and increase the trade volume so that there will be no fractions but then slippage is very unrealistic.
Or modify zipline internals or use an inherited object

About passing live trade data to an algorithm:
I'm thinking of creating a new TradingAlgorithm object that is passed a DateTimeIndex and will take trade data from the phantom sqlite3 db and create OHCL and pass both simulated live data and OHCL to an algorithm under test.  So the wrapper gets a datetime and frequency from zipline and extracts the data from the phantom db, passes this to the algo inder test and then passes the results back to zipline.
Just an idea bu this might also be a way to fix the fractional trades issue.

Updated - http://nbviewer.ipython.org/5572250


Edit: In fact even if the zipline results records are always daily it does not matter because we can put minute results into something else, or accumulate them for a day.  Depends on what you want to log in the results.  Currently, I guess that, the same record gets overwritten a lot for each minute in a day so only the last record of the day is saved in the results
Also seams like there is a bug in my example because the trade volumes are messed up and I've lost a lot of money :/

Edit:  All working now.  I was not using ohlc resampling
2  Economy / Trading Discussion / Re: zipline / Quantopian - backtesting / trading framework on: May 13, 2013, 11:16:44 PM
I've not found a good tutorial on zipline so have just been reading the source code.

This new book pulls data from bitcoincharts
http://nbviewer.ipython.org/5572250

You can use non daily data but the results from TradingAlgorithm.run() are daily so you have to play around a bit at the end.
The simulation will run correctly though.

Can not place fractional orders.
To fix the issue of not being able to place fractional orders we will have to use MtGox order volumes which are satoshi.

And there are no buys or sells in results when using M15,H1 etc... even though the buy or sell takes place during the simulation.

3  Economy / Trading Discussion / Re: zipline / Quantopian - backtesting / trading framework on: May 12, 2013, 08:12:44 PM
You have to use panel because 'data' is a DataFrame dict of TimeSeries.  As far as I know a TimeSeries can only have one value for each time-stamped row, and that is why the previous notebook only passed the single ['price'] TimeSeries and not the rest.  We need multiple values/observations (open,high,low,close,volume...) per row in the TimeSeries so we use the panel method.  Reading the Quantopian forum and zipline commit logs you can see that this is the chosen and agreed upon method for passing around OHCL sets.
I just took adjusted from the load_bars_from_yahoo() source and use the defaulted values but I'll delete the code on Monday as Bitcoin is without splits and dividends.  I was in a rush to post before the roast.

You can use whatever data you want with the simulator but you will need to turn it into a panel if you want to be able to pass around multiple observations per tick, and also if you want to have the TradingAlgorithm be able to issue orders in the handle_data(), unless you build your own datasource tick generator wich might not be a bad thing.
Anyway, It is very easy now.
handle_data() is your brains and it can view all the OHCL per tick and issue order(buy/sell) with the results tracked for easy analysis of performance.
Everything is now possible.
Add bitcoincharts json files with selectable time collapse and then we are are really cooking
..but that is work for Monday.

Edit:  notebook has been updated from the lastest DMA example shipped with zipline source. 
The bugs with extra values added and non showing graph arrows have been resolved.

Edit: we still need to work MtGox fees into the analysis but I'm doing that on a goxtool bot so I have the accurate code
4  Economy / Trading Discussion / Re: zipline / Quantopian - backtesting / trading framework on: May 12, 2013, 12:47:45 PM
Yeah I'm also wondering about the not seeing the buy/sell (^ and v)
I think this ties in with me having to add extra values to those two series.
I finished that stuff off drunk last night...but that charts show it would have made profit  Tongue

I've updated the notebook now to use OHLC and it works.
I took the code from load_bars_from_yahoo() so we can use stuff like data['BTC']['open'] within the handler.
Works well.

Almost there....
I also agree that we need to use a better data source and that should probably be bitcoincharts.

Bots just puke up machine language.
Time to talk to some humans down the pub.  Sunday Roast!!! Smiley
5  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: May 12, 2013, 12:47:43 AM
Great links with the zipline stuff

Here is my notebook with dual moving average on bitcoin data with 20/10 day windows.
http://nbviewer.ipython.org/5561936
I just don't know why I have to add 2 items to the moving average lists.

Though this stuff does not really belong in this goxtool thread..ooOOO  what to do?
In fact I only post in this thread so I don't care, the rest of the forum is the wild west to me.
And this stuff will belong in here when we make zipline work with a goxtool strategy.

Someone make a new thread..I'm going to bed... Undecided
6  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: May 10, 2013, 02:57:17 PM
Which project are you talking of ?

goxgui
I had seen it before but when looking for a trading framework I skipped over it as I wasn't looking for a GUI.
In fact I still don't really need anything other than goxtool but I'll keep goxgui in mind when playing around.
What I really like about goxtool is that it's not a lot of source code and it's not hard to digest the whole lot.
If you plan to build a bot that plays dice with your own money then you really must look at all the code and have a feel for it.
Also goxtool is not compiled, or anything like that, and once you have examined the source you can safely use it and only
have to review each new commit.
There are many things I really like about goxtool.

I just tried goxgui but it was too big for my laptop screen and I couldn't shrink the dialog any more.
Not a great start....

Anyway I've been working on a buy/sell helper bot that could be useful to some people and I'm just finishing of the testing.
My plan is to have this advanced buying/selling being controlled by a strategy bot that takes inputs from other providers like
lumptrade/lag/ta-lib/...  Have it all modular.  I'll release it all when I'm happy it works as planned.

Would be nice to have a fake MtGox to test out the buying and selling on and simulate market events.  Less cost and less time.

And at the moment in that bot I'm using key-press with feedback in the log file but it would be nice to be able to pop up some ncursors
dialogs from inside a strategy.

Also we should think about what to do with multiple accounts or accounts trading in different quotes and bases
Should each account need another instance of goxgui running? If so at least a colour change would be useful for recognising an account.
Colours in goxtool.ini anyone??
Maybe an account selector via command line argument or start-up prompt.  With each account able to colourise the gui, along with inline switching.
I know I'm saying goxtool is lean but this does not add much weight.


Lots to play about with Smiley

Edit: Scrap the online account switching idea as this would be a nightmare for the bots...I have been drunk.
I think all that's needed is select config from command line argument and have some limited colour options in the config.
7  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: May 08, 2013, 09:25:11 PM
Hello,

I don't see why using a 3D plot for a problem with only 2 variables (price, volume)...
unless third variable is time Huh (but I'm not sure that it's what you are drawing)

Yep it's time.
Here is a graphic of an hours worth of change on the orderbook using rickshaw.
I just think these things look cool.



I'll have a look at the Qt but I'm having fun messing around with python and js at the moment.
I hadn't looked at that project before - now exploring.
8  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: May 08, 2013, 01:42:35 PM
Having been mightily impressed by MtQuid's .js tomfoolery, perhaps someone is interested in using something like this to display the orderbook history:
http://code.google.com/p/javascript-surface-plot/

Ok I tried this out and got it working even though I think there must be better ways to represent the data.
See the files
https://github.com/MtQuid/goxtool/blob/master/strategies/dump_order_book.py
https://github.com/MtQuid/goxtool/blob/master/www-root/orderbook-3d.html

It can take some time to calculate the points within the browser window so I'm sure the solution can be optimised, though it is a lot faster when var fillPly = false;
Default settings are added to goxtool.ini on first load.  I have used this config solution in other strategies and I like it because all the configs are in one file.
If you want to keep the same folder layout then have a look at some of my other commits.



See what you think and tell me if there are any bugs.
9  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: May 02, 2013, 10:39:23 AM
I just want to inform you, guys, that an interesting python open source software for backtesting strategy is available here
http://gbeced.github.io/pyalgotrade/

mixing both goxtool and pyalgotrade could make a good "recipe".

I'm looking for a 1 hour timeframe data history for MtGox BTCUSD

I tried http://bitcoincharts.com/t/trades.csv?symbol=mtgoxUSD&start=1335830400&end=1367366400

as 1335830400 is UNIX Timestamp for Tue, 01 May 2012 00:00:00 GMT
and 1367366400 for Wed, 01 May 2013 00:00:00 GMT

but Bitcoincharts only reply with data from 1367249267 (Mon, 29 Apr 2013 15:27:47 GMT)

Cool.
Have a look here http://www.reddit.com/r/Bitcoin/comments/11iz5b/the_history_of_gox_mountain/
I just downloaded the torrent and this is the head
Code:
http://data.mtgox.com/api/2/BTCUSD/money/trades/fetch?since=0
insert into trades values ("USD", "1279408157", "Y", "BTC", "limit", "1", "2000000000", "20", "4951", "", "0.04951")
insert into trades values ("USD", "1279424586", "Y", "BTC", "limit", "2", "5001000000", "50.01", "5941", "", "0.05941")
insert into trades values ("USD", "1279475336", "Y", "BTC", "limit", "3", "500000000", "5", "8080", "", "0.0808")
insert into trades values ("USD", "1279489451", "Y", "BTC", "limit", "4", "1000000000", "10", "8585", "", "0.08585")
insert into trades values ("USD", "1279490426", "Y", "BTC", "limit", "5", "500000000", "5", "8584", "", "0.08584")
insert into trades values ("USD", "1279490436", "Y", "BTC", "limit", "6", "500000000", "5", "8584", "", "0.08584")
insert into trades values ("USD", "1279511584", "Y", "BTC", "limit", "7", "500000000", "5", "9090", "", "0.0909")
insert into trades values ("USD", "1279556653", "Y", "BTC", "limit", "9", "8000000000", "80", "9307", "", "0.09307")
insert into trades values ("USD", "1279559013", "Y", "BTC", "limit", "10", "10000000000", "100", "8911", "", "0.08911")

and the tail
Code:
insert into trades values ("USD", "1367047706", "Y", "BTC", "limit", "1367047706332857", "9980000", "0.0998", "13683040", "ask", "136.8304")
insert into trades values ("USD", "1367047791", "Y", "BTC", "limit", "1367047791768232", "25712723", "0.25712723", "13688009", "bid", "136.88009")
insert into trades values ("USD", "1367047804", "Y", "BTC", "market", "1367047804704062", "80186290", "0.8018629", "13682010", "ask", "136.8201")
insert into trades values ("USD", "1367047805", "Y", "BTC", "limit", "1367047805062520", "79726342", "0.79726342", "13688009", "bid", "136.88009")
insert into trades values ("USD", "1367047825", "Y", "BTC", "limit", "1367047825872682", "497020000", "4.9702", "13682020", "ask", "136.8202")
insert into trades values ("USD", "1367047826", "Y", "BTC", "limit", "1367047826181247", "500000000", "5", "13682000", "ask", "136.82")
insert into trades values ("USD", "1367047853", "Y", "BTC", "limit", "1367047853175671", "502980000", "5.0298", "13682000", "bid", "136.82")
insert into trades values ("USD", "1367047879", "Y", "BTC", "market", "1367047879486765", "800000000", "8", "13686000", "bid", "136.86")
insert into trades values ("USD", "1367047879", "Y", "BTC", "market", "1367047879775307", "600000000", "6", "13688000", "bid", "136.88")
insert into trades values ("USD", "1367047879", "Y", "BTC", "market", "1367047879824265", "122345236", "1.22345236", "13688009", "bid", "136.88009")

Would be cool to link up with pyalgotrade.
10  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: May 01, 2013, 01:52:09 PM
hy
i started to look into goxbot and i must admit, i dont get anything, but anyway i will reach the goal of my own little bot. Wink plz help!

in my strategy module i got (copied most of it of the 50/50-balance boot)

Code:
import strategy
import goxapi


class Strategy(strategy.Strategy):

    """a protfolio rebalancing bot"""
    def __init__(self, gox):
        strategy.Strategy.__init__(self, gox)
        self.temp_halt = False


    def slot_before_unload(self, _sender, _data):
        pass

    def slot_keypress(self, gox, (key)):
if key == ord("b"):
     self.debug("canceling all rebalancing orders")






If key "b" is pressed goxtool displays: someone pressed key b,
But it doesnt display "canceling all rebalancing orders".
so, how to write messages to the log output?


btw: is there an easy way to get rid of the Winconsole output in the main window? I am running tail on the log in a second window, so its just superfluid for me?

Your strategy file works perfectly though it won't pay the bills.
Make sure you are loading it.  e.g.
Code:
python ./goxtool.py --protocol=websocket --strategy=_test.py
And you should see the load and unload in goxtool.log
Code:
2013-05-01 14:47:32,433:DEBUG:_test.Strategy:_test.Strategy loaded
...
2013-05-01 14:47:38,099:DEBUG:_test.Strategy:canceling all rebalancing orders
...
2013-05-01 14:47:40,974:DEBUG:_test.Strategy:_test.Strategy unloaded
11  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: April 30, 2013, 11:57:01 PM
EDIT: MtQuid tried your code and its working perfectly. Im sending you a small tip.

Tip recieved. 
Thanks hugolp Smiley
12  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: April 30, 2013, 05:24:31 PM
EDIT: Also, can anyone give me a hint on how to create a separate debug file for my bot? The regular debug goxtool file gets too big and its very cumbersome to check.

To reduce debug clutter for writing strategies I use a crafty tail.  I did have a cut on the end but it lagged
Code:
tail -f goxtool.log| grep -E 'Strat|Trace|^[^0-9]'

Yeah, thats an option, but I still would preffer to have a different file so I can check it from time to time.

Quote
And how are the results?.

I just loaded the chart along with current data from bitcoin charts and they are pretty similar. 
http://bitcoincharts.com/charts/chart.json?m=mtgoxUSD&r=1&i=15-min
I don't know what weighted moving average function they are using but the data is close enough for my use though I probably should check it out on a few more days before I invest the yacht Cheesy
If you use this data you will need to alter the timestamp for HighStock.js  add on 000 (three zeros)

EDIT: I haven't checked for the 1 hour EMAs but will do some more testing and experimenting later.  Using the TA-Lib you can specify the timeperiod and then ignore those first timeperiod output values as the results settle down.  I've used the default values from bitcoincharts.  Simple at 10 and Weighted at 25.

Thanks for the direction, I will try using ta-lib between today and tomorrow, hopefully it gives better results that my manually created one (which its not difficult...).

I just put this together.
This will create a log file for each loaded strategy as well as sending log messages to the main log.
Maybe prof7bit could check it over and put it into the main source.
It's a quick hack but it works.
And you call self.debug() just like before and there is a new function self.info() for different log level.

Change Strategy.py to look like this.

Code:
"""
trading robot breadboard
"""

import goxapi
import logging


class Strategy(goxapi.BaseObject):
    # pylint: disable=C0111,W0613,R0201

    def __init__(self, gox):
        goxapi.BaseObject.__init__(self)
        self.signal_debug.connect(gox.signal_debug)
        gox.signal_keypress.connect(self.slot_keypress)
        gox.signal_strategy_unload.connect(self.slot_before_unload)
        gox.signal_ticker.connect(self.slot_tick)
        gox.signal_depth.connect(self.slot_depth)
        gox.signal_trade.connect(self.slot_trade)
        gox.signal_userorder.connect(self.slot_userorder)
        gox.orderbook.signal_owns_changed.connect(self.slot_owns_changed)
        gox.history.signal_changed.connect(self.slot_history_changed)
        gox.signal_wallet.connect(self.slot_wallet_changed)
        self.gox = gox
        self.name = "%s.%s" % \
            (self.__class__.__module__, self.__class__.__name__)

        self.logger = logging.getLogger(self.name)
        self.fh = logging.FileHandler(self.name + '.log')
        formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s')
        self.fh.setFormatter(formatter)
        self.fh.setLevel(logging.DEBUG)
        self.logger.addHandler(self.fh)

        self.info("%s loaded" % self.name)

    def info(self, *args):
        """write info event to log.  use our logger then send to the base logger"""
        msg = " ".join([str(x) for x in args])
        self.logger.info(msg)
        goxapi.BaseObject.debug(self, *args)


    def debug(self, *args):
        """write debug event to log.  use our logger then send to the base logger"""
        msg = " ".join([str(x) for x in args])
        self.logger.debug(msg)
        goxapi.BaseObject.debug(self, *args)

    def __del__(self):
        """the strategy object will be garbage collected now, this mainly
        only exists to produce the log message, so you can make sure it
        really garbage collects and won't stay in memory on reload. If you
        don't see this log mesage on reload then you have circular references"""
        self.debug("%s unloaded" % self.name)
        self.logger.removeHandler(self.fh)

    def slot_before_unload(self, _sender, _data):
        """the strategy is about to be unloaded. Use this signal to persist
        any state and also use it to forcefully destroy any circular references
        to allow it to be properly garbage collected (you might need to do
        this if you instantiated linked lists or similar structures, the
        symptom would be that you don't see the 'unloaded' message above."""
        pass

    def slot_keypress(self, gox, (key)):
        """a key in has been pressed (only a..z without "q" and "l")"""
        self.debug("someone pressed the %s key" % chr(key))

    def slot_tick(self, gox, (bid, ask)):
        """a tick message has been received from the streaming API"""
        pass

    def slot_depth(self, gox, (typ, price, volume, total_volume)):
        """a depth message has been received. Use this only if you want to
        keep track of the depth and orderbook updates yourself or if you
        for example want to log all depth messages to a database. This
        signal comes directly from the streaming API and the gox.orderbook
        might not yet be updated at this time."""
        pass

    def slot_trade(self, gox, (date, price, volume, typ, own)):
        """a trade message has been received. Note that this signal comes
        directly from the streaming API, it might come before orderbook.owns
        list has been updated, don't rely on the own orders and wallet already
        having been updated when this is fired."""
        pass

    def slot_userorder(self, gox, (price, volume, typ, oid, status)):
        """this comes directly from the API and owns list might not yet be
        updated, if you need the new owns list then use slot_owns_changed"""
        pass

    def slot_owns_changed(self, orderbook, _dummy):
        """this comes *after* userorder and orderbook.owns is updated already.
        Also note that this signal is sent by the orderbook object, not by gox,
        so the sender argument is orderbook and not gox. This signal might be
        useful if you want to detect whether an order has been filled, you
        count open orders, count pending orders and compare with last count"""
        pass

    def slot_wallet_changed(self, gox, _dummy):
        """this comes after the wallet has been updated. You can access the
        new balance like so: gox.wallet["BTC"] or gox.wallet[gox.currency]"""
        pass

    def slot_history_changed(self, history, _dummy):
        """this is fired whenever a new trade is inserted into the history,
        you can also use this to query the close price of the most recent
        candle which is effectvely the price of the last trade message.
        Contrary to the slot_trade this also fires when streaming API
        reconnects and re-downloads the trade history, you can use this
        to implement a stoploss or you could also use it for example to detect
        when a new candle is opened"""
        pass

EDIT1: Changed formatter to
Code:
formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s')
EDIT2: Changed call to base debug to use *arg instead of arg so is now
Code:
goxapi.BaseObject.debug(self, *args)
No more edits.  Code is flawless Cheesy
EDIT3: Need to remove file handler on strategy unload so that it is not reloaded twice or more during a strategy reload
I think that should do it now.  Code is never flawless Sad
Price is also going down.... Time for tea
13  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: April 30, 2013, 02:06:09 PM
MtQuid...
Would you be willing to share a basic demo of the Ta-lib integration?  I do not live in python so it would be very helpful for me, and I am sure others. 

Ok.

We will use TA-Lib for bython from these lovely chaps
http://mrjbq7.github.io/ta-lib/index.html

Assuming you are working on a debian system you will need to do the following to install ta-lib for python
If on windows or mac then check http://mrjbq7.github.io/ta-lib/install.html

Code:
sudo apt-get install python-dev 
sudo easy_install cython

wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
tar -xvf ta-lib-0.4.0-src.tar.gz
cd ta-lib
./configure --prefix=/usr
make

#can use make install here if you want
sudo checkinstall

sudo easy_install TA-Lib

Once you have TA-Lib installed this is your strategy
Code:
"""
_talib.py

This example strategy will produce some HighStock.js compatible JSON files
"""

import numpy as np
import talib
import datetime
import json
import strategy
import goxapi
from goxapi import OHLCV

class OHLCV_Encoder(json.JSONEncoder):
    """JSONEncoder for class OHLCV()"""
    def default(self, obj):
        if isinstance(obj, OHLCV):
            return [obj.tim, obj.opn, obj.hig, obj.low, obj.cls, obj.vol]
        else:
            return json.JSONEncoder.default(self, obj)

class Strategy(strategy.Strategy):
    """a protfolio rebalancing bot"""
    def __init__(self, gox):       
        strategy.Strategy.__init__(self, gox)

    def slot_history_changed(self, history, _dummy):
        """History has changed so recalculate EMAs"""
        candles = []

        # read them all - don't wory about the history parameter
        for c in reversed(self.gox.history.candles):
            candles.append(
                OHLCV(
                    # fix time for HighStock.js JSON import
                    c.tim * 1000,
                    # adjust values to be human readable
                    goxapi.int2float(c.opn, self.gox.currency),
                    goxapi.int2float(c.hig, self.gox.currency),
                    goxapi.int2float(c.low, self.gox.currency),
                    goxapi.int2float(c.cls, self.gox.currency),
                    goxapi.int2float(c.vol, "BTC")
                )
            ) 

        self.debug("New EMAs from history with %d candles" % len(candles))               

        rng = range(len(candles))
        iterable = (candles[i].opn for i in rng)
        a_opn = np.fromiter(iterable, np.float)

        iterable = (candles[i].hig for i in rng)
        a_hig = np.fromiter(iterable, np.float)
       
        iterable = (candles[i].low for i in rng)
        a_low = np.fromiter(iterable, np.float)
       
        iterable = (candles[i].cls for i in rng)
        a_cls = np.fromiter(iterable, np.float)

        iterable = (candles[i].vol for i in rng)
        a_vol = np.fromiter(iterable, np.float)

        iterable = (candles[i].tim for i in rng)
        a_tim = np.fromiter(iterable, np.int64)


        a_sma = talib.SMA(a_cls, 10)
        a_wma = talib.WMA(a_cls, 25)
        a_chaikin = talib.AD(a_hig, a_low, a_cls, a_vol)
        a_cdlCounterAttack = talib.CDLCOUNTERATTACK(a_opn, a_hig, a_low, a_cls)

        # create json compatible with HighStock.js
        with open("talib_ohlcv.json", 'w') as outfile:                   
            json.dump(candles, outfile, cls = OHLCV_Encoder)
        with open("talib_sma.json", 'w') as outfile:
            # first 10 elements contain Nan
            json.dump(np.dstack((a_tim[10:], a_sma[10:])).tolist()[0], outfile)
        with open("talib_wma.json", 'w') as outfile:
            # first 25 elements contain Nan
            json.dump(np.dstack((a_tim[25:], a_wma[25:])).tolist()[0], outfile)
        with open("talib_chaikin.json", 'w') as outfile:
            json.dump(np.dstack((a_tim, a_chaikin)).tolist()[0], outfile)
        with open("talib_cdlCounterAttack.json", 'w') as outfile:
            json.dump(np.dstack((a_tim, a_cdlCounterAttack)).tolist()[0], outfile)

        # Current price in relation to EMA
        self.debug("SMA = %f" % a_sma[-1])
        self.debug("WMA = %f" % a_wma[-1])
        self.debug("CLS = %f" % a_cls[-1])

Run goxtool like so

Code:
python ./goxtool.py --protocol=websocket --strategy=_talib.py

Create index.html like this

Code:
<!DOCTYPE HTML>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>BTC Highstock - Goxtool/TA-Lib</title>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script>
<script type="text/javascript" src="http://code.highcharts.com/stock/highstock.js"></script>
<script type="text/javascript" src="http://code.highcharts.com/modules/exporting.js"></script>
<script type="text/javascript">
$(function () {
    var seriesOptions = [],
        ohlc = [],
        volume = []
        yAxisOptions = [],
        seriesCounter = 0,
        names = ['OHLCV', 'SMA', 'WMA', 'chaikin'],
        colors = Highcharts.getOptions().colors,
        groupingUnits = [
            ['week', [1]],
            ['month', [1, 2, 3, 4, 6]]
        ];

    $.each(names, function (i, name) {
        $.getJSON('/talib_' + name.toLowerCase() + '.json', function (data) {
            console.log("Loading " + name);
            if (name == 'OHLCV') {
                // split the data set into ohlc and volume        
                var dataLength = data.length;
                for (i = 0; i < dataLength; i++) {
                    ohlc.push([
                        data[i][0], // the date
                        data[i][1], // open
                        data[i][2], // high
                        data[i][3], // low
                        data[i][4] // close
                    ]);

                    volume.push([
                        data[i][0], // the date
                        data[i][5] // the volume
                    ])
                }
            } else if (name == 'chaikin') {
                seriesOptions[i] = {

                    yAxis: 2,
                    name: name,
                    data: data
                };

            } else {
                seriesOptions[i] = {
                    name: name,
                    data: data
                };
            }

            // As we're loading the data asynchronously, we don't know what order it will arrive. So
            // we keep a counter and create the chart when all the data is loaded.
            seriesCounter++;

            if (seriesCounter == names.length) {
                createChart();
            }
        });
    });

    // create the chart when all data is loaded

    function createChart() {
        console.log("Creating chart");

        $('#container').highcharts('StockChart', {
            credits: {
                enabled: false
            },
            chart: {
                renderTo: container,
                backgroundColor: {
                    linearGradient: [0, 0, 0, 500],
                    stops: [
                        [0, 'rgb(255, 255, 255)'],
                        [1, 'rgb(215, 229, 240)']
                    ]
                }
            },
            rangeSelector: {
                buttons: [{
                        type: 'day',
                        count: 1,
                        text: '1d'
                    }, {
                        type: 'week',
                        count: 1,
                        text: '1w'
                    }, {
                        type: 'month',
                        count: 1,
                        text: '1m'
                    }
                ],
                selected: 0,
                inputEnabled: false
            },

            title: {
                text: 'BTC Historical - Goxtool/TA-Lib'
            },

            yAxis: [{
                    title: {
                        text: 'OHLC'
                    },
                    type: 'logarithmic',
                    height: 200,
                    lineWidth: 2
                }, {
                    title: {
                        text: 'Volume'
                    },
                    type: 'linear',
                    top: 300,
                    height: 100,
                    offset: 0,
                    lineWidth: 2
                }, {
                    title: {
                        text: 'Chakin'
                    },
                    type: 'linear',
                    top: 450,
                    height: 100,
                    offset: 0,
                    lineWidth: 2
                }
            ],

            plotOptions: {
                candlestick: {
                    color: '#DD2F29',
                    upColor: '#87DA3B',
                    lineColor: '#666'
                }
            },

            tooltip: {
                pointFormat: '<span style="color:{series.color}">{series.name}</span>: <b>{point.y}</b><br/>',
                valueDecimals: 2
            },

            series: [{
                    type: 'candlestick',
                    name: 'BTC',
                    data: ohlc,
                    dataGrouping: {
                        units: groupingUnits
                    }
                }, {
                    type: 'column',
                    name: 'Volume',
                    data: volume,
                    yAxis: 1,
                    dataGrouping: {
                        units: groupingUnits
                    }
                },
                seriesOptions[1], seriesOptions[2], seriesOptions[3]
            ]
        });

    }

});
</script>
</head>
<body>
<div id="container" style="height: 1000px; min-width: 600px"></div>
</body>
</html>

And run a simple web server like this
Code:
python -m SimpleHTTPServer 8080

Then go to http://localhost:8080/index.html and see sunshine

You could have the index.html auto reload with simple script or refresh with your mouse Smiley


EDIT: Also, can anyone give me a hint on how to create a separate debug file for my bot? The regular debug goxtool file gets too big and its very cumbersome to check.

To reduce debug clutter for writing strategies I use a crafty tail.  I did have a cut on the end but it lagged
Code:
tail -f goxtool.log| grep -E 'Strat|Trace|^[^0-9]'



And how are the results?.

I just loaded the chart along with current data from bitcoin charts and they are pretty similar. 
http://bitcoincharts.com/charts/chart.json?m=mtgoxUSD&r=1&i=15-min
I don't know what weighted moving average function they are using but the data is close enough for my use though I probably should check it out on a few more days before I invest the yacht Cheesy
If you use this data you will need to alter the timestamp for HighStock.js  add on 000 (three zeros)

EDIT: I haven't checked for the 1 hour EMAs but will do some more testing and experimenting later.  Using the TA-Lib you can specify the timeperiod and then ignore those first timeperiod output values as the results settle down.  I've used the default values from bitcoincharts.  Simple at 10 and Weighted at 25.
14  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: April 30, 2013, 08:58:48 AM
Yep they do look good.  I robbed some of the colour styling straight from MtGox.com

I only wanted the graph output to make sure the ta-lib functions were returning the correct values so I could use them to control my bot.  I don't think this functionality needs to be part of the main goxtool branch but a good strategy plugin using ta-lib would be handy.  Like providing values and firing signals that can be caught.

And because the data sets are small (1 days worth of trade) you could write the whole index.html file to include the data and then there would be no need for the .json files or a web server.  Just pops out a stand alone html page every now and again if that's what you require.

I've also moved my function so it automatically gets called when new history arrives.

Code:
    def slot_history_changed(self, history, _dummy):
        """History has changed so recalculate EMAs"""

Now to try and find the best usage for the ta-lib functions and make some coins..  Grin
15  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: April 29, 2013, 07:30:14 PM
@MtQuid

How far back does that 'historical' day pull from?  Since GoxTool started?

It's using the same history candles that prof7bit uses for the trade graph.  So it goes back a day in 15 minute blocks.
Though you can change the history_timeframe value in goxtool.ini to change the block size.
16  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: April 29, 2013, 06:59:32 PM
I'm not very interested by you "speaker" strategy ... for now... but maybe one day it could be useful...

I tried to play with a Python wrapper for TA-Lib
http://ta-lib.org/
http://mrjbq7.github.io/ta-lib/
this wrapper is total...
you just need to install cython, TA-Lib, and this wrapper


I just tried this out and wanted to plot the results so I dumped the data to some files and used Highstock.js to render the view.

Code:
import goxapi
from goxapi import OHLCV
import datetime
import numpy as np
import talib
import json

class OHLCV_Encoder(json.JSONEncoder):
    """JSONEncoder for class OHLCV()"""
    def default(self, obj):
        if isinstance(obj, OHLCV):
            return [obj.tim, obj.opn, obj.hig, obj.low, obj.cls, obj.vol]
        else:
            return json.JSONEncoder.default(self, obj)

class Strategy(goxapi.BaseObject):
    ........
    ........

    def slot_keypress(self, gox, (key)):
        if key == ord("t"):
            """testing talib"""       
            candles = []

            for c in reversed(self.gox.history.candles):
                candles.append(
                    OHLCV(
                        # fix time for HighStock.js JSON import
                        c.tim * 1000,
                        # adjust values to be human readable
                        goxapi.int2float(c.opn, self.gox.currency),
                        goxapi.int2float(c.hig, self.gox.currency),
                        goxapi.int2float(c.low, self.gox.currency),
                        goxapi.int2float(c.cls, self.gox.currency),
                        goxapi.int2float(c.vol, "BTC")
                    )
                ) 

            self.debug("Test history with %d candles" % len(candles))               

            rng = range(len(candles))
            iterable = (candles[i].cls for i in rng)
            a_cls = np.fromiter(iterable, np.float)
                                 
            iterable = (candles[i].tim for i in rng)
            a_tim = np.fromiter(iterable, np.int64)


            a_sma = talib.SMA(a_cls, 10)
            a_wma = talib.WMA(a_cls, 25)

            # create json compatible with HighStock.js
            with open("talib_ohlcv.json", 'w') as outfile:                   
                json.dump(candles, outfile, cls = OHLCV_Encoder)
            with open("talib_sma.json", 'w') as outfile:
                # first 10 elements contain Nan
                json.dump(np.dstack((a_tim[10:], a_sma[10:])).tolist()[0], outfile)
            with open("talib_wma.json", 'w') as outfile:
                # first 25 elements contain Nan
                json.dump(np.dstack((a_tim[25:], a_wma[25:])).tolist()[0], outfile)

with

Code:
<!DOCTYPE HTML>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>BTC Highstock - Goxtool/TA-Lib</title>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script>
<script type="text/javascript" src="http://code.highcharts.com/stock/highstock.js"></script>
<script type="text/javascript" src="http://code.highcharts.com/modules/exporting.js"></script>
<script type="text/javascript">
$(function () {
    var seriesOptions = [],
        ohlc = [],
        volume = []
        yAxisOptions = [],
        seriesCounter = 0,
        names = ['OHLCV', 'SMA', 'WMA'],
        colors = Highcharts.getOptions().colors,
        groupingUnits = [
            ['week', [1]],
            ['month', [1, 2, 3, 4, 6]]
        ];

    $.each(names, function (i, name) {
        $.getJSON('/talib_' + name.toLowerCase() + '.json', function (data) {
            console.log("Loading " + name);
            if (name == 'OHLCV') {
                // split the data set into ohlc and volume        
                var dataLength = data.length;
                for (i = 0; i < dataLength; i++) {
                    ohlc.push([
                        data[i][0], // the date
                        data[i][1], // open
                        data[i][2], // high
                        data[i][3], // low
                        data[i][4] // close
                    ]);

                    volume.push([
                        data[i][0], // the date
                        data[i][5] // the volume
                    ])
                }
            } else {
                seriesOptions[i] = {
                    name: name,
                    data: data
                };
            }

            // As we're loading the data asynchronously, we don't know what order it will arrive. So
            // we keep a counter and create the chart when all the data is loaded.
            seriesCounter++;

            if (seriesCounter == names.length) {
                createChart();
            }
        });
    });

    // create the chart when all data is loaded

    function createChart() {
        console.log("Creating chart");

        $('#container').highcharts('StockChart', {
            credits: {
                enabled: false
            },
            chart: {
                renderTo: container,
                backgroundColor: {
                    linearGradient: [0, 0, 0, 500],
                    stops: [
                        [0, 'rgb(255, 255, 255)'],
                        [1, 'rgb(215, 229, 240)']
                    ]
                }
            },
            rangeSelector: {
                buttons: [{
                        type: 'day',
                        count: 1,
                        text: '1d'
                    }, {
                        type: 'week',
                        count: 1,
                        text: '1w'
                    }, {
                        type: 'month',
                        count: 1,
                        text: '1m'
                    }
                ],
                selected: 0,
                inputEnabled: false
            },

            title: {
                text: 'BTC Historical - Goxtool/TA-Lib'
            },

            yAxis: [{
                    title: {
                        text: 'OHLC'
                    },
                    type: 'logarithmic',
                    height: 200,
                    lineWidth: 2
                }, {
                    title: {
                        text: 'Volume'
                    },
                    type: 'linear',
                    top: 300,
                    height: 100,
                    offset: 0,
                    lineWidth: 2
                }
            ],

            plotOptions: {
                candlestick: {
                    color: '#DD2F29',
                    upColor: '#87DA3B',
                    lineColor: '#666'
                }
            },

            tooltip: {
                pointFormat: '<span style="color:{series.color}">{series.name}</span>: <b>{point.y}</b><br/>',
                valueDecimals: 2
            },

            series: [{
                    type: 'candlestick',
                    name: 'BTC',
                    data: ohlc,
                    dataGrouping: {
                        units: groupingUnits
                    }
                }, {
                    type: 'column',
                    name: 'Volume',
                    data: volume,
                    yAxis: 1,
                    dataGrouping: {
                        units: groupingUnits
                    }
                },
                seriesOptions[1], seriesOptions[2]
            ]
        });

    }

});
</script>
</head>
<body>
<div id="container" style="height: 500px; min-width: 600px"></div>
</body>
</html>

Serve index.html up with
Code:
python -m SimpleHTTPServer 8080

EDIT:  Using close instead of open for EMAs and added python server example
17  Economy / Trading Discussion / Re: ncurses based MtGox live monitor and trading-bot-framework on: April 23, 2013, 09:56:33 PM
+1 for 2.6
Debian squeeze has 2.5, 2.6 and 3.0 in repo.

I just installed goxtool on a squeeze box and I made this script to help anyone else in the same situation.

Code:
# Installing python2.7 on debian stable and running goxtool
# Tested on virgin debian squeeze install with sudo installed

####################################################
# Part 1: Install python2.7

# First get some tools
sudo apt-get install build-essential
# and some libs
sudo apt-get install libsqlite3-dev zlib1g-dev libncurses5-dev libgdbm-dev libbz2-dev libreadline5-dev libssl-dev libdb-dev
 
# Optional bits and bobs
#sudo apt-get build-dep python

# starting at home
cd ~

# Get and compile Python-2.7.4
wget wget http://python.org/ftp/python/2.7.4/Python-2.7.4.tar.bz2
tar xf Python-2.7.4.tar.bz2
cd Python-2.7.4/
./configure --prefix=/usr/local
make

# This make takes about 5 mins on an old virtual machine

# Now you have 2 options:
# 1. install the new python
#  sudo make altinstall  
# 2. create a new debian package and install python 2.7 into debian package system
#  sudo checkinstall make altinstall

# I prefer checkinstall, though it will take longer
sudo apt-get install checkinstall

# There is a bug and checkinstall will fail because some directories could not be created.
# So just create them manually right now.
sudo mkdir -p /usr/local/lib/pkgconfig
sudo mkdir -p /usr/local/lib/python2.7/config

# Install and create the debian package
sudo checkinstall --pkgname=python2.7 --pkgversion=2.7.4 make altinstall

# checkinstall took about 45 minutes on my slow VM

# have a look at the new python
python2.7 --version
# and still we have the original 2.6 as the system wide default
python --version

# get back to home
cd ~

# install a python package manager
# this will install the script 'easy_install-2.7'
wget https://pypi.python.org/packages/source/d/distribute/distribute-0.6.36.tar.gz
tar xvf distribute-0.6.36.tar.gz
cd distribute-0.6.36/
sudo python2.7 setup.py install

####################################################
# Part 2: Install goxtool

# get back to home
cd ~

# install a means to segregate python applications
sudo easy_install-2.7 virtualenv

# get latest goxtool
git clone https://github.com/prof7bit/goxtool.git

# create a new python2.7 environment for goxtool
virtualenv-2.7 --distribute goxtool

# switch to the new environment
source goxtool/bin/activate

# check it worked - you should see 2.7.4 as the version number
python --version

# get the packages required for goxtool
easy_install-2.7 pycrypto

# fire her up and get trading
python goxtool/goxtool.py --protocol=socketio --use-http


I've only just discovered this project with it's balance bot and I love it and have been running it for the last day and a half on the 7% default.
I'm going to try some different margins on other accounts with like half a coin balance and see how they each perform over time.
My first trading bot.

One issue.  During setup I balanced the account and then hit 'p' to place the buy/sell orders but I think I went too fast because the price was around 123 and the bot miscalculated my center and put in buys and sells at 80/90 something.  MtGox ignored the low sell but I did panic.

Also the graph is using a lot of screen space so could do with a toggle for switching between other information sets.  I'm a C++ programmer by trade but I'm keen to help out with this projects so I'm currently looking at the code.
Graph area could switch between log, graph, more detailed order book stats, strategy provided info.
Strategy provided would be the obvious solution to all as then you could have empty strategies that only provide information screens.  Then just have a mechanism to cycle through them.  So even the current graph would end up in a strategy.

Thanks for your work prof7bit.
18  Other / Beginners & Help / Re: MT Gox account compromised. on: April 23, 2013, 08:09:36 PM
Just a pastry, heh. Actually my name has origins in Arabic.

I am still curious. How would Mt. Gox confirm that I'm the real owner?

Collect every bit of information relating to your dealings with the account and use them to prove who you are.
If you made payments into the account and you can send MtGox the verification documents to prove who you are then that would help.  Did you make any deposits or withdrawls? and what methods did you use? and what days?
How old is your account?
How often do you use it?
Where do you access it from?
What browser do you usually use to access it?
Did you use the same PC to access your account all the time?  If so then it is linked by the IP address of your PC.  This can change when you reset your router but it will change to within a range.  Go to the url http://mtgox.com/this_is_danish_and_my_username_is_xxx_help_pox.html  this will appear in their web logs and might match the IP address you last logged in with.  Do it.

The villains will be able to look through your email and trade history so you need to find things that are not listed in those places.
Bank accounts.  Letters to your home.  Photo ID, Locations of access.

Maybe you are the villain and you want to prove you own someone else's account or steal another persons.
I don't know.
19  Other / Beginners & Help / Re: $10 Bitcoins on: April 23, 2013, 07:36:50 PM
Do you live in a caravan?
20  Other / Beginners & Help / Re: MT Gox account compromised. on: April 23, 2013, 06:20:27 PM
How do you install 2 factor authentication if you have a Windows PC?

If you know how to use git and have java installed then pull the repo from https://github.com/mclamp/JAuth.git

Otherwise there is a windows installer in the downloads section https://github.com/mclamp/JAuth/downloads
I'm not on windows so don't know how that will work out for you but it should be ok.

There are probably other implementations around but I've used this one on MtGox without problems.

You then create a file with the secret key in and launch the program with the filename as a parameter, or you can even launch with the key as a parameter.

Might be different on windows but I doubt it.

MtGox requires a correct two factor login code using a key before you can enable the two factor login process.  In this way if you have managed to enable  two factor login then you know you are generating the right keys.

Remember to back the secret key up on another machine or device or even a piece of paper, it's not long.


Pages: [1] 2 »
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!