Show Posts
|
Pages: [1] 2 »
|
New feature : variable lot size (instead of fixed lot size) self.trade_volume is now a function which returns volume to trade according portfolio cash and BTCUSD price. http://nbviewer.ipython.org/587a80f5e2eb9cf41d6dalpha= 138.7% max DD = 7.5% should I order the rolls-royce? ![Grin](https://bitcointalk.org/Smileys/default/grin.gif) Nice work.... if only we could go back in time ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) I think there were some missing rows in the method I used to pull from bitcoincharts so I've made some modifications and a cache that stays up to date. I've also include frequency information but still 'minute' runs are not working correctly. We might have to tamper with zipline internals to get them going. You should update your work to at least use the better bitcoincharts methods. Now in my testing I keep an up to date cache of minute data from bitcoincharts and resample that to how I want it. Fractional trades are still an issue. A fix would be to use satoshi but then the prices will all have to be suitable scaled and the results will just look a mess. Or bump the initial portfolio value up to a few billion and increase the trade volume so that there will be no fractions but then slippage is very unrealistic. Or modify zipline internals or use an inherited object About passing live trade data to an algorithm: I'm thinking of creating a new TradingAlgorithm object that is passed a DateTimeIndex and will take trade data from the phantom sqlite3 db and create OHCL and pass both simulated live data and OHCL to an algorithm under test. So the wrapper gets a datetime and frequency from zipline and extracts the data from the phantom db, passes this to the algo inder test and then passes the results back to zipline. Just an idea bu this might also be a way to fix the fractional trades issue. Updated - http://nbviewer.ipython.org/5572250Edit: In fact even if the zipline results records are always daily it does not matter because we can put minute results into something else, or accumulate them for a day. Depends on what you want to log in the results. Currently, I guess that, the same record gets overwritten a lot for each minute in a day so only the last record of the day is saved in the results Also seams like there is a bug in my example because the trade volumes are messed up and I've lost a lot of money :/Edit: All working now. I was not using ohlc resampling
|
|
|
I've not found a good tutorial on zipline so have just been reading the source code. This new book pulls data from bitcoincharts http://nbviewer.ipython.org/5572250You can use non daily data but the results from TradingAlgorithm.run() are daily so you have to play around a bit at the end. The simulation will run correctly though. Can not place fractional orders. To fix the issue of not being able to place fractional orders we will have to use MtGox order volumes which are satoshi. And there are no buys or sells in results when using M15,H1 etc... even though the buy or sell takes place during the simulation.
|
|
|
You have to use panel because 'data' is a DataFrame dict of TimeSeries. As far as I know a TimeSeries can only have one value for each time-stamped row, and that is why the previous notebook only passed the single ['price'] TimeSeries and not the rest. We need multiple values/observations (open,high,low,close,volume...) per row in the TimeSeries so we use the panel method. Reading the Quantopian forum and zipline commit logs you can see that this is the chosen and agreed upon method for passing around OHCL sets. I just took adjusted from the load_bars_from_yahoo() source and use the defaulted values but I'll delete the code on Monday as Bitcoin is without splits and dividends. I was in a rush to post before the roast.You can use whatever data you want with the simulator but you will need to turn it into a panel if you want to be able to pass around multiple observations per tick, and also if you want to have the TradingAlgorithm be able to issue orders in the handle_data(), unless you build your own datasource tick generator wich might not be a bad thing. Anyway, It is very easy now. handle_data() is your brains and it can view all the OHCL per tick and issue order(buy/sell) with the results tracked for easy analysis of performance. Everything is now possible. Add bitcoincharts json files with selectable time collapse and then we are are really cooking ..but that is work for Monday. Edit: notebook has been updated from the lastest DMA example shipped with zipline source. The bugs with extra values added and non showing graph arrows have been resolved. Edit: we still need to work MtGox fees into the analysis but I'm doing that on a goxtool bot so I have the accurate code
|
|
|
Yeah I'm also wondering about the not seeing the buy/sell (^ and v) I think this ties in with me having to add extra values to those two series. I finished that stuff off drunk last night...but that charts show it would have made profit ![Tongue](https://bitcointalk.org/Smileys/default/tongue.gif) I've updated the notebook now to use OHLC and it works. I took the code from load_bars_from_yahoo() so we can use stuff like data['BTC']['open'] within the handler. Works well. Almost there.... I also agree that we need to use a better data source and that should probably be bitcoincharts. Bots just puke up machine language. Time to talk to some humans down the pub. Sunday Roast!!! ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif)
|
|
|
Great links with the zipline stuff Here is my notebook with dual moving average on bitcoin data with 20/10 day windows. http://nbviewer.ipython.org/5561936I just don't know why I have to add 2 items to the moving average lists. Though this stuff does not really belong in this goxtool thread..ooOOO what to do? In fact I only post in this thread so I don't care, the rest of the forum is the wild west to me. And this stuff will belong in here when we make zipline work with a goxtool strategy. Someone make a new thread..I'm going to bed... ![Undecided](https://bitcointalk.org/Smileys/default/undecided.gif)
|
|
|
Which project are you talking of ?
goxgui I had seen it before but when looking for a trading framework I skipped over it as I wasn't looking for a GUI. In fact I still don't really need anything other than goxtool but I'll keep goxgui in mind when playing around. What I really like about goxtool is that it's not a lot of source code and it's not hard to digest the whole lot. If you plan to build a bot that plays dice with your own money then you really must look at all the code and have a feel for it. Also goxtool is not compiled, or anything like that, and once you have examined the source you can safely use it and only have to review each new commit. There are many things I really like about goxtool. I just tried goxgui but it was too big for my laptop screen and I couldn't shrink the dialog any more. Not a great start.... Anyway I've been working on a buy/sell helper bot that could be useful to some people and I'm just finishing of the testing. My plan is to have this advanced buying/selling being controlled by a strategy bot that takes inputs from other providers like lumptrade/lag/ta-lib/... Have it all modular. I'll release it all when I'm happy it works as planned. Would be nice to have a fake MtGox to test out the buying and selling on and simulate market events. Less cost and less time. And at the moment in that bot I'm using key-press with feedback in the log file but it would be nice to be able to pop up some ncursors dialogs from inside a strategy. Also we should think about what to do with multiple accounts or accounts trading in different quotes and bases Should each account need another instance of goxgui running? If so at least a colour change would be useful for recognising an account. Colours in goxtool.ini anyone?? Maybe an account selector via command line argument or start-up prompt. With each account able to colourise the gui, along with inline switching. I know I'm saying goxtool is lean but this does not add much weight.Lots to play about with ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) Edit: Scrap the online account switching idea as this would be a nightmare for the bots...I have been drunk. I think all that's needed is select config from command line argument and have some limited colour options in the config.
|
|
|
Hello, I don't see why using a 3D plot for a problem with only 2 variables (price, volume)... unless third variable is time ![Huh](https://bitcointalk.org/Smileys/default/huh.gif) (but I'm not sure that it's what you are drawing) Yep it's time. Here is a graphic of an hours worth of change on the orderbook using rickshaw. I just think these things look cool. ![](https://ip.bitcointalk.org/?u=http%3A%2F%2Fwww.imagehosting.cz%2Fthumbs%2F3dordebsb.png&t=663&c=s_uh_JqAxTt7Xw) I'll have a look at the Qt but I'm having fun messing around with python and js at the moment.I hadn't looked at that project before - now exploring.
|
|
|
Ok I tried this out and got it working even though I think there must be better ways to represent the data. See the files https://github.com/MtQuid/goxtool/blob/master/strategies/dump_order_book.pyhttps://github.com/MtQuid/goxtool/blob/master/www-root/orderbook-3d.htmlIt can take some time to calculate the points within the browser window so I'm sure the solution can be optimised, though it is a lot faster when var fillPly = false; Default settings are added to goxtool.ini on first load. I have used this config solution in other strategies and I like it because all the configs are in one file. If you want to keep the same folder layout then have a look at some of my other commits. ![](https://ip.bitcointalk.org/?u=http%3A%2F%2Fwww.imagehosting.cz%2Fthumbs%2F3dorderboo.png&t=663&c=zcJ5Pz5qVjkR5A) See what you think and tell me if there are any bugs.
|
|
|
Cool. Have a look here http://www.reddit.com/r/Bitcoin/comments/11iz5b/the_history_of_gox_mountain/I just downloaded the torrent and this is the head http://data.mtgox.com/api/2/BTCUSD/money/trades/fetch?since=0 insert into trades values ("USD", "1279408157", "Y", "BTC", "limit", "1", "2000000000", "20", "4951", "", "0.04951") insert into trades values ("USD", "1279424586", "Y", "BTC", "limit", "2", "5001000000", "50.01", "5941", "", "0.05941") insert into trades values ("USD", "1279475336", "Y", "BTC", "limit", "3", "500000000", "5", "8080", "", "0.0808") insert into trades values ("USD", "1279489451", "Y", "BTC", "limit", "4", "1000000000", "10", "8585", "", "0.08585") insert into trades values ("USD", "1279490426", "Y", "BTC", "limit", "5", "500000000", "5", "8584", "", "0.08584") insert into trades values ("USD", "1279490436", "Y", "BTC", "limit", "6", "500000000", "5", "8584", "", "0.08584") insert into trades values ("USD", "1279511584", "Y", "BTC", "limit", "7", "500000000", "5", "9090", "", "0.0909") insert into trades values ("USD", "1279556653", "Y", "BTC", "limit", "9", "8000000000", "80", "9307", "", "0.09307") insert into trades values ("USD", "1279559013", "Y", "BTC", "limit", "10", "10000000000", "100", "8911", "", "0.08911")
and the tail insert into trades values ("USD", "1367047706", "Y", "BTC", "limit", "1367047706332857", "9980000", "0.0998", "13683040", "ask", "136.8304") insert into trades values ("USD", "1367047791", "Y", "BTC", "limit", "1367047791768232", "25712723", "0.25712723", "13688009", "bid", "136.88009") insert into trades values ("USD", "1367047804", "Y", "BTC", "market", "1367047804704062", "80186290", "0.8018629", "13682010", "ask", "136.8201") insert into trades values ("USD", "1367047805", "Y", "BTC", "limit", "1367047805062520", "79726342", "0.79726342", "13688009", "bid", "136.88009") insert into trades values ("USD", "1367047825", "Y", "BTC", "limit", "1367047825872682", "497020000", "4.9702", "13682020", "ask", "136.8202") insert into trades values ("USD", "1367047826", "Y", "BTC", "limit", "1367047826181247", "500000000", "5", "13682000", "ask", "136.82") insert into trades values ("USD", "1367047853", "Y", "BTC", "limit", "1367047853175671", "502980000", "5.0298", "13682000", "bid", "136.82") insert into trades values ("USD", "1367047879", "Y", "BTC", "market", "1367047879486765", "800000000", "8", "13686000", "bid", "136.86") insert into trades values ("USD", "1367047879", "Y", "BTC", "market", "1367047879775307", "600000000", "6", "13688000", "bid", "136.88") insert into trades values ("USD", "1367047879", "Y", "BTC", "market", "1367047879824265", "122345236", "1.22345236", "13688009", "bid", "136.88009")
Would be cool to link up with pyalgotrade.
|
|
|
hy i started to look into goxbot and i must admit, i dont get anything, but anyway i will reach the goal of my own little bot. ![Wink](https://bitcointalk.org/Smileys/default/wink.gif) plz help! in my strategy module i got (copied most of it of the 50/50-balance boot) import strategy import goxapi
class Strategy(strategy.Strategy):
"""a protfolio rebalancing bot""" def __init__(self, gox): strategy.Strategy.__init__(self, gox) self.temp_halt = False
def slot_before_unload(self, _sender, _data): pass
def slot_keypress(self, gox, (key)): if key == ord("b"): self.debug("canceling all rebalancing orders")
If key "b" is pressed goxtool displays: someone pressed key b, But it doesnt display "canceling all rebalancing orders". so, how to write messages to the log output? btw: is there an easy way to get rid of the Winconsole output in the main window? I am running tail on the log in a second window, so its just superfluid for me? Your strategy file works perfectly though it won't pay the bills. Make sure you are loading it. e.g. python ./goxtool.py --protocol=websocket --strategy=_test.py And you should see the load and unload in goxtool.log 2013-05-01 14:47:32,433:DEBUG:_test.Strategy:_test.Strategy loaded ... 2013-05-01 14:47:38,099:DEBUG:_test.Strategy:canceling all rebalancing orders ... 2013-05-01 14:47:40,974:DEBUG:_test.Strategy:_test.Strategy unloaded
|
|
|
EDIT: MtQuid tried your code and its working perfectly. Im sending you a small tip.
Tip recieved. Thanks hugolp ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif)
|
|
|
EDIT: Also, can anyone give me a hint on how to create a separate debug file for my bot? The regular debug goxtool file gets too big and its very cumbersome to check.
To reduce debug clutter for writing strategies I use a crafty tail. I did have a cut on the end but it lagged tail -f goxtool.log| grep -E 'Strat|Trace|^[^0-9]'
Yeah, thats an option, but I still would preffer to have a different file so I can check it from time to time. And how are the results?.
I just loaded the chart along with current data from bitcoin charts and they are pretty similar. http://bitcoincharts.com/charts/chart.json?m=mtgoxUSD&r=1&i=15-minI don't know what weighted moving average function they are using but the data is close enough for my use though I probably should check it out on a few more days before I invest the yacht ![Cheesy](https://bitcointalk.org/Smileys/default/cheesy.gif) If you use this data you will need to alter the timestamp for HighStock.js add on 000 (three zeros) EDIT: I haven't checked for the 1 hour EMAs but will do some more testing and experimenting later. Using the TA-Lib you can specify the timeperiod and then ignore those first timeperiod output values as the results settle down. I've used the default values from bitcoincharts. Simple at 10 and Weighted at 25. Thanks for the direction, I will try using ta-lib between today and tomorrow, hopefully it gives better results that my manually created one (which its not difficult...). I just put this together. This will create a log file for each loaded strategy as well as sending log messages to the main log. Maybe prof7bit could check it over and put it into the main source. It's a quick hack but it works. And you call self.debug() just like before and there is a new function self.info() for different log level. Change Strategy.py to look like this. """ trading robot breadboard """
import goxapi import logging
class Strategy(goxapi.BaseObject): # pylint: disable=C0111,W0613,R0201
def __init__(self, gox): goxapi.BaseObject.__init__(self) self.signal_debug.connect(gox.signal_debug) gox.signal_keypress.connect(self.slot_keypress) gox.signal_strategy_unload.connect(self.slot_before_unload) gox.signal_ticker.connect(self.slot_tick) gox.signal_depth.connect(self.slot_depth) gox.signal_trade.connect(self.slot_trade) gox.signal_userorder.connect(self.slot_userorder) gox.orderbook.signal_owns_changed.connect(self.slot_owns_changed) gox.history.signal_changed.connect(self.slot_history_changed) gox.signal_wallet.connect(self.slot_wallet_changed) self.gox = gox self.name = "%s.%s" % \ (self.__class__.__module__, self.__class__.__name__)
self.logger = logging.getLogger(self.name) self.fh = logging.FileHandler(self.name + '.log') formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s') self.fh.setFormatter(formatter) self.fh.setLevel(logging.DEBUG) self.logger.addHandler(self.fh)
self.info("%s loaded" % self.name)
def info(self, *args): """write info event to log. use our logger then send to the base logger""" msg = " ".join([str(x) for x in args]) self.logger.info(msg) goxapi.BaseObject.debug(self, *args)
def debug(self, *args): """write debug event to log. use our logger then send to the base logger""" msg = " ".join([str(x) for x in args]) self.logger.debug(msg) goxapi.BaseObject.debug(self, *args)
def __del__(self): """the strategy object will be garbage collected now, this mainly only exists to produce the log message, so you can make sure it really garbage collects and won't stay in memory on reload. If you don't see this log mesage on reload then you have circular references""" self.debug("%s unloaded" % self.name) self.logger.removeHandler(self.fh)
def slot_before_unload(self, _sender, _data): """the strategy is about to be unloaded. Use this signal to persist any state and also use it to forcefully destroy any circular references to allow it to be properly garbage collected (you might need to do this if you instantiated linked lists or similar structures, the symptom would be that you don't see the 'unloaded' message above.""" pass
def slot_keypress(self, gox, (key)): """a key in has been pressed (only a..z without "q" and "l")""" self.debug("someone pressed the %s key" % chr(key))
def slot_tick(self, gox, (bid, ask)): """a tick message has been received from the streaming API""" pass
def slot_depth(self, gox, (typ, price, volume, total_volume)): """a depth message has been received. Use this only if you want to keep track of the depth and orderbook updates yourself or if you for example want to log all depth messages to a database. This signal comes directly from the streaming API and the gox.orderbook might not yet be updated at this time.""" pass
def slot_trade(self, gox, (date, price, volume, typ, own)): """a trade message has been received. Note that this signal comes directly from the streaming API, it might come before orderbook.owns list has been updated, don't rely on the own orders and wallet already having been updated when this is fired.""" pass
def slot_userorder(self, gox, (price, volume, typ, oid, status)): """this comes directly from the API and owns list might not yet be updated, if you need the new owns list then use slot_owns_changed""" pass
def slot_owns_changed(self, orderbook, _dummy): """this comes *after* userorder and orderbook.owns is updated already. Also note that this signal is sent by the orderbook object, not by gox, so the sender argument is orderbook and not gox. This signal might be useful if you want to detect whether an order has been filled, you count open orders, count pending orders and compare with last count""" pass
def slot_wallet_changed(self, gox, _dummy): """this comes after the wallet has been updated. You can access the new balance like so: gox.wallet["BTC"] or gox.wallet[gox.currency]""" pass
def slot_history_changed(self, history, _dummy): """this is fired whenever a new trade is inserted into the history, you can also use this to query the close price of the most recent candle which is effectvely the price of the last trade message. Contrary to the slot_trade this also fires when streaming API reconnects and re-downloads the trade history, you can use this to implement a stoploss or you could also use it for example to detect when a new candle is opened""" pass
EDIT1: Changed formatter to formatter = logging.Formatter('%(asctime)s:%(levelname)s:%(message)s') EDIT2: Changed call to base debug to use *arg instead of arg so is now goxapi.BaseObject.debug(self, *args) No more edits. Code is flawless ![Cheesy](https://bitcointalk.org/Smileys/default/cheesy.gif) EDIT3: Need to remove file handler on strategy unload so that it is not reloaded twice or more during a strategy reload I think that should do it now. Code is never flawless Price is also going down.... Time for tea
|
|
|
MtQuid... Would you be willing to share a basic demo of the Ta-lib integration? I do not live in python so it would be very helpful for me, and I am sure others.
Ok. We will use TA-Lib for bython from these lovely chaps http://mrjbq7.github.io/ta-lib/index.htmlAssuming you are working on a debian system you will need to do the following to install ta-lib for python If on windows or mac then check http://mrjbq7.github.io/ta-lib/install.htmlsudo apt-get install python-dev sudo easy_install cython
wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz tar -xvf ta-lib-0.4.0-src.tar.gz cd ta-lib ./configure --prefix=/usr make
#can use make install here if you want sudo checkinstall
sudo easy_install TA-Lib
Once you have TA-Lib installed this is your strategy """ _talib.py
This example strategy will produce some HighStock.js compatible JSON files """
import numpy as np import talib import datetime import json import strategy import goxapi from goxapi import OHLCV
class OHLCV_Encoder(json.JSONEncoder): """JSONEncoder for class OHLCV()""" def default(self, obj): if isinstance(obj, OHLCV): return [obj.tim, obj.opn, obj.hig, obj.low, obj.cls, obj.vol] else: return json.JSONEncoder.default(self, obj)
class Strategy(strategy.Strategy): """a protfolio rebalancing bot""" def __init__(self, gox): strategy.Strategy.__init__(self, gox)
def slot_history_changed(self, history, _dummy): """History has changed so recalculate EMAs""" candles = []
# read them all - don't wory about the history parameter for c in reversed(self.gox.history.candles): candles.append( OHLCV( # fix time for HighStock.js JSON import c.tim * 1000, # adjust values to be human readable goxapi.int2float(c.opn, self.gox.currency), goxapi.int2float(c.hig, self.gox.currency), goxapi.int2float(c.low, self.gox.currency), goxapi.int2float(c.cls, self.gox.currency), goxapi.int2float(c.vol, "BTC") ) )
self.debug("New EMAs from history with %d candles" % len(candles))
rng = range(len(candles)) iterable = (candles[i].opn for i in rng) a_opn = np.fromiter(iterable, np.float)
iterable = (candles[i].hig for i in rng) a_hig = np.fromiter(iterable, np.float) iterable = (candles[i].low for i in rng) a_low = np.fromiter(iterable, np.float) iterable = (candles[i].cls for i in rng) a_cls = np.fromiter(iterable, np.float)
iterable = (candles[i].vol for i in rng) a_vol = np.fromiter(iterable, np.float)
iterable = (candles[i].tim for i in rng) a_tim = np.fromiter(iterable, np.int64)
a_sma = talib.SMA(a_cls, 10) a_wma = talib.WMA(a_cls, 25) a_chaikin = talib.AD(a_hig, a_low, a_cls, a_vol) a_cdlCounterAttack = talib.CDLCOUNTERATTACK(a_opn, a_hig, a_low, a_cls)
# create json compatible with HighStock.js with open("talib_ohlcv.json", 'w') as outfile: json.dump(candles, outfile, cls = OHLCV_Encoder) with open("talib_sma.json", 'w') as outfile: # first 10 elements contain Nan json.dump(np.dstack((a_tim[10:], a_sma[10:])).tolist()[0], outfile) with open("talib_wma.json", 'w') as outfile: # first 25 elements contain Nan json.dump(np.dstack((a_tim[25:], a_wma[25:])).tolist()[0], outfile) with open("talib_chaikin.json", 'w') as outfile: json.dump(np.dstack((a_tim, a_chaikin)).tolist()[0], outfile) with open("talib_cdlCounterAttack.json", 'w') as outfile: json.dump(np.dstack((a_tim, a_cdlCounterAttack)).tolist()[0], outfile)
# Current price in relation to EMA self.debug("SMA = %f" % a_sma[-1]) self.debug("WMA = %f" % a_wma[-1]) self.debug("CLS = %f" % a_cls[-1])
Run goxtool like so python ./goxtool.py --protocol=websocket --strategy=_talib.py
Create index.html like this <!DOCTYPE HTML> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>BTC Highstock - Goxtool/TA-Lib</title> <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script> <script type="text/javascript" src="http://code.highcharts.com/stock/highstock.js"></script> <script type="text/javascript" src="http://code.highcharts.com/modules/exporting.js"></script> <script type="text/javascript"> $(function () { var seriesOptions = [], ohlc = [], volume = [] yAxisOptions = [], seriesCounter = 0, names = ['OHLCV', 'SMA', 'WMA', 'chaikin'], colors = Highcharts.getOptions().colors, groupingUnits = [ ['week', [1]], ['month', [1, 2, 3, 4, 6]] ];
$.each(names, function (i, name) { $.getJSON('/talib_' + name.toLowerCase() + '.json', function (data) { console.log("Loading " + name); if (name == 'OHLCV') { // split the data set into ohlc and volume var dataLength = data.length; for (i = 0; i < dataLength; i++) { ohlc.push([ data[i][0], // the date data[i][1], // open data[i][2], // high data[i][3], // low data[i][4] // close ]);
volume.push([ data[i][0], // the date data[i][5] // the volume ]) } } else if (name == 'chaikin') { seriesOptions[i] = {
yAxis: 2, name: name, data: data };
} else { seriesOptions[i] = { name: name, data: data }; }
// As we're loading the data asynchronously, we don't know what order it will arrive. So // we keep a counter and create the chart when all the data is loaded. seriesCounter++;
if (seriesCounter == names.length) { createChart(); } }); });
// create the chart when all data is loaded
function createChart() { console.log("Creating chart");
$('#container').highcharts('StockChart', { credits: { enabled: false }, chart: { renderTo: container, backgroundColor: { linearGradient: [0, 0, 0, 500], stops: [ [0, 'rgb(255, 255, 255)'], [1, 'rgb(215, 229, 240)'] ] } }, rangeSelector: { buttons: [{ type: 'day', count: 1, text: '1d' }, { type: 'week', count: 1, text: '1w' }, { type: 'month', count: 1, text: '1m' } ], selected: 0, inputEnabled: false },
title: { text: 'BTC Historical - Goxtool/TA-Lib' },
yAxis: [{ title: { text: 'OHLC' }, type: 'logarithmic', height: 200, lineWidth: 2 }, { title: { text: 'Volume' }, type: 'linear', top: 300, height: 100, offset: 0, lineWidth: 2 }, { title: { text: 'Chakin' }, type: 'linear', top: 450, height: 100, offset: 0, lineWidth: 2 } ],
plotOptions: { candlestick: { color: '#DD2F29', upColor: '#87DA3B', lineColor: '#666' } },
tooltip: { pointFormat: '<span style="color:{series.color}">{series.name}</span>: <b>{point.y}</b><br/>', valueDecimals: 2 },
series: [{ type: 'candlestick', name: 'BTC', data: ohlc, dataGrouping: { units: groupingUnits } }, { type: 'column', name: 'Volume', data: volume, yAxis: 1, dataGrouping: { units: groupingUnits } }, seriesOptions[1], seriesOptions[2], seriesOptions[3] ] });
}
}); </script> </head> <body> <div id="container" style="height: 1000px; min-width: 600px"></div> </body> </html>
And run a simple web server like this python -m SimpleHTTPServer 8080
Then go to http://localhost:8080/index.html and see sunshine You could have the index.html auto reload with simple script or refresh with your mouse ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) EDIT: Also, can anyone give me a hint on how to create a separate debug file for my bot? The regular debug goxtool file gets too big and its very cumbersome to check.
To reduce debug clutter for writing strategies I use a crafty tail. I did have a cut on the end but it lagged tail -f goxtool.log| grep -E 'Strat|Trace|^[^0-9]'
And how are the results?.
I just loaded the chart along with current data from bitcoin charts and they are pretty similar. http://bitcoincharts.com/charts/chart.json?m=mtgoxUSD&r=1&i=15-minI don't know what weighted moving average function they are using but the data is close enough for my use though I probably should check it out on a few more days before I invest the yacht ![Cheesy](https://bitcointalk.org/Smileys/default/cheesy.gif) If you use this data you will need to alter the timestamp for HighStock.js add on 000 (three zeros) EDIT: I haven't checked for the 1 hour EMAs but will do some more testing and experimenting later. Using the TA-Lib you can specify the timeperiod and then ignore those first timeperiod output values as the results settle down. I've used the default values from bitcoincharts. Simple at 10 and Weighted at 25.
|
|
|
Yep they do look good. I robbed some of the colour styling straight from MtGox.com I only wanted the graph output to make sure the ta-lib functions were returning the correct values so I could use them to control my bot. I don't think this functionality needs to be part of the main goxtool branch but a good strategy plugin using ta-lib would be handy. Like providing values and firing signals that can be caught. And because the data sets are small (1 days worth of trade) you could write the whole index.html file to include the data and then there would be no need for the .json files or a web server. Just pops out a stand alone html page every now and again if that's what you require. I've also moved my function so it automatically gets called when new history arrives. def slot_history_changed(self, history, _dummy): """History has changed so recalculate EMAs"""
Now to try and find the best usage for the ta-lib functions and make some coins.. ![Grin](https://bitcointalk.org/Smileys/default/grin.gif)
|
|
|
@MtQuid
How far back does that 'historical' day pull from? Since GoxTool started?
It's using the same history candles that prof7bit uses for the trade graph. So it goes back a day in 15 minute blocks. Though you can change the history_timeframe value in goxtool.ini to change the block size.
|
|
|
I'm not very interested by you "speaker" strategy ... for now... but maybe one day it could be useful... I tried to play with a Python wrapper for TA-Lib http://ta-lib.org/http://mrjbq7.github.io/ta-lib/this wrapper is total... you just need to install cython, TA-Lib, and this wrapper I just tried this out and wanted to plot the results so I dumped the data to some files and used Highstock.js to render the view. import goxapi from goxapi import OHLCV import datetime import numpy as np import talib import json
class OHLCV_Encoder(json.JSONEncoder): """JSONEncoder for class OHLCV()""" def default(self, obj): if isinstance(obj, OHLCV): return [obj.tim, obj.opn, obj.hig, obj.low, obj.cls, obj.vol] else: return json.JSONEncoder.default(self, obj)
class Strategy(goxapi.BaseObject): ........ ........
def slot_keypress(self, gox, (key)): if key == ord("t"): """testing talib""" candles = []
for c in reversed(self.gox.history.candles): candles.append( OHLCV( # fix time for HighStock.js JSON import c.tim * 1000, # adjust values to be human readable goxapi.int2float(c.opn, self.gox.currency), goxapi.int2float(c.hig, self.gox.currency), goxapi.int2float(c.low, self.gox.currency), goxapi.int2float(c.cls, self.gox.currency), goxapi.int2float(c.vol, "BTC") ) )
self.debug("Test history with %d candles" % len(candles))
rng = range(len(candles)) iterable = (candles[i].cls for i in rng) a_cls = np.fromiter(iterable, np.float) iterable = (candles[i].tim for i in rng) a_tim = np.fromiter(iterable, np.int64)
a_sma = talib.SMA(a_cls, 10) a_wma = talib.WMA(a_cls, 25)
# create json compatible with HighStock.js with open("talib_ohlcv.json", 'w') as outfile: json.dump(candles, outfile, cls = OHLCV_Encoder) with open("talib_sma.json", 'w') as outfile: # first 10 elements contain Nan json.dump(np.dstack((a_tim[10:], a_sma[10:])).tolist()[0], outfile) with open("talib_wma.json", 'w') as outfile: # first 25 elements contain Nan json.dump(np.dstack((a_tim[25:], a_wma[25:])).tolist()[0], outfile)
with <!DOCTYPE HTML> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> <title>BTC Highstock - Goxtool/TA-Lib</title> <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script> <script type="text/javascript" src="http://code.highcharts.com/stock/highstock.js"></script> <script type="text/javascript" src="http://code.highcharts.com/modules/exporting.js"></script> <script type="text/javascript"> $(function () { var seriesOptions = [], ohlc = [], volume = [] yAxisOptions = [], seriesCounter = 0, names = ['OHLCV', 'SMA', 'WMA'], colors = Highcharts.getOptions().colors, groupingUnits = [ ['week', [1]], ['month', [1, 2, 3, 4, 6]] ];
$.each(names, function (i, name) { $.getJSON('/talib_' + name.toLowerCase() + '.json', function (data) { console.log("Loading " + name); if (name == 'OHLCV') { // split the data set into ohlc and volume var dataLength = data.length; for (i = 0; i < dataLength; i++) { ohlc.push([ data[i][0], // the date data[i][1], // open data[i][2], // high data[i][3], // low data[i][4] // close ]);
volume.push([ data[i][0], // the date data[i][5] // the volume ]) } } else { seriesOptions[i] = { name: name, data: data }; }
// As we're loading the data asynchronously, we don't know what order it will arrive. So // we keep a counter and create the chart when all the data is loaded. seriesCounter++;
if (seriesCounter == names.length) { createChart(); } }); });
// create the chart when all data is loaded
function createChart() { console.log("Creating chart");
$('#container').highcharts('StockChart', { credits: { enabled: false }, chart: { renderTo: container, backgroundColor: { linearGradient: [0, 0, 0, 500], stops: [ [0, 'rgb(255, 255, 255)'], [1, 'rgb(215, 229, 240)'] ] } }, rangeSelector: { buttons: [{ type: 'day', count: 1, text: '1d' }, { type: 'week', count: 1, text: '1w' }, { type: 'month', count: 1, text: '1m' } ], selected: 0, inputEnabled: false },
title: { text: 'BTC Historical - Goxtool/TA-Lib' },
yAxis: [{ title: { text: 'OHLC' }, type: 'logarithmic', height: 200, lineWidth: 2 }, { title: { text: 'Volume' }, type: 'linear', top: 300, height: 100, offset: 0, lineWidth: 2 } ],
plotOptions: { candlestick: { color: '#DD2F29', upColor: '#87DA3B', lineColor: '#666' } },
tooltip: { pointFormat: '<span style="color:{series.color}">{series.name}</span>: <b>{point.y}</b><br/>', valueDecimals: 2 },
series: [{ type: 'candlestick', name: 'BTC', data: ohlc, dataGrouping: { units: groupingUnits } }, { type: 'column', name: 'Volume', data: volume, yAxis: 1, dataGrouping: { units: groupingUnits } }, seriesOptions[1], seriesOptions[2] ] });
}
}); </script> </head> <body> <div id="container" style="height: 500px; min-width: 600px"></div> </body> </html>
Serve index.html up with python -m SimpleHTTPServer 8080 EDIT: Using close instead of open for EMAs and added python server example
|
|
|
+1 for 2.6 Debian squeeze has 2.5, 2.6 and 3.0 in repo.
I just installed goxtool on a squeeze box and I made this script to help anyone else in the same situation. # Installing python2.7 on debian stable and running goxtool # Tested on virgin debian squeeze install with sudo installed
#################################################### # Part 1: Install python2.7
# First get some tools sudo apt-get install build-essential # and some libs sudo apt-get install libsqlite3-dev zlib1g-dev libncurses5-dev libgdbm-dev libbz2-dev libreadline5-dev libssl-dev libdb-dev # Optional bits and bobs #sudo apt-get build-dep python
# starting at home cd ~
# Get and compile Python-2.7.4 wget wget http://python.org/ftp/python/2.7.4/Python-2.7.4.tar.bz2 tar xf Python-2.7.4.tar.bz2 cd Python-2.7.4/ ./configure --prefix=/usr/local make
# This make takes about 5 mins on an old virtual machine
# Now you have 2 options: # 1. install the new python # sudo make altinstall # 2. create a new debian package and install python 2.7 into debian package system # sudo checkinstall make altinstall
# I prefer checkinstall, though it will take longer sudo apt-get install checkinstall
# There is a bug and checkinstall will fail because some directories could not be created. # So just create them manually right now. sudo mkdir -p /usr/local/lib/pkgconfig sudo mkdir -p /usr/local/lib/python2.7/config
# Install and create the debian package sudo checkinstall --pkgname=python2.7 --pkgversion=2.7.4 make altinstall
# checkinstall took about 45 minutes on my slow VM
# have a look at the new python python2.7 --version # and still we have the original 2.6 as the system wide default python --version
# get back to home cd ~
# install a python package manager # this will install the script 'easy_install-2.7' wget https://pypi.python.org/packages/source/d/distribute/distribute-0.6.36.tar.gz tar xvf distribute-0.6.36.tar.gz cd distribute-0.6.36/ sudo python2.7 setup.py install
#################################################### # Part 2: Install goxtool
# get back to home cd ~
# install a means to segregate python applications sudo easy_install-2.7 virtualenv
# get latest goxtool git clone https://github.com/prof7bit/goxtool.git
# create a new python2.7 environment for goxtool virtualenv-2.7 --distribute goxtool
# switch to the new environment source goxtool/bin/activate
# check it worked - you should see 2.7.4 as the version number python --version
# get the packages required for goxtool easy_install-2.7 pycrypto
# fire her up and get trading python goxtool/goxtool.py --protocol=socketio --use-http
I've only just discovered this project with it's balance bot and I love it and have been running it for the last day and a half on the 7% default. I'm going to try some different margins on other accounts with like half a coin balance and see how they each perform over time. My first trading bot. One issue. During setup I balanced the account and then hit 'p' to place the buy/sell orders but I think I went too fast because the price was around 123 and the bot miscalculated my center and put in buys and sells at 80/90 something. MtGox ignored the low sell but I did panic. Also the graph is using a lot of screen space so could do with a toggle for switching between other information sets. I'm a C++ programmer by trade but I'm keen to help out with this projects so I'm currently looking at the code. Graph area could switch between log, graph, more detailed order book stats, strategy provided info. Strategy provided would be the obvious solution to all as then you could have empty strategies that only provide information screens. Then just have a mechanism to cycle through them. So even the current graph would end up in a strategy. Thanks for your work prof7bit.
|
|
|
Just a pastry, heh. Actually my name has origins in Arabic.
I am still curious. How would Mt. Gox confirm that I'm the real owner?
Collect every bit of information relating to your dealings with the account and use them to prove who you are. If you made payments into the account and you can send MtGox the verification documents to prove who you are then that would help. Did you make any deposits or withdrawls? and what methods did you use? and what days? How old is your account? How often do you use it? Where do you access it from? What browser do you usually use to access it? Did you use the same PC to access your account all the time? If so then it is linked by the IP address of your PC. This can change when you reset your router but it will change to within a range. Go to the url http://mtgox.com/this_is_danish_and_my_username_is_xxx_help_pox.html this will appear in their web logs and might match the IP address you last logged in with. Do it. The villains will be able to look through your email and trade history so you need to find things that are not listed in those places. Bank accounts. Letters to your home. Photo ID, Locations of access. Maybe you are the villain and you want to prove you own someone else's account or steal another persons. I don't know.
|
|
|
Do you live in a caravan?
|
|
|
How do you install 2 factor authentication if you have a Windows PC?
If you know how to use git and have java installed then pull the repo from https://github.com/mclamp/JAuth.gitOtherwise there is a windows installer in the downloads section https://github.com/mclamp/JAuth/downloadsI'm not on windows so don't know how that will work out for you but it should be ok. There are probably other implementations around but I've used this one on MtGox without problems. You then create a file with the secret key in and launch the program with the filename as a parameter, or you can even launch with the key as a parameter. Might be different on windows but I doubt it. MtGox requires a correct two factor login code using a key before you can enable the two factor login process. In this way if you have managed to enable two factor login then you know you are generating the right keys. Remember to back the secret key up on another machine or device or even a piece of paper, it's not long.
|
|
|
|