goxtool bot: portfolio rebalancing

[whizter]

Nice, never heard of screen. thanks for the info, it's working now, even if I close the ssh client.

[ErebusBat]

Nice, never heard of screen. thanks for the info, it's working now, even if I close the ssh client.

I have some scripts I wrote to help manage mine to make sure it is launched in screen (so I don't forget then close the session and think the bot is running):

Code:

$ cat botwindow.sh
#!/bin/sh
screen -D -R -A -S bot

$ cat ./startbot.sh
#!/bin/zsh
if [[ -z $STY ]] then
	echo "Not in screen! Run ~/botwindow.sh"
else
	cd ~/goxtool
	./bot_balancer.sh
fi

So I can login and just type ./bot[tab] (botwindow.sh) and be connected to goxtool.  If it isn't running I can THEN run ./startbot.sh which will check to make sure that I am running in screen and then launch the bot.

The screen flags:

-D detach other instances if needed
-R reattach to this instance if it already exists
-A Adapt the window size to the new client... i.e. change the window size between my laptop and ipad as I switch back and forth.
-S name the session 'bot'.  This allows me to create other screen sessions without affecting my scripts.

[c0inbuster]

I also forget that I also add

Code:

screen -r

to my

Code:

~/.bashrc

so when I connect through ssh
I always reattach my screen (I only use only one screen with several "windows")

[sturle]

It will do so by placing limit orders above and below current price that will restore the 50/50 ratio once price moves there and fills the order. As soon as one order is filled it will cancel the other and then calculate and place 2 new orders above and below that new price.

This is a good bot!  I don't know if the strategy is good or bad (probably among the better), but it provides liquidity on both sides of the spread.  This helps stabilizing the market price, reducing the amplitude of the price movements.  A more stable price will help adoption of Bitcoin as a currency, which is good for the price in the long run.

Many bots spend too much time trying to analyze the market in every possible way, getting it wrong 50% of the time.  I have a plain stupid no nonsense bot myself, and it is very profitable.  It's intelligence is on the same level as this bot, and it is a slow market maker bot like this one.  I wrote it in two hours in perl.  The strategy is different, but just as mechanical.  Stupid strategies often work best.

My strategy for surviving DDoS attacks on MtGox is to just cancel all orders as soon as lag > 60 seconds, and sit in the corner and wait until lag is back to < 1 second.  This far from perfect, but so far the best I've found.

[c0inbuster]

I have some philosophical questions:

Is it good for us if it's good for market ?
or is it good for us when it's bad for market ?

Should we help to improve usage of BTC as currency (by stabilizing price
instead of taking advantage of the price change) ?

and a question of a more greedy geek
Is it profitable ?

[K1773R]

I have some philosophical questions:

Is it good for us if it's good for market ?
or is it good for us when it's bad for market ?

Should we help to improve usage of BTC as currency (by stabilizing price
instead of taking advantage of the price change) ?

the more volatile the market, the more income

[c0inbuster]

this is also the point of view of an amateur FX trader like me...
but it could be quite different here

moreover it will be profitable for us (only) if MtGox was paying interest on fiat currency (USD, EUR) deposit.
I don't know if they do it.

[K1773R]

you didnt seem to understand it, take a further look here:
http://en.wikipedia.org/wiki/Rebalancing_investments
http://en.wikipedia.org/wiki/Constant_proportion_portfolio_insurance

[BTC-engineer]

I didn't already look deep into the code, but from what I saw I can say it looks very clean to me. Good work prof7bit!

However, I've done a quick live test today and stopped it because of unexpected trading behavior.
I didn't found the time to track down the problem, nor I'm sure I will do it in the next time.

But I would like to write down the misbehavior. Maybe it helps others to improve this nice bot.  

After starting the bot-trading, the bot placed correctly two orders (+/- distance from middle).
After a few hours, mtgox could fill the higher order in one step, so there was only one open order on mtgox.
A few seconds later the bot correctly deleted the old open order.
So far so good, but now it goes strange.
The bot paced two new orders which were exactly the same like the first placement which were done hours ago.
Also from this placement the higher order was filled by mtgox (this time in two steps).
A few seconds later the bot correctly deleted the old open order.
But this time the bot placed two new (correct) orders. Which means the orders were correctly placed +/- distance from the middle.

[fnordfnordfnord]

got this running on my raspberry pi now for testing purposes with small amount of funds, and it's working as intended so far.
But I got a little problem: how can I run the bot in background, I can't fork the process to background using "&" because I won't be prompted for the decryption password, and if I close the SSH session the process gets killed.

Try this: After it is up and running, hit "Ctrl-z" and then type "bg" then press Enter.

[prof7bit]

The bot paced two new orders which were exactly the same like the first placement which were done hours ago.

This means it missed the wallet update message or it came too late. In my tests so far the wallet message always was the first message, then the trade message and finally the message to remove the order. Thats the reason I trigger the trading when the number of open orders changes (assuming that the wallet has been updated at this time already).

Maybe I need some more code to detect whether the wallet has really been updated already. Unfortunately there will arrive 3 wallet mesages, one for the USD update, one for the BTC update and a 3rd one for the fees. Might make the code a bit more complicated but still possible.

***

BTW: Today MtGox installed a new websocket server that is much more reliable than the socketio server. My recommendation for running goxtool are now:

./goxtool --protocol=websocket --use-http --strategy=_balancer.py

And no more --use-http as it was needed with socketio because sending orders over websocket is fast and reliable now.[Edit: turns out http is still needed, added it back to the above command line] The new websocket server should also reduce the risk of missing messages due to disconnects, actually I did not have even one single disconnect anymore the entire day since I switched to websocket when I heared about the new server this morning!

BTW: There is also an update for goxtool on github (fixes some problems with the handling of some error messages from the server and also fixing a problem when using use_ssl=False in combiation with websocket), please
git pull
the latest version of goxtool.

[BTC-engineer]

Hello prof7bit,

thanks for your feedback and help. I think I will give your bot a second try.

I have one short, general question to this bot strategy.
It looks to me like the bot doesn't consider the trading fee when calculating the new two distance orders points and the two new order volumes.
Have I missed something? And if not, does the consideration of the trading fee when calculating the new order points and volumes has only a insignificant, or even no effect on the efficiency of the strategy?

[prof7bit]

It looks to me like the bot doesn't consider the trading fee when calculating the new two distance orders points and the two new order volumes.

This is true. The fees are low enough to be not significant for the calculation of the new acount balance, so I just ignore them and if you choose the distance >> 1.2% there is also enough room for profit for every consecutive buy-sell-sequence.

[BTC-engineer]

Hi everybody

I've just done some back-testing of this trading strategy and would like to let you know about my results:

For this test I've written a small test-bench in C which tries to imitate this strategy as close as possible. I've stimulated my test-bench with historical data. I've used the mtgox ticker from the beginning of this year until now (5 minute ticker). So the strategy was tested with over 32K datapoints.

I've always put a specific number of virtuall bitcoins into this test and looked at the end with how much virtual bitcoins (bitcoins+(FIAT*price)) I ended up with.
I've done this with different trading fees and different DISTANCE values.

The result for this specific test setup was:
In all scenarios I ended up with only about 1/3 of my initial virtual bitcoins.
So I lost about 2/3 of my virtual bitcoins.  

Important notes:
1. I've carefully written the test-bench, but can not guarantee that I've made a mistake.
2. I know that the gox chart of the last month was quite crazy, and the result may look different if I would test with different data.

Have fun!
 

[prof7bit]

I've used the mtgox ticker from the beginning of this year until now (5 minute ticker). So the strategy was tested with over 32K datapoints.

In all scenarios I ended up with only about 1/3 of my initial virtual bitcoins.
So I lost about 2/3 of my virtual bitcoins.  

This is expected. And you haven't actually lost them, you have just sold them on the way up ;-)

If it trends so strongly then the most profitable strategy would buy and hold. But if it would have gone back to the price where your test started then you should end up with slightly more coins. Generally if it is going up it will slowly sell coins on the way up and when it goes down it will buy them all back again (at a slightly cheaper price).Try to choose a test period that has the same start and end price.

How did the USD balance develop in comparison? Could you upload some plots of USD, BTC and price?

[ErebusBat]

Would it also be worth calculating USD Worth (USD+(BTC*PRICE))  I have found that is a a better indicator with my personal results.

[ErebusBat]

Anyone have problems with hitting a sell point and the bot not canceling/placing new orders?

This happened twice to me today (although I can only find one instance in the log ):

Code:

2013-04-24 06:21:18,548:DEBUG:_balancer.Strategy:canceling all rebalancing orders
2013-04-24 06:21:19,663:DEBUG:goxapi.Gox:### got ack for order/cancel: 07e56a1e-abe2-4189-879f-f9daebe52473
2013-04-24 06:21:33,621:DEBUG:_balancer.Strategy:adding new initial rebalancing orders
2013-04-24 06:21:33,622:DEBUG:_balancer.Strategy:new buy order 0.043029 at 146.294770
2013-04-24 06:21:33,622:DEBUG:_balancer.Strategy:new sell order 0.015061 at 161.694270
2013-04-24 06:21:35,557:DEBUG:goxapi.Gox:### got ack for order/add: bid 14629477 4302918 530644ad-3734-42e9-946c-1c91ea79a43a
2013-04-24 06:21:37,455:DEBUG:goxapi.Gox:### got ack for order/add: ask 16169427 1506124 76ed7e56-50e2-4266-a993-39c458a5ec25

2013-04-24 08:22:34,714:DEBUG:_balancer.Strategy:*** sold 0.015061 at 161.694270
2013-04-24 10:10:49,760:DEBUG:_balancer.Strategy:canceling all rebalancing orders
2013-04-24 10:10:55,232:DEBUG:goxapi.Gox:### got ack for order/cancel: 530644ad-3734-42e9-946c-1c91ea79a43a
2013-04-24 10:10:59,359:DEBUG:_balancer.Strategy:adding new initial rebalancing orders
2013-04-24 10:10:59,360:DEBUG:_balancer.Strategy:new buy order 0.014208 at 157.676270
2013-04-24 10:10:59,361:DEBUG:_balancer.Strategy:new sell order 0.040420 at 174.273770
2013-04-24 10:11:05,381:DEBUG:goxapi.Gox:### got ack for order/add: bid 15767627 1420833 5d19afcc-bbfc-43e9-beb7-61393d16da93
2013-04-24 10:11:11,662:DEBUG:goxapi.Gox:### got ack for order/add: ask 17427377 4041987 32713cab-d754-4a23-8a74-4b24c1cb9446

Notice the ~2h gap between 08:22-10:10 where the bot sold and I manually canceled and readded the orders (c,p)  lag was not an issue, at 10:00 anyway.

I am on commit 2e615b9b6272ac99de88693b148f6aa8f2e41708 (respect use_ssl setting everywhere. don't retry on http 500 error) for the tool, and I *believe* the latest balancer bot (although it isn't in git so I can't tell for sure)  Starting with ./goxtool.py --protocol=websocket --strategy=_balancer.py.

Am I doing something wrong?

[BTC-engineer]

Would it also be worth calculating USD Worth (USD+(BTC*PRICE))  I have found that is a a better indicator with my personal results.

I personally do not calculate in USD anymore.
In this case the initial USD value increased nearly 4 times.

[supert]

Thank you prof7bit for providing this strategy example. I am using a variation of it. You will get a proportion of the profits (if any).

[ErebusBat]

Thought I would post a tweak I made to the bot to display your current wallet and last trade price in the "i" output.   This is nice for me because I can just hit "i" then copy/paste from my log into my tracking sheet.  I do not consider myself a python expert at all so I am sure there are more succinct ways of doing this.  Replace the entire "i" section.

Code:

        if key == ord("i"):
            price = (gox.orderbook.bid + gox.orderbook.ask) / 2
            vol_buy = self.get_buy_at_price(price)
            line1 = "BTC difference: " + goxapi.int2str(vol_buy, "BTC")            
            # Log current wallet
            if len(self.gox.wallet):
                line1 += "\t"
                for currency in self.gox.wallet:
                    line1 += currency + " " \
                    + goxapi.int2str(self.gox.wallet[currency], currency).strip() \
                    + " + "
                line1 = line1.strip(" +")
            # Log last price
            price = (self.gox.orderbook.ask + self.gox.orderbook.bid) / 2
            line1 += "\tLast Price: %f" % goxapi.int2float(price, self.gox.currency)
            
            # Print the whole thing out
            self.debug(line1)

Has output like:

Code:

2013-04-24 10:50:31,489:DEBUG:_balancer.Strategy:BTC difference:       0.01234567	USD 987.65432 + BTC 1.01234567	Last Price: 158.550000

I also have a log wrapper & rotator scripts I wrote to preserve and monitor the log, you can see them in the main thread here: https://bitcointalk.org/index.php?topic=148462.msg1932608#msg1932608