prof7bit (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 20, 2013, 05:38:00 PM Last edit: April 24, 2013, 07:19:35 PM by prof7bit |
|
I'm making separate threads for individual goxtool bots to keep the goxtool thread for goxtool related stuff only. I choose this forum because the goxtool thread was moved into this forum too. (Similar software sometimes gets moved into different forums, I believe there is a forum missing for all kinds of software that is bitcoin related but *not* a bitcoin client/wallet itself) And here is the balancer bot: _balancer.py: https://gist.github.com/prof7bit/5395900This bot will maintain a constant asset allocation ratio Bitcoin/Fiat = 50/50 http://en.wikipedia.org/wiki/Rebalancing_investments#Rebalancing_bonusIt will do so by placing limit orders above and below current price that will restore the 50/50 ratio once price moves there and fills the order. As soon as one order is filled it will cancel the other and then calculate and place 2 new orders above and below that new price. The price distance of the orders is 7%, that can be changed in the variable DISTANCE you should not make it smaller than 2%). This strategy will produce a small profit in the long run (a few percent per week if there are huge price swings or nothing at all when its going sideways for many days), it seems a promising strategy for BTC/USD because of its insanely high volatility but it will also make your account balance go up and down directly in tandem with BTC price, so prepared to see huge swings in your account. You should write down BTC-price, USD and BTC balance in your account once a week and plot these curves after a few months or years to see the effect. This is not a get rich quick bot, this needs patience! Installation and usage:- prerequisites: you are familiar with the usage of goxtool.py
- (most important) update goxtool to the latest version (git pull)
- save the file from the above link as _balancer.py in the same directory as goxtool
- ./goxtool.py --strategy=_balancer.py --protocol=websocket --use-http
- (optional) open a separate terminal and tail -f goxtool.log | grep Strategy to follow the debug output of the strategy module
- cancel all orders and bring your account into 50/50 balance, do so by pressing the b key
- start the bot by pressing the p key
Disclaimer: If the above sounds chinese to you or you cannot read the code or don't understand what it does or you are not a Linux user (needed for goxtool) then algo trading is not for you. You should have the skills to implement this yourself from scratch given the description of the strategy or be able to read, understand and debug every aspect of the existing code or it will only end with a huge disappointment or maybe even a desaster.As you can see in the code the the bot has some some hotkeys bound to certain actions to control its operations: i show info about current status (how much out of balance) c cancel all open rebalancing orders and suspend trading b rebalance immediately with market order at current price p place a pair of fresh rebalancing orders and start/resume trading u manually force update of own order list and other stuff (should only be needed when socketio connection behaves strange or is down or other related problems, not used for normal operations) like all other goxtool strategies it can be reloaded at runtime without restarting goxtool (for example after editing DISTANCE) by pressing the l key (lowercase L). As you also can see from the code it is triggering its trades when it has detected the number of orders being exactly 1, this does not happen in the trade message, the user_order message is the last API message that is sent after a series of messages updating the wallet and the trade message (and this signal is also triggered after reconnect), so this is the best place to trigger new orders, knowing that the wallet is updated already (we need the new account balance for the calculations). When the bot ever comes out of sync, I have seen this happening when price is jumping around too fast and you choose DISTANCE too small and mtgox is under ddos at the same time and the connection becomes really unreliable then you need to cancel all orders (press c to cancel and halt trading, also check with F6 that really all orders have been canceled, maybe use u to manually update if socketio is down and if its screwed up totally then restart goxtool. This should not happen very often) and then check if you need to rebalance (press i and see the log) and if needed manually rebalance (press b) and then resume automatic rebalancing (press p).
|
|
|
|
jubalix
Legendary
Offline
Activity: 2632
Merit: 1022
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 21, 2013, 06:02:26 AM Last edit: April 21, 2013, 06:48:46 AM by jubalix |
|
is there a debug mode/test mode where it can just hook into gox data, and i put in a test fiat /btc amount and see how it runs with mocked up fees and every thing.... I guess my worry is in spikey fast trading with bad connections sometimes computer programs behave in unexpected manners. I guess the other tool would be a face feed to simulate a jumpy/flaky gox feed.... then we would have a good test environs.... or is this already present? I'm making separate threads for individual goxtool bots to keep the goxtool thread for goxtool related stuff only. I choose this forum because the goxtool thread was moved into this forum too. (Similar software sometimes gets moved into different forums, I believe there is a forum missing for all kinds of software that is bitcoin related but *not* a bitcoin client/wallet itself) And here is the balancer bot: _balancer.py: https://gist.github.com/prof7bit/5395900This bot will maintain a constant asset allocation ratio Bitcoin/Fiat = 50/50 http://en.wikipedia.org/wiki/Rebalancing_investments#Rebalancing_bonusIt will do so by placing limit orders above and below current price that will restore the 50/50 ratio once price moves there and fills the order. As soon as one order is filled it will cancel the other and then calculate and place 2 new orders above and below that new price. The price distance of the orders is 7%, that can be changed in the variable DISTANCE you should not make it smaller than 2%). This strategy will produce a small profit in the long run (a few percent per week if there are huge price swings or nothing at all when its going sideways for many days), it seems a promising strategy for BTC/USD because of its insanely high volatility but it will also make your account balance go up and down directly in tandem with BTC price, so prepared to see huge swings in your account. You should write down BTC-price, USD and BTC balance in your account once a week and plot these curves after a few months or years to see the effect. This is not a get rich quick bot, this needs patience! Installation and usage:- prerequisites: you are familiar with the usage of goxtool.py
- (most important) update goxtool to the latest version (git pull)
- save the file from the above link as _balancer.py in the same directory as goxtool
- ./goxtool.py --strategy=_balancer.py --protocol=socketio --use-http
- (optional) open a separate terminal and tail -f goxtool.log | grep Strategy to follow the debug output of the strategy module
- cancel all orders and bring your account into 50/50 balance, do so by pressing the b key
- start the bot by pressing the p key
Disclaimer: If the above sounds chinese to you or you cannot read the code or don't understand what it does or you are not a Linux user (needed for goxtool) then algo trading is not for you. You should have the skills to implement this yourself from scratch given the description of the strategy or be able to read, understand and debug every aspect of the existing code or it will only end with a huge disappointment or maybe even a desaster.As you can see in the code the the bot has some some hotkeys bound to certain actions to control its operations: i show info about current status (how much out of balance) c cancel all open rebalancing orders and suspend trading b rebalance immediately with market order at current price p place a pair of fresh rebalancing orders and start/resume trading u manually force update of own order list and other stuff (should only be needed when socketio connection behaves strange or is down or other related problems, not used for normal operations) like all other goxtool strategies it can be reloaded at runtime without restarting goxtool (for example after editing DISTANCE) by pressing the l key (lowercase L). As you also can see from the code it is triggering its trades when it has detected the number of orders being exactly 1, this does not happen in the trade message, the user_order message is the last API message that is sent after a series of messages updating the wallet and the trade message (and this signal is also triggered after reconnect), so this is the best place to trigger new orders, knowing that the wallet is updated already (we need the new account balance for the calculations). When the bot ever comes out of sync, I have seen this happening when price is jumping around too fast and you choose DISTANCE too small and mtgox is under ddos at the same time and the connection becomes really unreliable then you need to cancel all orders (press c to cancel and halt trading, also check with F6 that really all orders have been canceled, maybe use u to manually update if socketio is down and if its screwed up totally then restart goxtool. This should not happen very often) and then check if you need to rebalance (press i and see the log) and if needed manually rebalance (press b) and then resume automatic rebalancing (press p).
|
|
|
|
smooth
Legendary
Offline
Activity: 2968
Merit: 1198
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 21, 2013, 06:41:11 AM |
|
I guess my worrie is in spikey fast tradeing with bad conections sometimes computer programs behace in unexpected manners.
There are definitely issues with lag, dropped gox connections, etc. Some of those are discussed on the main thread, as they aren't really specific to this strategy. Another rebalancing strategy discussed on the same wiki page might also be interesting to play with: http://en.wikipedia.org/wiki/Constant_proportion_portfolio_insurance
|
|
|
|
|
1Pakis
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 21, 2013, 08:37:44 AM |
|
I set DISTANCE at 5. I was balanced at 126.30 First trade went smoothly at 119.97 New orders opened
|
Tips are welcome at this address 18DVZkpSwmejPjekX3QMKvRRtR8Bfx65LN.
|
|
|
ErebusBat
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 21, 2013, 11:23:28 PM |
|
So... Did anyone else have the bot freak out and place 30+ orders because of GOX lag today?
|
|
|
|
jubalix
Legendary
Offline
Activity: 2632
Merit: 1022
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 01:41:06 AM |
|
So... Did anyone else have the bot freak out and place 30+ orders because of GOX lag today?
hmm i may have mentioned this.....need to make something like if(watch.goxLag()) { trade } goxLag() { if {last.ping() > choose interval return false} ...condition false ....condition true } etc
|
|
|
|
ErebusBat
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 02:33:39 AM |
|
So... Did anyone else have the bot freak out and place 30+ orders because of GOX lag today?
hmm i may have mentioned this.....need to make something like if(watch.goxLag()) { trade } goxLag() { if {last.ping() > choose interval return false} ...condition false ....condition true } etc It wasn't just lag tho. When I looked there was only like 60s. However I kept getting 503 errors all over the place. I think a nice feature (although far from full proof) would be to cancel all orders if there are >2 bot orders, then place them again. However it is still possible that some of those rouge orders get filled in the meantime.
|
|
|
|
whizter
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 07:28:18 AM |
|
I like the idea, is it also able to trade BTC/EUR ?
|
|
|
|
prof7bit (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 09:52:33 AM Last edit: April 22, 2013, 02:29:16 PM by prof7bit |
|
I think a nice feature (although far from full proof) would be to cancel all orders if there are >2 bot orders, then place them again. However it is still possible that some of those rouge orders get filled in the meantime.
I'm trying to understand wat might have happened to cause these symptoms, i don't have any idea yet. It should not trade if there are != 1 open orders, it should not trade if there are != 0 submitted but not yet acked orders. How did you start goxtool, did you use --use-http or did you send the orders via the streming API? I cannot imagine any scenario where sending an order would *not* increment gox.count_submitted *or* the number of own orders in orderbook.owns. Either the sent order doesn't get acked then count_submitted will be != 0 or there comes the ack with order id for each order then count_submitted = 0 but orderbook.owns has a new element added. In both cases it shoudl stop sending more orders. The only way I could vaguely imagine was if you did not --use-http and submitted via socketio, never received the ack and order ID, a reconnect would reset the count_submitted counter [Edit: actually only a *successful* download of own orders will reset count_submitted, not a simple reconnect] and because of ddos and 502 error you would never get the full own order list download but for some reason all the submitted orders (submitted through socketio and never acked) would later go through all at once [because they were hanging around somewhere on the server in some kind of separate (undocumented?) queue all the time]. Thats why I recommend --use-http because that way you *WILL* get an ack for every order under all circumstances because the http call will either succeed (ack) or fail. Are you running the latest version of goxtool.py and the latet version of the bot (3rd revision) from the github gist in post #1 of this thread or are you still using one of the older versions from the other thread? If this happens again please save the log file (*before* you restart goxtool because restarting will truncate the old log)
|
|
|
|
prof7bit (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 09:55:19 AM |
|
I like the idea, is it also able to trade BTC/EUR ?
Yes, just change the currency in goxtool.ini from USD to EUR.
|
|
|
|
c0inbuster
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 10:07:47 AM |
|
I don't like the idea of this .ini file because you can't have 2 goxtool instance running in 2 differents GNU screen (one for BTC/USD and an other one for BTC/EUR) (if you want to do this you will have to copy goxtool) maybe you should add flag to CLI or
|
|
|
|
c0inbuster
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 02:06:44 PM Last edit: April 22, 2013, 02:30:10 PM by c0inbuster |
|
an other idea could be to allow to pass config file as parameter like this python goxtool.py --config /path/to/goxtool.ini so it will be possible to have a screen with BTCUSD and an other one with BTCEUR **but** if your strategies are writing files it could be a problem because 2 differents goxtools are writing on same file... Let's say that you are implementing a trailing stop strategy... you will store stop loss price in a file (because you want not to lose value if goxtool is stopped and started) let's call sl.txt this file (ini, json or yaml file) maybe you should add a "name" parameter in goxtool.ini [gox] name = MyGox1 currency = USD use_ssl = True use_plain_old_websocket = False use_http_api = False load_fulldepth = True load_history = True history_timeframe = 15 secret_key = secret_secret =
[goxtool] set_xterm_title = True and strategy that writes files (to store data such as last stop price) will store data as MyGox1_sl.txt so differents goxtool could work together (english is not my mother tongue... so please excuse me if it's not very clear) Adding config parameter is quite easy to do in goxtool.py file argp.add_argument('--config', action="store", default="goxtool.ini", help="name of config file, default=goxtool.ini")
args = argp.parse_args()
config = goxapi.GoxConfig(args.config)
So you can now have a goxtoolBTCEUR.ini with [gox] currency = EUR use_ssl = True use_plain_old_websocket = False use_http_api = False load_fulldepth = True load_history = True history_timeframe = 15 secret_key = secret_secret =
[goxtool] set_xterm_title = True and call goxtool with this config file as parameter using python goxtool.py --config goxtoolBTCUSD.ini or with my Raspberry Pi running my Bitcoin/Litecoin softwares in GNU Screen screen -t goxtoolBTCUSD 50 python /home/pi/src/goxtool/goxtool.py --config /home/pi/src/goxtool/goxtoolBTCUSD.ini screen -t goxtoolBTCEUR 51 python /home/pi/src/goxtool/goxtool.py --config /home/pi/src/goxtool/goxtoolBTCEUR.ini
|
|
|
|
prof7bit (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 03:32:05 PM |
|
I don't like the idea of this .ini file
because you can't have 2 goxtool instance running in 2 differents GNU screen (one for BTC/USD and an other one for BTC/EUR)
I'm aware of the problem. And there is also the logfile that will be written by each running gox instance. I'm already thinking about possible solutions. Currently the simplest workaround is to either copy the entire folder or start it from within different cwd. This will improve eventually but at the moment I'm still finding new bugs in the client code (handling some erroneous conditions, missed messages, previously unknown error replies from mtgox API that result from various kinds of mtgox outage and other strange mtgox behavior that could happen) these kinds of bugfixes have highest priority.
|
|
|
|
c0inbuster
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 03:44:02 PM |
|
Thanks... maybe the "name" parameter in config file is a KISS solution (Keep it Simple, Stupid) (even for log file) good luck hunting bugs ![Wink](https://bitcointalk.org/Smileys/default/wink.gif)
|
|
|
|
K1773R
Legendary
Offline
Activity: 1792
Merit: 1008
/dev/null
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 03:49:14 PM |
|
I don't like the idea of this .ini file
because you can't have 2 goxtool instance running in 2 differents GNU screen (one for BTC/USD and an other one for BTC/EUR)
I'm aware of the problem. And there is also the logfile that will be written by each running gox instance. I'm already thinking about possible solutions. Currently the simplest workaround is to either copy the entire folder or start it from within different cwd. This will improve eventually but at the moment I'm still finding new bugs in the client code (handling some erroneous conditions, missed messages, previously unknown error replies from mtgox API that result from various kinds of mtgox outage and other strange mtgox behavior that could happen) these kinds of bugfixes have highest priority. then how about applying the patch i did send u? ![Tongue](https://bitcointalk.org/Smileys/default/tongue.gif)
|
[GPG Public Key]BTC/DVC/TRC/FRC: 1 K1773RbXRZVRQSSXe9N6N2MUFERvrdu6y ANC/XPM A K1773RTmRKtvbKBCrUu95UQg5iegrqyeA NMC: N K1773Rzv8b4ugmCgX789PbjewA9fL9Dy1 LTC: L Ki773RBuPepQH8E6Zb1ponoCvgbU7hHmd EMC: E K1773RxUes1HX1YAGMZ1xVYBBRUCqfDoF BQC: b K1773R1APJz4yTgRkmdKQhjhiMyQpJgfN
|
|
|
whizter
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 07:41:07 PM |
|
got this running on my raspberry pi now for testing purposes with small amount of funds, and it's working as intended so far. But I got a little problem: how can I run the bot in background, I can't fork the process to background using "&" because I won't be prompted for the decryption password, and if I close the SSH session the process gets killed.
|
|
|
|
arsenische
Legendary
Offline
Activity: 1199
Merit: 1012
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 07:45:12 PM |
|
got this running on my raspberry pi now for testing purposes with small amount of funds, and it's working as intended so far. But I got a little problem: how can I run the bot in background, I can't fork the process to background using "&" because I won't be prompted for the decryption password, and if I close the SSH session the process gets killed.
probably you can run it from screen
|
|
|
|
c0inbuster
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 07:48:29 PM |
|
You can use GNU screen run it using $ screen -t goxtool 51 python /home/pi/src/goxtool/goxtool.py
CTRL + a + " to change screen CTRL + a + c to create new screen If you want to run it at startup, I can say you what I did (there is probably a better solution) put in /home/pi/.screenrc screen -t goxtool 51 python /home/pi/src/goxtool/goxtool.py
and add su - pi -c "/usr/bin/screen -dmS rclocal" in your /etc/rc.local If you consider that what I'm doing for you is valuable you can send me some crypto-coins. https://sites.google.com/site/working4coins/donate
|
|
|
|
ErebusBat
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 22, 2013, 08:06:56 PM |
|
I think a nice feature (although far from full proof) would be to cancel all orders if there are >2 bot orders, then place them again. However it is still possible that some of those rouge orders get filled in the meantime.
I'm trying to understand wat might have happened to cause these symptoms, i don't have any idea yet. It should not trade if there are != 1 open orders, it should not trade if there are != 0 submitted but not yet acked orders. How did you start goxtool, did you use --use-http or did you send the orders via the streming API? I cannot imagine any scenario where sending an order would *not* increment gox.count_submitted *or* the number of own orders in orderbook.owns. Either the sent order doesn't get acked then count_submitted will be != 0 or there comes the ack with order id for each order then count_submitted = 0 but orderbook.owns has a new element added. In both cases it shoudl stop sending more orders. The only way I could vaguely imagine was if you did not --use-http and submitted via socketio, never received the ack and order ID, a reconnect would reset the count_submitted counter [Edit: actually only a *successful* download of own orders will reset count_submitted, not a simple reconnect] and because of ddos and 502 error you would never get the full own order list download but for some reason all the submitted orders (submitted through socketio and never acked) would later go through all at once [because they were hanging around somewhere on the server in some kind of separate (undocumented?) queue all the time]. Thats why I recommend --use-http because that way you *WILL* get an ack for every order under all circumstances because the http call will either succeed (ack) or fail. Are you running the latest version of goxtool.py and the latet version of the bot (3rd revision) from the github gist in post #1 of this thread or are you still using one of the older versions from the other thread? If this happens again please save the log file (*before* you restart goxtool because restarting will truncate the old log) I am using the version from this post: https://bitcointalk.org/index.php?topic=148462.msg1886914#msg1886914My startup script for the bot: ./goxtool.py --protocol=socketio --use-http --strategy=_balancer.py
OK.. perhaps it was user error then. I did notice that the bot did not get the updated orders OR the trade so I canceled the remaining single bot order and DID manually try to "P"lace orders multiple times. I didn't think it was that high, and I do believe the log indicated that it was also placing the orders itself. Go and put my son down for a nap... fell asleep with him and my gox history page is full of sells. It isn't a big deal... like you said things can and will go wrong (esp when dealing with the gox api under load), and in fact I was able to buy it back without loss so it wasn't a huge deal anyway. So it was very possible it was just PEBKAK.
|
|
|
|
|