Isis ATP [Automated Trading Platform] - Discussion

[Aido]

@Aido: Do you want me to just push code fixes as bugs on the tracker and you can push them into the repository?

For the time being this may be a good way to track suggestions that require code change. After discussion in this thread, a specific targeted task can be put in the issues page of github. It's just a suggestion to do it this way so, yea or nay?
Or a new forum can be created as suggested by new overcomer.

so you're saying all the market connection and data streaming is coming from a different project?

Yes, I think rather than re-inventing the wheel the original developer choose to use XChange so as to get IsisATP up and running a lot faster. If the XChange project no longer suits our purpose I suppose it could be forked on github and our own requirements implemented. However, looking at the planned milestones for XChange project, the developers plan to expand the code for other exchanges:

https://github.com/timmolter/XChange/wiki/Milestones

And as mentioned previously, one of the developers is going to investigate adding code for testing and reconnecting socketIO connections which is currently an issue for IsisATP:

https://github.com/timmolter/XChange/issues/6

Just to recap on a few things and explain some of the changes I made to the original.

The most fundamental change was to switch from a polling to streaming ticker. The reason I decided to use a streaming ticker was so that it would react faster to the market. Now, when a tick is received it triggers an arbitrage thread and a trend observer thread. Each making a decision on whether to trade or not. Another upshot of this was I found that the original IsisATP was using 100% of my CPU. By changing how the threads were launched I could use the thread.join() Java method rather than the thread.yield() method of the original which apparently can have negative effects on a Linux system.

I also made some changes to the learning functionality. Originally it learned for 1 hour and cleared out any ticks older than that time. So trade decisions were based on an hour of ticks that were polled every 1 minute. The learning time was hardcoded.

Now it learns for at least 16 streamed ticks and keeps data for 2 hours. So depending on how many ticks it received in that two hours it will have more data for a 'popular' currency like dollars. By coincidence, I think this change may make it a bit easier to implement some of the SMA functionality mentioned in the previous few posts. We'll see....

I think the first step is to make these values configurable from the preferences file (or the registry for you Windows fanboys! ).

I think maybe we should concentrate on core functionality and algorithms first before getting fancy with GUIs and the like. I'm an old-school command line type person anyway and always favour function over form.

[daybyter]

It really seems, these 2 project complement each other in some ways. I don't have a good trading code yet, but a gui and a trade site interface to connect to trade sites without any external services.

[scrybe]

Thanks Aido!

I look forward to more configuration options in the registry (preferences file for you Linux pukes )

edit: have you uploaded this build yet? your github page only has one from 7 days ago...

[Aido]

commit 39cbd305a93dad949ede0d173933de7599e16362

    Removed deprecated --debug-live command line parameter

Self explanatory


commit 19e615923ba6e1356af02bfb25a752e0a44597a7

    Added minTickSize and maxTickAge to config file

I look forward to more configuration options in the registry (preferences file for you enlightened Linux people)

I recommend using the --clear-config option to ceate a new configuration file with these additional settings. Also, a reiminder that changes to config file settings are read on-the-fly so IsisATP does not need to be stopped if the values are tweaked.


commit 9ca26ca85989b6c370ebfa5c4f7bced9a028a57c

    Renamed TradingAgent to TrendTradingAgent

In preparation for adding a new MovingAverageTradingAgent class

[scrybe]

I see what you did there...

Thanks dude!

[Aido]

OK, things were a bit quiet here for a while but it looks like interest has been rekindled over the past few days.

I've looked at the code again and I think implementing a simple moving average will be relatively easy. I may be proven wrong once I actually start to code it. Hopefully not.

Anyway, before writing another single line of code I'm going to do a bit of housekeeping, tidy up a few things on github and utilise the power of git a bit more.

I'll first write a better description on the project readme. What is IsisATP, what does it do, how does it do it, how is it configured etc. It might save people having to ask questions here on this forum.

I might also start stamping a version tag on it so it's easier to download specific working versions. I'll start at version 0.1.0 for where we are now. Groups of bug fixes increasing the minor version number. Minor functionality changes or additions increasing the mid version number and major changes in functionality increasing the major version number e.g. implementing a moving average will push the version to 0.2.0

I then might create a few milestones to mark where it is at now and where it is going. To give the project a few goals, goals are good!

So, bear with me, normal service will resume shortly.

[new overcomer]

In other news, I think it might be a good idea to actually set up a forum for serious development talks and discussion instead of using this single thread.  I have the resources to set one up right now.

Would non-isis users be allowed to join such a forum? I'm interested in a working trade algo, too.

Definitely!

For the time being this may be a good way to track suggestions that require code change. After discussion in this thread, a specific targeted task can be put in the issues page of github. It's just a suggestion to do it this way so, yea or nay?
Or a new forum can be created as suggested by new overcomer.

Yeah this is why having a forum to discuss different tasks, changes, issues, etc in separate threads would be helpful to all instead of reading through the already 19 pages of comments etc just to find answers.  Plus it helps to segment dev discussion from questions, etc.

The most fundamental change was to switch from a polling to streaming ticker. The reason I decided to use a streaming ticker was so that it would react faster to the market. Now, when a tick is received it triggers an arbitrage thread and a trend observer thread. Each making a decision on whether to trade or not.

I agree with your logic of this, it makes more sense this way.  Like you figured, there is no better way to be as accurate.  I was mulling around in my head the other day if there was a better way to trigger buy/sell logic than to simply wait for the next incoming transaction.  I'm not sure if there is any sufficient need for a different approach though.

I think maybe we should concentrate on core functionality and algorithms first before getting fancy with GUIs and the like. I'm an old-school command line type person anyway and always favour function over form.

Yeah, same here, much more power in the command line and it sounds like daybyter has a good skill/desire for the GUI areas of development; fortunately they don't rely on each other.  Basically the GUI can just hook into the underlying engine without needing to disturb your stuff once he gets the data feeds.  Actually it's of great benefit to have people working on different areas harnessing their expertise.


So I was just reminded of something regarding the forum.  I would like to set up a TOR forum for anonymity sake but I'm not sure how familiar everyone is with this technology.  There are a lot of benefits going this route, but it can be a tad confusing at first if you're not familiar.  Please share your thoughts!!!

[scrybe]

This is a very small group right now, I suggest staying here and starting tangent threads for larger conversations. This allows a wider audience to see what we are doing, and hopefully get involved.

Tor is a non-starter for me. I'm involved in a few small ventures that could later be cast in a nefarious light if I had a Tor capability in my datacenter. Some involve confidentiality clauses in contracts, etc. If I ever have a legitamate reason to setup Tor I'll have to do so in an isolated network somewhere.

Plus there is the guy who was just busted for kiddy porn in Austria for operating an exit node.

I think Tor is cool for folks who need it, but for me the costs of Tor outweigh the benefits.

[AnubisTheJackle]

Is there a way to get the Mt. Gox historical trading data, broken down by hour?

[Aido]

commit 778c84b6cb785e14aaed5550e4fc3da4bcdb7104

    Made number of ticks used to calculate Short SMA configurable from config file

Now the number of ticks used to calculate the short SMA is configurable. The long SMA is based on maxTickAge minutes in config file.


commit 05024af6dd9652fb88b72459d3ab97c2126f83b4

    Added Simple Moving Average calculation method to TrendTradingAgent

Easy enough to do. Now I need to decide how best to use this data to make a trade decision. And when best to use this algorithm over the existing VWAP based trade algorithm.


commit 969e44f5369951c21f48c12a3e0c815a7acc95a1

    Updated README

Self explanatory.

[ewibit]

Will you be building another jar for those not familar with compiling java?

this comes here with the downloaded bin file:
java -jar isis.jar --debug-live=true
Error: Invalid or corrupt jarfile isis.jar

[Aido]

All self explanatory

commit 503002463cecc0f3f4840c14cf0e17bde4e5a00e

    Added Exponential Moving Average calculation method to TrendObserver

commit 6e36689b780caad56cf6f6a852dda446a1ea2d3c

    Tweaked TrendObserver ticker loop

commit f1331a323b61fb1d910081a2fdf32bf6ec17c607

    Moved SMA calculation method to TrendObserver class

this comes here with the downloaded bin file:
java -jar isis.jar --debug-live=true
Error: Invalid or corrupt jarfile isis.jar

@ewibit: There have been considerable changes made to IsisATP since MaxSan's comment on September 19th. The original IsisATP has been forked and this newer version is available at https://github.com/aido/IsisATP. Reading some of the more recent posts will bring you up to speed.

[ewibit]

@ewibit: There have been considerable changes made to IsisATP since MaxSan's comment on September 19th. The original IsisATP has been forked and this newer version is available at https://github.com/aido/IsisATP. Reading some of the more recent posts will bring you up to speed.

thx, now I have tried the new version, but I get this:

Code:

ERROR: Caught unexpected exception, shutting down now!.Details are listed below.
java.lang.NullPointerException
	at com.xeiam.xchange.mtgox.v1.MtGoxAdapters.adaptWallets(MtGoxAdapters.java:133)
	at com.xeiam.xchange.mtgox.v1.service.account.MtGoxPollingAccountService.getAccountInfo(MtGoxPollingAccountService.java:52)
	at org.open.payment.alliance.isis.atp.AccountManager.<init>(AccountManager.java:60)
	at org.open.payment.alliance.isis.atp.AccountManager.getInstance(AccountManager.java:40)
	at org.open.payment.alliance.isis.atp.Application.start(Application.java:112)
	at org.open.payment.alliance.isis.atp.Application.main(Application.java:61)

(@Ubuntu 12.04)
TIA

[Aido]

It seems to have a problem getting your MtGox account information.

Please check that your MtGox API key and MtGox secret key are correct and your intenet connection is working.

[ewibit]

It seems to have a problem getting your MtGox account information.

Please check that your MtGox API key and MtGox secret key are correct and your intenet connection is working.

thx
now I have tried with a new MtGox API key and secret and get this:

Code:

WARNING: Testing connection to exchange
2012-12-03 20:25:43,960 : Isis ATP has started successfully
2012-12-03 20:25:43,962 : Using trend following to decide some trades.
2012-12-03 20:25:43,963 : Using arbitrage to decide some trades.
Exception in thread "Thread-2" com.xeiam.xchange.HttpException: Problem POSTing (IO)
	at com.xeiam.xchange.utils.HttpTemplate.postForString(HttpTemplate.java:230)
	at com.xeiam.xchange.utils.HttpTemplate.postForJsonObject(HttpTemplate.java:103)
	at com.xeiam.xchange.mtgox.v1.service.account.MtGoxPollingAccountService.getAccountInfo(MtGoxPollingAccountService.java:46)
	at org.open.payment.alliance.isis.atp.AccountManager.refreshAccounts(AccountManager.java:119)
	at org.open.payment.alliance.isis.atp.AccountManager.run(AccountManager.java:99)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: https://mtgox.com/api/1//generic/private/info?raw
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
	at com.xeiam.xchange.utils.HttpTemplate.postForString(HttpTemplate.java:222)
	... 5 more
Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: https://mtgox.com/api/1//generic/private/info?raw
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(Unknown Source)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getHeaderField(Unknown Source)
	at com.xeiam.xchange.utils.HttpTemplate.getResponseEncoding(HttpTemplate.java:295)
	at com.xeiam.xchange.utils.HttpTemplate.postForString(HttpTemplate.java:221)
	... 5 more

internet connection is ok...
is it a problem with yubi key? (I am not logged in now)

[daybyter]

Could be, that the URL is the problem, since gox returns 403?

https://mtgox.com/api/1//generic/private/info?raw

I can't find the '?raw' argument in the docs? And I would avoid the doubled slash, although I don't think it's the actual problem.

[Aido]

Could be, that the URL is the problem, since gox returns 403?

hhmmm; the url is hardcoded i.e. not configurable. And I'm not having this problem, nor anyone else besides ewibit by the look of things.

@ewibit: You mentioned you use a yubikey. How is the yubikey connected to this problem? IsisATP just uses the MtGox API key and secret key. Have you these set up on your MtGox account? Maybe try creating a new key pair in the 'Advanced API Key Creation' section of the MtGox Security page: https://mtgox.com/security

[Aido]

Some fundamental changes in most recent commit, so first please read:

https://github.com/aido/IsisATP#isis-atp

Then please test:

https://github.com/aido/IsisATP/raw/master/bin/aido.jar

There have been several additions to the configuration file so you will need to use java -jar aido.jar --clear-config=true first.

Please report any profit/losses for a given configuration.

[meowmeowbrowncow]

@Aido

I tried that 'interesting bash command.'  It worked once, but then there were problems.  What gives?

[ewibit]

Then please test:

https://github.com/aido/IsisATP/raw/master/bin/aido.jar

Please report any profit/losses for a given configuration.

I have tried it now with the new version AND another account without a yubikey and get this again:

Code:

2012-12-07 12:15:20,735 : WARNING: Testing connection to exchange
2012-12-07 12:15:20,839 : Isis ATP has started successfully
2012-12-07 12:15:20,840 : Using trend following to decide some trades.
2012-12-07 12:15:20,841 : Using arbitrage to decide some trades.
Exception in thread "Thread-2" com.xeiam.xchange.HttpException: Problem POSTing (IO)
	at com.xeiam.xchange.utils.HttpTemplate.postForString(HttpTemplate.java:230)
	at com.xeiam.xchange.utils.HttpTemplate.postForJsonObject(HttpTemplate.java:103)
	at com.xeiam.xchange.mtgox.v1.service.account.MtGoxPollingAccountService.getAccountInfo(MtGoxPollingAccountService.java:46)
	at org.open.payment.alliance.isis.atp.AccountManager.refreshAccounts(AccountManager.java:120)
	at org.open.payment.alliance.isis.atp.AccountManager.run(AccountManager.java:100)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: https://mtgox.com/api/1//generic/private/info?raw
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
	at com.xeiam.xchange.utils.HttpTemplate.postForString(HttpTemplate.java:222)
	... 5 more

TIA

edit:
this:
URL: https://mtgox.com/api/1//generic/private/info?raw
means
{"result":"error","error":"Identification required to access private API","token":"login_required"}

-> have I to be logged in before I start the program?