Hi,
Here is some feedback about the platform after our initial period of beta testing (
started late july).
First about who we are and where we're going:- Our company sells DotNetNuke modules and DotNetNuke projects. The latter made up most of our revenue over the last years, but we hope that some of our most unique modules may find their own market at some point.
- In late 2010, we released a firewall module for DNN, the engine of which we ported into a bot farm mid 2011, as an intermediate step towards a long time project to develop a system of web metering proxy agents monitored by a facade of Restful services. The agents will come as a 3rd flavor of the same engine, and the firewall engine will host the web services
- In order to demonstrate the bot farm, we've included in the default version a series of Bitcoin trading bots connecting to several Echange APIs: Mt Gox, Tradehill, ExchangeBitcoins (Keyur Mithawala from Camp BX also contacted us recently, we'll support them ASAP)
- We also published the dnnbitcoin.aricie.com website to provide a hosted version of the DNN module, and to demonstrate hosting user bots with dedicated user parameters. The packaged DNN module is also available for download on the same web site.
Unfortunately, I had to postpone any updates til now, so we dealt all along with that early version, which currently has several problems to be fixed.
Anyway, it has provided experiences of many kinds, which I'm happy to share here.
Here are issues we faced since late july (that was with btc price falling until now):
On a technical side:- We quickly had a couple of wrong passwords on Mt Gox, which got our IP banned on a regular basis, with a tedious procedure to identify the faulty accounts and get the system back on track. Thankfully this hasn't happened recently.
- We had major failures with our hosting server. We have started installing a new web farm since then. Hopefully, we can get most websites on the new platform by the end of this month
- After each run, each user bot gets encrypted with parameters and data and updated in DB: that resulted in extensive db usage, massive DB logs, crashes and other performance issues. Moving to simple recovery mode seemed to do the trick though.
- Web access to the Exchanges would be quite unreliable, especially Mt Gox during rallies, which is fine to some extent, but I left a bug where exceptions from timeouts sometimes leave a synchronization monitor in the wild. More crashes
- Mt Gox would initially consolidate open orders of the same price, so I didn't take care of the consolidation myself. When they stopped and for other exchanges, that resulted in numerous small orders being issued, making it difficult to monitor the corresponding wallet manually and to perform the appropriate cancellation if needed.
Basically the site would not work reliably at all during the whole time, which of course didn't help with getting the bots doing the right job timely.
Now the financial side taught lessons too:The pricer works in a trading band, it plots orders linearly in value between the given extreme orders at each side of the band, and the central orders around the last price, accounting for the transaction fees and valued as a percentage of the extreme orders.
Now the distribution of orders also is an important parameter, and I chose to halve the distance from the closest order to a limit defined by the ticker and the fees as a mean to define new orders.
Overall it results in a non predictive market-making kind of strategy, buying increasingly on down trends, selling increasingly on up trends, and slowly leveling btcs/usds levels on stable periods.
I think the general idea was OK, but it has some obvious problems too:
- The major show stopper was the trading band itself. As the price was quite stable around 15$/btc with very low activity when the platform was started, the bots defaults to a narrow trading band, the lower bound of which was hit several times since then with the price constantly falling. It would result on the bots slowly adjusting with a new lower bond by selling more than it would have otherwise, the bound being adjusted manually, or worse; if the order cancelling mechanism stalled on the way down, the bot would find itself completely stuck, with open asks order placed too high, and no bid orders at all. I've seen such unattended bots stopped with a hand all btcs, that kept losing while the price continued falling
- Then there was the pricing itself. You can adjust the aggressiveness of the bot with parameters that account for the value and density of orders around the ticker, then the orders at the extreme of the band would define the price increase. For 2 given extreme orders, at any market price, there are an infinity of functions of the kind "amount = a + b /(price - c )" passing through the extreme and central points, and considering the exponential distribution, the ones defined by the linear valuation don't seem optimal, especially when the ticker gets closer to the trading band's lowest bound. I'd hope someone with a good math background can give a hand there.
- Defining the exponential distribution by halving the distance to the target limit wasn't a smart move either. It would require to define a very low limit value to get significantly close, with the then low volatility. I later replaced the default halving expression to measure the distance to the ticker instead (switched the strategy expression from (LowestAsk.price + LowestAskLimitPrice) / 2 to (LowestAsk.price + Market.Ticker.last) / 2), which seemed a better tradeoff for low/high volatilities.
- The cancellation mechanism was not completely reliable. It works by modulating updates with a logarithmic plot of their density to their distance from the ticker, and then cancels the resulting holes in the price distribution. With the wrong parameters, the bot could run out of resources to perform the appropriate updates, failing to leave holes, that would in turn fail to return the given resources for new orders and updates. That seemed to happen on several unattended bots. Adjusting the band manually was key to avoiding those situations.
Finally, here is an Excel extract (with French numbering) from an Mt Gox account which was attended from end August to account for the trading band issue described above.
Date FixedPrice Price btcs btcs value usds Total TotalFixedPrice Fixed btc Fixed $ Total btc
Fri 08 Jul 2011 09:21:25 PM GMT $5,63597 $14,26474 9,63918 $137,50 $4 292,36 $4 429,86 $4 346,68 $4 429,20 $4 429,86 310,54605
Sat 09 Jul 2011 04:41:51 AM GMT $5,63597 $14,23338 7,96158 $113,32 $4 316,28 $4 429,60 $4 361,15 $4 419,46 $4 429,86 311,21182
Sat 09 Jul 2011 07:31:55 AM GMT $5,63597 $14,26037 6,88541 $98,19 $4 331,65 $4 429,83 $4 370,45 $4 427,84 $4 429,86 310,63949
Sat 09 Jul 2011 09:04:42 AM GMT $5,63597 $14,18257 8,29591 $117,66 $4 311,60 $4 429,26 $4 358,36 $4 403,69 $4 429,86 312,30318
Sat 09 Jul 2011 07:52:01 PM GMT $5,63597 $14,42982 4,97490 $71,79 $4 359,08 $4 430,86 $4 387,11 $4 480,46 $4 429,86 307,06294
Sun 10 Jul 2011 07:15:53 PM GMT $5,63597 $15,04696 5,44503 $81,93 $4 350,90 $4 432,83 $4 381,58 $4 672,08 $4 429,86 294,59953
Mon 11 Jul 2011 08:51:54 AM GMT $5,63597 $14,96470 6,65819 $99,64 $4 332,92 $4 432,55 $4 370,44 $4 646,54 $4 429,86 296,20067
Mon 11 Jul 2011 12:28:29 PM GMT $5,63597 $14,91241 5,23260 $78,03 $4 354,35 $4 432,38 $4 383,84 $4 630,30 $4 429,86 297,22768
Mon 11 Jul 2011 02:26:50 PM GMT $5,63597 $13,97904 9,21048 $128,75 $4 296,29 $4 425,04 $4 348,20 $4 340,49 $4 429,86 316,54821
Mon 11 Jul 2011 03:24:40 PM GMT $5,63597 $14,27320 8,36217 $119,35 $4 308,33 $4 427,69 $4 355,46 $4 431,83 $4 429,86 310,20988
Mon 11 Jul 2011 04:34:24 PM GMT $5,63597 $14,21820 7,90234 $112,36 $4 314,71 $4 427,07 $4 359,25 $4 414,75 $4 429,86 311,36633
Tue 12 Jul 2011 02:54:59 PM GMT $5,63597 $13,95124 8,26546 $115,31 $4 309,65 $4 424,97 $4 356,24 $4 331,86 $4 429,86 317,17361
Wed 13 Jul 2011 10:25:12 AM GMT $5,63597 $14,06000 18,23546 $256,39 $4 169,06 $4 425,45 $4 271,84 $4 365,63 $4 429,86 314,75497
Thu 14 Jul 2011 12:09:06 AM GMT $5,63597 $13,98009 68,48748 $957,46 $3 461,61 $4 419,07 $3 847,60 $4 340,82 $4 429,86 316,09734
Thu 14 Jul 2011 06:30:49 PM GMT $5,63597 $13,87766 84,49957 $1 172,66 $3 239,30 $4 411,95 $3 715,53 $4 309,01 $4 429,86 317,91768
Fri 15 Jul 2011 11:08:50 AM GMT $5,63597 $14,00692 93,92795 $1 315,64 $3 107,71 $4 423,35 $3 637,08 $4 349,15 $4 429,86 315,79724
Fri 15 Jul 2011 09:53:40 PM GMT $5,63597 $13,79261 130,17504 $1 795,45 $2 606,24 $4 401,69 $3 339,90 $4 282,61 $4 429,86 319,13429
Sat 16 Jul 2011 11:08:17 AM GMT $5,63597 $13,78508 142,56421 $1 965,26 $2 437,01 $4 402,27 $3 240,50 $4 280,27 $4 429,86 319,35023
Sun 17 Jul 2011 07:42:33 AM GMT $5,63597 $13,40066 181,58074 $2 433,30 $1 907,66 $4 340,96 $2 931,05 $4 160,90 $4 429,86 323,93659
Sun 17 Jul 2011 01:50:35 PM GMT $5,63597 $13,27595 183,39558 $2 434,75 $1 883,38 $4 318,13 $2 917,00 $4 122,18 $4 429,86 325,25994
Sun 17 Jul 2011 06:51:51 PM GMT $5,63597 $13,29426 170,33082 $2 264,42 $2 056,67 $4 321,09 $3 016,65 $4 127,87 $4 429,86 325,03450
Sun 17 Jul 2011 09:45:53 PM GMT $5,63597 $13,34937 164,20130 $2 191,98 $2 138,11 $4 330,10 $3 063,55 $4 144,98 $4 429,86 324,36704
Mon 18 Jul 2011 06:55:06 AM GMT $5,63597 $13,10708 160,67890 $2 106,03 $2 184,89 $4 290,92 $3 090,47 $4 069,75 $4 429,86 327,37438
Mon 18 Jul 2011 01:28:21 PM GMT $5,63597 $13,03686 162,13379 $2 113,72 $2 165,87 $4 279,59 $3 079,66 $4 047,95 $4 429,86 328,26847
Mon 18 Jul 2011 10:51:59 PM GMT $5,63597 $13,47424 126,49718 $1 704,45 $2 645,42 $4 349,87 $3 358,35 $4 183,75 $4 429,86 322,82871
Tue 19 Jul 2011 01:59:52 AM GMT $5,63597 $13,84900 95,33973 $1 320,36 $3 081,97 $4 402,33 $3 619,30 $4 300,11 $4 429,86 317,88068
Wed 20 Jul 2011 01:43:40 AM GMT $5,63597 $13,77000 99,61441 $1 371,69 $3 023,33 $4 395,02 $3 584,75 $4 275,59 $4 429,86 319,17355
Thu 21 Jul 2011 03:44:14 PM GMT $5,63597 $13,60848 142,35682 $1 937,26 $2 444,04 $4 381,30 $3 246,36 $4 225,43 $4 429,86 321,95375
Thu 21 Jul 2011 07:52:54 PM GMT $5,63597 $13,50986 146,62829 $1 980,93 $2 386,26 $4 367,18 $3 212,65 $4 194,81 $4 429,86 323,25902
Fri 22 Jul 2011 07:07:03 AM GMT $5,63597 $13,67958 135,35421 $1 851,59 $2 540,37 $4 391,96 $3 303,23 $4 247,51 $4 429,86 321,05976
Sat 23 Jul 2011 11:27:37 PM GMT $5,63597 $13,65410 139,38223 $1 903,14 $2 485,98 $4 389,12 $3 271,54 $4 239,60 $4 429,86 321,45076
Mon 25 Jul 2011 08:25:46 AM GMT $5,63597 $14,28070 107,74583 $1 538,69 $2 927,88 $4 466,56 $3 535,13 $4 434,16 $4 429,86 312,76909
Mon 25 Jul 2011 10:46:00 PM GMT $5,63597 $14,00195 126,43315 $1 770,31 $2 664,10 $4 434,41 $3 376,67 $4 347,61 $4 429,86 316,69952
Tue 26 Jul 2011 10:41:50 AM GMT $5,63597 $14,01760 138,97174 $1 948,05 $2 489,33 $4 437,38 $3 272,57 $4 352,46 $4 429,86 316,55746
Thu 28 Jul 2011 11:14:36 AM GMT $5,63597 $13,61043 154,58973 $2 104,03 $2 275,27 $4 379,31 $3 146,54 $4 226,04 $4 429,86 321,76110
Thu 28 Jul 2011 09:50:05 PM GMT $5,63597 $13,40005 169,63393 $2 273,10 $2 070,80 $4 343,90 $3 026,85 $4 160,72 $4 429,86 324,17036
Tue 09 Aug 2011 12:29:24 PM GMT $5,63597 $11,89202 296,74755 $3 528,93 $847,91 $4 376,83 $2 520,37 $3 692,47 $4 429,86 368,04801
Mon 22 Aug 2011 06:06:26 AM GMT $5,63597 $11,19690 218,41435 $2 445,56 $1 726,71 $4 172,27 $2 957,69 $3 476,64 $4 429,86 372,62748
Tue 23 Aug 2011 06:42:55 PM GMT $5,63597 $10,93342 214,45385 $2 344,71 $1 771,30 $4 116,02 $2 979,96 $3 394,83 $4 429,86 376,46189
Thu 25 Aug 2011 07:42:02 AM GMT $5,63597 $10,59635 218,31705 $2 313,36 $1 729,45 $4 042,81 $2 959,88 $3 290,17 $4 429,86 381,52881
Thu 25 Aug 2011 02:21:40 PM GMT $5,63597 $10,05150 235,35915 $2 365,71 $1 555,20 $3 920,91 $2 881,68 $3 120,99 $4 429,86 390,08253
Thu 25 Aug 2011 08:33:19 PM GMT $5,63597 $9,80000 226,95467 $2 224,16 $1 634,56 $3 858,71 $2 913,67 $3 042,90 $4 429,86 393,74630
Sun 28 Aug 2011 04:30:10 PM GMT $5,63597 $9,13762 235,23474 $2 149,49 $1 506,51 $3 655,99 $2 832,28 $2 837,23 $4 429,86 400,10330
Tue 30 Aug 2011 10:17:17 PM GMT $5,63597 $8,77355 241,11609 $2 115,44 $1 460,01 $3 575,45 $2 818,93 $2 724,19 $4 429,86 407,52631
Mon 05 Sep 2011 09:19:15 AM GMT $5,63597 $7,91654 294,50808 $2 331,49 $1 021,94 $3 353,43 $2 681,78 $2 458,09 $4 429,86 423,59790
Mon 05 Sep 2011 11:24:40 PM GMT $5,63597 $7,57906 284,98577 $2 159,92 $1 092,76 $3 252,68 $2 698,93 $2 353,30 $4 429,86 429,16687
Tue 06 Sep 2011 10:05:44 AM GMT $5,63597 $6,69282 328,51593 $2 198,70 $784,71 $2 983,41 $2 636,22 $2 078,12 $4 429,86 445,76308
Tue 06 Sep 2011 01:57:18 PM GMT $5,63597 $6,79929 246,26328 $1 674,42 $1 332,65 $3 007,06 $2 720,58 $2 111,18 $4 429,86 442,26138
Tue 06 Sep 2011 03:56:51 PM GMT $5,63597 $6,59802 240,52972 $1 587,02 $1 372,83 $2 959,85 $2 728,44 $2 048,69 $4 429,86 448,59614
Tue 06 Sep 2011 07:56:14 PM GMT $5,63597 $6,54316 218,58852 $1 430,26 $1 516,71 $2 946,97 $2 748,67 $2 031,65 $4 429,86 450,38965
Tue 06 Sep 2011 11:00:16 PM GMT $5,63597 $6,76624 210,38674 $1 423,53 $1 573,33 $2 996,85 $2 759,06 $2 100,92 $4 429,86 442,91267
Wed 07 Sep 2011 01:45:28 AM GMT $5,63597 $6,63101 196,11771 $1 300,46 $1 673,01 $2 973,47 $2 778,32 $2 058,93 $4 429,86 448,41864
Wed 07 Sep 2011 05:32:05 AM GMT $5,63597 $6,99114 186,51185 $1 303,93 $1 741,24 $3 045,17 $2 792,41 $2 170,75 $4 429,86 435,57537
Wed 07 Sep 2011 06:53:12 AM GMT $5,63597 $6,93537 171,84789 $1 191,83 $1 848,87 $3 040,70 $2 817,40 $2 153,43 $4 429,86 438,43410
Wed 07 Sep 2011 09:27:33 AM GMT $5,63597 $7,14080 173,23303 $1 237,02 $1 839,75 $3 076,77 $2 816,08 $2 217,22 $4 429,86 430,87174
Wed 07 Sep 2011 02:35:18 PM GMT $5,63597 $6,85824 174,31176 $1 195,47 $1 832,37 $3 027,84 $2 814,79 $2 129,48 $4 429,86 441,48972
Thu 08 Sep 2011 07:13:11 PM GMT $5,63597 $6,67655 172,93449 $1 154,61 $1 845,85 $3 000,46 $2 820,51 $2 073,07 $4 429,86 449,40268
Fri 09 Sep 2011 01:11:24 AM GMT $5,63597 $6,00000 334,11319 $2 004,68 $841,45 $2 846,13 $2 724,50 $1 863,00 $4 429,86 474,35514
Fri 09 Sep 2011 03:14:05 AM GMT $5,63597 $6,44165 310,69363 $2 001,38 $989,93 $2 991,31 $2 740,99 $2 000,13 $4 429,86 464,37056
Fri 09 Sep 2011 06:25:03 AM GMT $5,63597 $6,14322 311,83903 $1 915,70 $983,62 $2 899,32 $2 741,14 $1 907,47 $4 429,86 471,95444
Fri 09 Sep 2011 09:29:17 AM GMT $5,63597 $5,98244 312,82058 $1 871,43 $978,22 $2 849,65 $2 741,26 $1 857,55 $4 429,86 476,33503
Fri 09 Sep 2011 09:39:49 PM GMT $5,63597 $4,89782 306,70844 $1 502,20 $976,77 $2 478,98 $2 705,37 $1 520,77 $4 429,86 506,13897
Sat 10 Sep 2011 11:54:56 AM GMT $5,63597 $5,19762 241,47630 $1 255,10 $1 314,86 $2 569,96 $2 675,81 $1 613,86 $4 429,86 494,44952
Sat 10 Sep 2011 08:25:49 PM GMT $5,63597 $4,70814 294,25141 $1 385,38 $1 056,96 $2 442,33 $2 715,35 $1 461,88 $4 429,86 518,74697
Sun 11 Sep 2011 11:21:46 AM GMT $5,63597 $4,90090 296,49635 $1 453,10 $1 048,95 $2 502,05 $2 720,00 $1 521,73 $4 429,86 510,52902
Sun 11 Sep 2011 07:02:01 PM GMT $5,63597 $6,04200 239,87118 $1 449,30 $1 464,76 $2 914,06 $2 816,67 $1 876,04 $4 429,86 482,30110
Sun 11 Sep 2011 11:28:52 PM GMT $5,63597 $5,81074 239,90927 $1 394,05 $1 481,06 $2 875,11 $2 833,18 $1 804,23 $4 429,86 494,79270
Mon 12 Sep 2011 08:17:54 PM GMT $5,63597 $5,63597 264,45589 $1 490,47 $1 342,16 $2 832,62 $2 832,62 $1 749,97 $4 429,86 502,59707
I guess there's plenty room for improvement but the recent results are encouraging.
Now about what I will be doing from now:- I'm aiming at releasing a new Beta / RC of the platform, with new bot features (to integrate proxies for instance, or to provide with a command to clear orders). Right now, DNN 6 support is almost complete.
- I'm also willing to update the pricing algo a the same time. And I'm calling for help here. Who would team up on a Codeplex project to maintain the lib? Or simply give advice concerning the pricing expressions? The original proj is VB.Net but I'm happy to switch to C# if that makes a difference.
Thanks for reading that far,
Jesse
CTO Aricie
PS:
A few personal speculations about the recent trends and bots:The market is more or less balancing miners / merchants / early birds asks + exchange fees with customers / new investors bids, in the process of which traders take a profit.
Some predictive trading strategies anticipate and accentuate market trends, making the best profit and contributing extra volatility. This is compensated by natural agents, non predictive strategies and some other predictive strategies, which will bring liquidity to the process. Our bots fall in that last category, making a profit only by working against volatility.
A notable distinction is also that day traders will sell out after each session, while other strategies will hold balanced hands of btcs and $. Only the former would have secured a profit over the last period and I reckon it was actually prejudicial for most, considering btc traders are usually also btc holders.
During the initial rally everyone could make a profit with the miners' happy asking low and the investors bidding high. Now the media coverage did not exactly match the recent difficulty, and while the unbalance started a down trend, I reckon a lot was burnt during the fall with predictive bots + panic selling, on different scales as suggested by those many floors in the history charts, and I suppose a fraction of what was lost could have hold the miners inflation at a reasonably higher price for quite some time. Many bid walls proved fake, and with sellers chasing them, you did not need much extra drama to feed the trend.
I don't have anything against predictive strategies, I'll be happy to account for market depth walls and last trades ASAP, but I suppose some of the bots that generated profits playing the down trend didn't account for the loss in their owners' backed up wallets, in other words we should try to make sure our strategies don't dry the market for the loss of everyone.
This is to say it is not only about the growing real economy, adjusting difficulty or PR now, it has also a lot to do about how we handle the trading collectively, to keep the price at its max that the market is really willing to cover. There's a prisoners dilemma that blind competing bots wouldn't solve by themselves.
I think one way to balance that now is by having more market-makers building up real walls, and I suppose some current holders could take a share of the corresponding profit, or mitigate their losses, while providing coverage against volatility.
TLDR; Some feedback about our trading bots platform, pitfalls and perspectives, calling for contributors to the open-source pricer, reflecting on the market.
