Bitcoin Forum
December 03, 2016, 01:57:15 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 ... 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 [54] 55 56 57 58 59 »
  Print  
Author Topic: Goomboo's Journal  (Read 250664 times)
Queeq
Sr. Member
****
Offline Offline

Activity: 426



View Profile
December 13, 2013, 09:16:27 AM
 #1061

WOW that great, thanks you a lot!!

this program can run @ WIN or just @linux ?

& can i humbly ask you draw 5 MIN?

thanks again!

5 min tick for dec.05 - dec.12:


Welcome.

The script can be run on both Win and Linux (or any other OS which has ability to run Python scripts). However it needs some adaptation for that, but I moved my focus to slightly different area. Though anyone can do that as it's open-source and available on Github.

Python backtesting and bot script at Github
Development halted due to other priorities.
1480773435
Hero Member
*
Offline Offline

Posts: 1480773435

View Profile Personal Message (Offline)

Ignore
1480773435
Reply with quote  #2

1480773435
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480773435
Hero Member
*
Offline Offline

Posts: 1480773435

View Profile Personal Message (Offline)

Ignore
1480773435
Reply with quote  #2

1480773435
Report to moderator
1480773435
Hero Member
*
Offline Offline

Posts: 1480773435

View Profile Personal Message (Offline)

Ignore
1480773435
Reply with quote  #2

1480773435
Report to moderator
mmccll
Newbie
*
Offline Offline

Activity: 7


View Profile
December 14, 2013, 10:23:19 AM
 #1062

Very interesting stuff Goomboo. Respect.
I've been running soms backtests using Gekko from Whydifficult (again many respect).

What I've done is creating a script which alters the ema pair in the config file and loops Gekko over this. I'm on my phone now so can't upload the results. Quite similar to queeq, but haven't read his post while making the script...

A very interesting outcome is that the ema pairs which have a short period bigger than the long gives very consistent high profit percentages. Way higher than the other way around. Also looking at the buy sell moments it tends to buy low and sell high pretty good.

data period is last 1.5 weeks and interval is 1h.

Any insights on this?

I will upload files when on laptop.
mmccll
Newbie
*
Offline Offline

Activity: 7


View Profile
December 14, 2013, 11:10:50 AM
 #1063

See below. Notice especially the area with the short between 41-43 and the long around 25.

http://www.canalphotos.nl/Afbeelding1.jpg

And a sample run of Gekko on the EMA pair short 42, long 25:

Code:
2013-12-14 12:09:10 (INFO):     ADVICE is to BUY @ 728.000 (7.710)      at      2013-12-07 12:59:57
2013-12-14 12:09:10 (INFO):     ADVICE is to SELL @ 962.900 (-0.511)    at      2013-12-09 09:59:59
2013-12-14 12:09:10 (INFO):     ADVICE is to BUY @ 908.610 (0.330)      at      2013-12-12 04:59:59
2013-12-14 12:09:10 (INFO):     ADVICE is to SELL @ 972.010 (-0.255)    at      2013-12-13 11:59:54


2013-12-14 12:09:10 (INFO):             WARNING: BACKTESTING FEATURE NEEDS PROPER TESTING
2013-12-14 12:09:10 (INFO):             WARNING: ACT ON THESE NUMBERS AT YOUR OWN RISK!


2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) start time:                      2013-12-07 11:59:59
2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) end time:                        2013-12-13 22:55:35
2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) timespan:                        6 days

2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) start price:                     1065.1
2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) end price:                       891
2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) Buy and Hold profit:             -16.345883%

2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) amount of trades:                4
2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) original simulated balance:      1000.000000 USD
2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) current simulated balance:       1403.670563 USD
2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) simulated profit:                403.670563 USD (40.367056%)
2013-12-14 12:09:10 (INFO):     (PROFIT REPORT) simulated yearly profit:         23951.120080 USD (2395.112008%)
Queeq
Sr. Member
****
Offline Offline

Activity: 426



View Profile
December 14, 2013, 12:40:17 PM
 #1064

Here's my result for almost the same period.



I believe we have different algorithms of backtesting.

In short, mine is the following:
  Build an array of closing prices for, say, 1 hour resolution (configurable).
  Build a number of simple and exponential moving averages from 1 to 50 (configurable) for these closing prices.
  Find all intersections of slow and fast MAs for each unique pair.
  Simulate sell or buy at that points (use closing price) depending on which MA goes below which: if fast goes below slow - sell, and vice versa. Substract a fee from each operation. 0.2% by default (configurable).
  See what amount of initial sum (in percent) is left.
  Show the result for each pair on heatmap.

Python backtesting and bot script at Github
Development halted due to other priorities.
mmccll
Newbie
*
Offline Offline

Activity: 7


View Profile
December 14, 2013, 12:56:42 PM
 #1065

That's interesting. I'm using Whydifficults Gekko trading bot which can be found at:

https://bitcointalk.org/index.php?topic=209149.0

(which is by the way pretty amazing stuff!)

Basically it lets you set the EMA pair and some thresholds. Input is a csv file with timestamp, open, high, low and close values. But i assume it does roughly the same as your exponential algorithm.

While making a php script to loop over various settings of Gekko I noticed that there are some very profitable results when you take the short average with a higher setting compared to the long average. That would be in your graphs any values below the diagonal line (which is in your graph blanked out).
Might be interesting to modify your script to see what is shown in this now white area.

Below is the candle-file which I used to make this graph:

http://www.canalphotos.nl/gekko/upload/candles.csv
Queeq
Sr. Member
****
Offline Offline

Activity: 426



View Profile
December 14, 2013, 02:08:59 PM
 #1066

That would be in your graphs any values below the diagonal line (which is in your graph blanked out).
Might be interesting to modify your script to see what is shown in this now white area.

That would mean to make my script act in another way round: buy when fast goes below slow and sell when it goes above. That is completely opposite of what Goomboo taught us Smiley But for the sake of experiment, here's the result Smiley


Python backtesting and bot script at Github
Development halted due to other priorities.
Bulbasaur
Newbie
*
Offline Offline

Activity: 13


View Profile
December 14, 2013, 03:59:57 PM
 #1067

Queeq is right, whether an average is fast or slow depends entirely on the number of periods, so to say that the fast one has a higher number than the slow means that it's no longer really the fast one in practice.

I would be hesitant to trade on any such strategy. You would expect it would probably just do as Queeq suggests, buying and selling on the 'wrong' crossovers, but it might also do entirely unexpected things since there's some chance that the code expects the 'fast' input to always actually be faster than the 'slow' one and hasn't been debugged for the reversed case.

On the other hand, your plot does look basically like a mirror image, so where the normal crossover strategy is unprofitable, the corresponding reversed crossover shows a fairly proportional profit. This does seem to make sense, if we assume the losses are due mainly to 'wrong' guesses rather than commission costs. Switching all the trades would transform all the 'wrong' guesses to 'right' guesses.

I would guess that what's happening is, the indicators are lagging enough that the crossover doesn't hit until the trend is over, at which point you would be taking a position just when the trend is reversing. So if you trade against the crossover, you (maybe) catch the new trend rather than the old one. But now you're getting away from the original idea of using the averages as a signal of recent momentum/current trend and relying more on hoping that the trend reversals will happen to align with your choice of lag time.
xybersurfer
Member
**
Offline Offline

Activity: 61


View Profile
December 14, 2013, 08:16:22 PM
 #1068

mmccll, i think you need to check to make sure you didn't accidentally reverse your short and long axii because they seem to be the complete opposite of what one would expect (not just in the normally missing area). mmccll i do enjoy seeing what the heatmap looks like on your more normal axii.

also the csv file from mmccll goes from:
2013-12-06 12:59:28 to 2013-12-13 21:55:35

while the data from Queeq goes from:
2013-12-05 00:00:00 to 2013-12-12 00:00:00

but i doubt this makes that much difference
mmccll
Newbie
*
Offline Offline

Activity: 7


View Profile
December 15, 2013, 12:41:25 PM
 #1069

I'm checking my scripts very carefully now, just to make sure that there are no mistakes in it. But basically what it does is looping over the gekko config file altering those parameters:
Code:
  // EMA weight (α)
  // the higher the weight, the more smooth (and delayed) the line
  short: 10,
  long: 21,
  // amount of candles to remember and base initial EMAs on

mmccll
Newbie
*
Offline Offline

Activity: 7


View Profile
December 15, 2013, 01:09:39 PM
 #1070

Checked for 10/21 and 21/10.
Heatmap. Notice the cells with bold lines around them. Showing calculated profit of 11.2% and 8.1%.
http://canalphotos.nl/gekko/upload/Afbeelding3.jpg

And the Gekko output to verify:

Code:
##### SHORT: 10   LONG: 21 #####
2013-12-15 13:52:18 (INFO):     ADVICE is to SELL @ 728.000 (-10.036)   at      2013-12-07 12:59:57
2013-12-15 13:52:18 (INFO):     ADVICE is to BUY @ 820.000 (0.767)      at      2013-12-09 00:59:58
2013-12-15 13:52:18 (INFO):     ADVICE is to SELL @ 938.000 (-0.661)    at      2013-12-11 14:59:59
2013-12-15 13:52:18 (INFO):     ADVICE is to BUY @ 950.000 (0.591)      at      2013-12-13 05:59:46
2013-12-15 13:52:18 (INFO):     ADVICE is to SELL @ 905.000 (-0.287)    at      2013-12-13 20:59:54

2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) start time:                      2013-12-07 11:59:59
2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) end time:                        2013-12-13 22:55:35
2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) timespan:                        6 days

2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) start price:                     1065.1
2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) end price:                       891
2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) Buy and Hold profit:             -16.345883%

2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) amount of trades:                5
2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) original simulated balance:      1000.000000 USD
2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) current simulated balance:       1081.025961 USD
2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) simulated profit:                81.025961 USD (8.102596%)
2013-12-15 13:52:18 (INFO):     (PROFIT REPORT) simulated yearly profit:         4807.540376 USD (480.754038%)

##### SHORT: 21   LONG: 10 #####
2013-12-15 13:54:15 (INFO):     ADVICE is to BUY @ 728.000 (10.036)     at      2013-12-07 12:59:57
2013-12-15 13:54:15 (INFO):     ADVICE is to SELL @ 820.000 (-0.767)    at      2013-12-09 00:59:58
2013-12-15 13:54:15 (INFO):     ADVICE is to BUY @ 938.000 (0.661)      at      2013-12-11 14:59:59
2013-12-15 13:54:15 (INFO):     ADVICE is to SELL @ 950.000 (-0.591)    at      2013-12-13 05:59:46
2013-12-15 13:54:15 (INFO):     ADVICE is to BUY @ 905.000 (0.287)      at      2013-12-13 20:59:54

2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) start time:                      2013-12-07 11:59:59
2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) end time:                        2013-12-13 22:55:35
2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) timespan:                        6 days

2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) start price:                     1065.1
2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) end price:                       891
2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) Buy and Hold profit:             -16.345883%

2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) amount of trades:                5
2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) original simulated balance:      1000.000000 USD
2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) current simulated balance:       1111.949512 USD
2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) simulated profit:                111.949512 USD (11.194951%)
2013-12-15 13:54:15 (INFO):     (PROFIT REPORT) simulated yearly profit:         6642.337697 USD (664.233770%)

And Queeg/Bulbasur you're right! It swaps the BUY/SELL moments as you can see above, effectively trading against the crossovers. Resulting in higher profits in the specific market dynamics of the specified time period. I'll be running some tests on other data sets (varying from/to and interval) to see how this holds against other periods. Might be interesting to see the results...
lx
Newbie
*
Offline Offline

Activity: 7


View Profile
December 15, 2013, 04:29:15 PM
 #1071

Hi everyone and thanks for this amazing thread.
I just wanted your opinion on the fact that the data we use for backtesting is 'polluted' by events that will most likely not happen again (bitcoin skyrocketing to > $1000 etc).
In the end backtesting is about the relevancy of the data you use to test your strategy, and I just feel like the data is not relevant to determine what's coming.

I've been playing around with Gekko and it seems that EMAs settings have to be adjusted (inverted ?) at some point ; which is not very natural (be gentle, I'm just a geek playing wall street)
Queeq
Sr. Member
****
Offline Offline

Activity: 426



View Profile
December 15, 2013, 05:05:48 PM
 #1072

I just wanted your opinion on the fact that the data we use for backtesting is 'polluted' by events that will most likely not happen again (bitcoin skyrocketing to > $1000 etc).
In the end backtesting is about the relevancy of the data you use to test your strategy, and I just feel like the data is not relevant to determine what's coming.

You're free to backtest on any time periods you want. You may exclude periods of fast growth to have a picture of what will be going on when market is steady. However you may potentially lose profits if price goes up/down again rapidly. So I do not only test on the large periods, but also on smaller ones, especially when price goes down to see how much would I lose in such market conditions and to be prepared for that.

Python backtesting and bot script at Github
Development halted due to other priorities.
notme
Legendary
*
Offline Offline

Activity: 1526


View Profile
December 18, 2013, 06:31:29 AM
 #1073

Daily Goomboos should have sold around $700.  Not bad for a few hours, not that Goomboos care for quick profits.

https://www.bitcoin.org/bitcoin.pdf
While no idea is perfect, some ideas are useful.
12jh3odyAAaR2XedPKZNCR4X4sebuotQzN
solidshotnosh
Sr. Member
****
Offline Offline

Activity: 309



View Profile
December 18, 2013, 06:51:01 AM
 #1074

Daily Goomboos should have sold around $700.  Not bad for a few hours, not that Goomboos care for quick profits.

The last sell signal I had was @ 857, what interval are you using?
Queeq
Sr. Member
****
Offline Offline

Activity: 426



View Profile
December 18, 2013, 08:18:42 AM
 #1075

According to my calculations, Goomboo's strategy is fairly unprofitable in downtrend. That's not surprising at all as signals are generated with lag, thus buying signals are generated with price higher than selling signals. I wonder of how could we minimize lost in such situations. There may be some additional indicators. Could someone point them out so I could add them to backtesting model?

Here are the results from 11th of December till now.

5 min


15 min


30 min


1 hour


2 hours

Python backtesting and bot script at Github
Development halted due to other priorities.
helluvaname
Newbie
*
Offline Offline

Activity: 28


View Profile
December 18, 2013, 09:08:03 PM
 #1076


Here are the results from 11th of December till now.


Hi Queeq!
Could you please add some axis on your graphs? It is very hard to understand what MA numbers are on your pictures.

By the way, I have done a little research of my own, it is EMA on 1 hour scale from 08.09.2011 till 08.12.13, btc-e data. I wonder, Queeq, if you could run a similar conditions test so we could compare the results?
The numbers are total balance at the final date, starting with 100$

the picture (I have some "disabled for security errors" so cant post a thumbnail):

http://postimg.org/image/5m32aninp/

And, while getting btc-e data from bitcoincharts I have found several "infinite" prices, which vere replaced vith valid prices taken from previous candles.
Queeq
Sr. Member
****
Offline Offline

Activity: 426



View Profile
December 19, 2013, 09:46:29 AM
 #1077

Hi Queeq!
Could you please add some axis on your graphs? It is very hard to understand what MA numbers are on your pictures.

By the way, I have done a little research of my own, it is EMA on 1 hour scale from 08.09.2011 till 08.12.13, btc-e data. I wonder, Queeq, if you could run a similar conditions test so we could compare the results?
The numbers are total balance at the final date, starting with 100$

the picture (I have some "disabled for security errors" so cant post a thumbnail):

http://postimg.org/image/5m32aninp/

And, while getting btc-e data from bitcoincharts I have found several "infinite" prices, which vere replaced vith valid prices taken from previous candles.

If you zoom in you can see MA lengths in braces. I purposely turned off the axis as they don't help much when you look somewhere in the middle of a map.

My numbers are relative percent of the starting sum, so they would be pretty much similar, except that if sum goes under 100 they would be negative.

Bitcoincharts have some gaps in their data, especially in the beginning. They can be easily seen on their graphs. My algorithm just fills these gaps with last known closing price.

Here's the result of my backtesting on the given interval (08.09.2011 - 08.12.2013) with BTC-e data as well. I haven't included longer MAs as they're all red and image becomes insanely big.



So the results are mostly different. What algorithm have you used?

Python backtesting and bot script at Github
Development halted due to other priorities.
helluvaname
Newbie
*
Offline Offline

Activity: 28


View Profile
December 20, 2013, 12:10:40 PM
 #1078

If you zoom in you can see MA lengths in braces. I purposely turned off the axis as they don't help much when you look somewhere in the middle of a map.

Oh now I see.

Bitcoincharts have some gaps in their data, especially in the beginning. They can be easily seen on their graphs. My algorithm just fills these gaps with last known closing price.

So do mine.

So the results are mostly different. What algorithm have you used?

Well, I made it the hard way Grin I have loaded all hourly "raw data" from bitcoincharts (there is a button under a candle graph) to excel, then wrote a VBA script imitating buy and sell.
I forgot to tell that I have included a 0,25% fee for each trade.
Now I am moving from excel to MT4 available at btc-e, but it doesn't contain historical data earlier than approx 1st of november 2013 that would be valid for backtesting it with MT4 expert advisor.

My algorythm acts when a new candle is opened, it checks if EMA1-EMA2 was > 0 on the previous candle and EMA1-EMA2 was < 0 on the cande before that or vice versa.

But my excel algorythm and MT4 expert advisor give similar results on timeframe from 1 november 2013 till now.

Right now I am working on feeding MT4 with historical data so I can backtest my expert advisor on all historical data and check if my excel algo is right or wrong.

I have loaded my 1 hour frame from excel to MT4, but I am getting this ugly graphs:

http://s11.postimg.org/ku7mhwuf3/123123_1.jpg

where blue and red lines intersection is a buy/sell point

My guess is that MT4 needs not only 1H scale but all the smaller ones including 1M. Or maybe it needs all ticks, but I can't find aa way to load ticks into it.

I wonder what algorythm do you use to extract candels from that csv data file?
Queeq
Sr. Member
****
Offline Offline

Activity: 426



View Profile
December 20, 2013, 01:55:39 PM
 #1079

I think I found an error in my EMA calculation. Different examples use numpy convolution function to do that. However I found that EMA changes more slowly despite that it must be more sensitive. Reversing the weights did the trick. So here's the result using corrected algorithm for data from 1st of the November:





My algorithm tracks each pair of MAs and does buy or sell simulation when they cross. Selection of buying or selling depends on which one goes below (if fast is going below slow - sell, and vice versa). I also subtract 0.2% fee from each operation.
I use an array of closing prices for each time interval (1 hour or other) and another array of corresponding values of MAs for these prices. Prices used for buy/sell simulation is the closing price of such interval.

Python backtesting and bot script at Github
Development halted due to other priorities.
helluvaname
Newbie
*
Offline Offline

Activity: 28


View Profile
December 21, 2013, 09:54:00 AM
 #1080

So here's the result using corrected algorithm for data from 1st of the November:

I have run my MT4 script on EMA 21,11 on the same timeframe, here is the report:

https://drive.google.com/file/d/0B3f4gdrrQPzZSXF1NFZhM21CVXM/edit?usp=sharing
https://docs.google.com/document/d/1_8Cm3wC-dQsrecPOqlLmQgLlR49hrhp7DUncCYcF_j4/edit?usp=sharing

It is an .htm file with a gif, can be opened with any browser

starting with $100 on november 1st it appears to end up with $440 at 21:00 December 20

I don't really know how to make those heatcharts with MT4, and I am sick and tired of my excel script, it is so ugly and I just want to forget about it Grin


And, could you please share the way you extract candles from that csv bitcoincharts file, maybe you could post the part of the code responsible for doing that?
Pages: « 1 ... 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 [54] 55 56 57 58 59 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!