Hello! Last weekend I optimized the exit of a strategy (contains transaction weight…) with "StochK". The best results were around 22% APR. I took over the parameters. On Monday I backtested it again and the APR was only 17.9%.
I optimized again. Again, the best values were about 22%. With this I started a backtest with a right click. Result: There it was again only 17.9%. At next optimisation just 20 % the highest APR's. What does is mean and how do I get the “right” APR?
Thanks.
Marko
I optimized again. Again, the best values were about 22%. With this I started a backtest with a right click. Result: There it was again only 17.9%. At next optimisation just 20 % the highest APR's. What does is mean and how do I get the “right” APR?
Thanks.
Marko
Rename
Hello,
This has been discussed many times before, please see the FAQ > Every time I run a Strategy I get a different result. Why?
This has been discussed many times before, please see the FAQ > Every time I run a Strategy I get a different result. Why?
Hello Eugene,
i meant, I just get the first time directly after optimization another (better) result. After that allways the same results. I already run the optimization with Exhaustive and Exhaustive (Non-Parallel). Nearly the same problem. Do you have another idea?
Thank you.
Marko
i meant, I just get the first time directly after optimization another (better) result. After that allways the same results. I already run the optimization with Exhaustive and Exhaustive (Non-Parallel). Nearly the same problem. Do you have another idea?
Thank you.
Marko
QUOTE:I'm not sure how to take this statement. Does it mean that you assigned transaction weight to the exits? (That would make not a difference. T.weights are used only to prioritize entering positions.)
I optimized the exit of a strategy (contains transaction weight…)
If that's not it, we probably need more information (ideally strategy code) to determine the source of the variation.
I found this video helpful as a starting point: https://youtu.be/ECCF5zdnC7U
They don't mention it in the video, but I'd use a different indicator for the transaction weight value than you use as the primary indicator in your strategy. For example, if you are buying when StochK rises above 20, most of your values for StochK should be near 20 so your transaction weights will be similar. You want different weights for each transaction which are consistent with each run of the optimizer.
They don't mention it in the video, but I'd use a different indicator for the transaction weight value than you use as the primary indicator in your strategy. For example, if you are buying when StochK rises above 20, most of your values for StochK should be near 20 so your transaction weights will be similar. You want different weights for each transaction which are consistent with each run of the optimizer.
i already have a transaction weight (lowest RSI in this case) to the entry. and i have 10% position size an 1,1 margin.
What makes difference is making sure there are zero NSFs in your backtest report. To get going we need more information: strategy code, backtest data settings etc.
NSFs are okay as long as all the entries are deterministically assigned the same values for all runs.
This could be different, though, when using unstable indicators (like RSI) to assign Weight if the starting dates are different.
Anyway, although Dr. Koch accuses us of all the time, we can't do black magic here, so we can't tell you what's going on without seeing what's going on. Make a video, give us a strategy, something.
This could be different, though, when using unstable indicators (like RSI) to assign Weight if the starting dates are different.
Anyway, although Dr. Koch accuses us of all the time, we can't do black magic here, so we can't tell you what's going on without seeing what's going on. Make a video, give us a strategy, something.
Hi Eugene and Cone,
in the positions are no NSF's. I made sure that.
Do you have a "personal" email to contact just you to give you more information?
Thank you.
Marko
in the positions are no NSF's. I made sure that.
Do you have a "personal" email to contact just you to give you more information?
Thank you.
Marko
Please send private queries to support@wealth-lab.com.
I send the email some minutes ago.
Marko
Marko
Thanks, received. I optimized it using Exhaustive (Non-Parallel) and on each of its 3 runs got identic results when choosing "Run a backtest with these parameter values".
So far, "Run a backtest with these Parameter Values" are always the same for me too.
Hi Cone,
but when I take these parameters (by hand) and run a backtest, I get 19,1% APR .
Could you please do it in this way ?
Thank you.
Marko
but when I take these parameters (by hand) and run a backtest, I get 19,1% APR .
Could you please do it in this way ?
Thank you.
Marko
Same result here (even a couple days later). Are you changing the parameters here, with the sliders?
No, I didn't change any parameters
right now I'm running once again...just 17,88% APR... as you can see in the picture below (without any changes).
and with parameters Eugene optimized (Period 21, Value StochD 88, Bars 3) I don't get 23,3% APR of Eugene's optimization ...
there are just 19,1 % (as you can see below)
there are just 19,1 % (as you can see below)
and with parameters Eugene optimized (Period 21, Value StochD 88, Bars 3) I don't get 23,3% APR of Eugene's optimization ...
there are just 19,1 % (as you can see below)
there are just 19,1 % (as you can see below)
which APR do you get by using Eugene's parameters?
Maybe you're having some data issue. Try to highlight the W-D S&P 100 DataSet in the Data Manager, switch to the Data Truncation tab (at the bottom), select all symbols and choose "Delete all data", then "Perform truncation". Once you notice that all symbols have 0 bars, update it and rerun the backtest.
You:
Me:
You:
If you're not clicking "Run a Backtest with these Parameter Values" from the optimization, and, you're not manipulating the sliders - which are the parameter values that the Strategy uses - it begs the question, "how exactly are you changing the parameters to duplicate the optimization results?"
QUOTE:
but when I take these parameters (by hand) and run a backtest, I get 19,1% APR
Me:
QUOTE:
Are you changing the parameters here, with the sliders?
You:
QUOTE:
No, I didn't change any parameters
If you're not clicking "Run a Backtest with these Parameter Values" from the optimization, and, you're not manipulating the sliders - which are the parameter values that the Strategy uses - it begs the question, "how exactly are you changing the parameters to duplicate the optimization results?"
In this case (strategie) I take over the values/numbers of the optimization and I put them in design surface by hand. Then I run a backtest.
What are your results, if you do it in the same way?
Thank you.
P.S.:By the way, another strategie has the same problem. And it doesn't matter wheather I do in the way here or when I Start the "button" "Run a backtest with these Parameter values". The APR's of the results of a backtest are always lower then the opimization. And I did optimization with "exhaustive non parallel" as Eugene said.
What are your results, if you do it in the same way?
Thank you.
P.S.:By the way, another strategie has the same problem. And it doesn't matter wheather I do in the way here or when I Start the "button" "Run a backtest with these Parameter values". The APR's of the results of a backtest are always lower then the opimization. And I did optimization with "exhaustive non parallel" as Eugene said.
Earlier today, I refreshed my Wealth-Data cache because of some errors in the data from last week (and I didn't have Preferences > Data > Allow Data Corrections from H.P.s' selected.
So that we're working with the same data:
1, please refresh your Wealth-Data cache. Right click Wealth-Data in the Historical Providers and "Delete Local Files" (even though we're only testing from 2000 to 2018).
2. go to Event Providers, check Wealth-Data, and put checks next to Splits and Dividends.
3. go to Preferences > Backtest > Collect Dividends on Backtest Positions - checked
4. just to be safe, Preferences > Backtest > Futures Mode - not checked
My baseline run with refreshed data is 21.53% APR (2000 - 2018, Advanced Settings: Retain NSF Positions - Not Checked). I saved the backtest Positions in case I get another run with different results for comparison. Dividends make a difference and since I'm always reconfiguring for testing, I don't know what my settings were for the previous test.
So that we're working with the same data:
1, please refresh your Wealth-Data cache. Right click Wealth-Data in the Historical Providers and "Delete Local Files" (even though we're only testing from 2000 to 2018).
2. go to Event Providers, check Wealth-Data, and put checks next to Splits and Dividends.
3. go to Preferences > Backtest > Collect Dividends on Backtest Positions - checked
4. just to be safe, Preferences > Backtest > Futures Mode - not checked
My baseline run with refreshed data is 21.53% APR (2000 - 2018, Advanced Settings: Retain NSF Positions - Not Checked). I saved the backtest Positions in case I get another run with different results for comparison. Dividends make a difference and since I'm always reconfiguring for testing, I don't know what my settings were for the previous test.
Hi Cone,
at first - thank you for your nice support.
I did all you said.
I just had a difference at point 3. Rest like you. A backtest with my original parameters gets 17,88% APR and by using Eugenes 19,1% .
There is a difference between yours...
at first - thank you for your nice support.
I did all you said.
I just had a difference at point 3. Rest like you. A backtest with my original parameters gets 17,88% APR and by using Eugenes 19,1% .
There is a difference between yours...
Parameters, right. I should have specified my set - they're the one's in my image above: 21, 88.00, 3 - the same as Eugene's.
Are you sure, that you're applying dividends?
If I remove the effect of dividends, APR goes to 19.71%.
Also, double check Position Sizing:
100K, 10% of Equity, Market Close this Bar, Margin Factor 1.1
Do me a favor, run the strategy with the settings above, right click, copy, and paste the Positions to text or Excel file, and send them to support@wealth-lab.com. (Ignore the auto-response email if you get one.)
Are you sure, that you're applying dividends?
If I remove the effect of dividends, APR goes to 19.71%.
Also, double check Position Sizing:
100K, 10% of Equity, Market Close this Bar, Margin Factor 1.1
Do me a favor, run the strategy with the settings above, right click, copy, and paste the Positions to text or Excel file, and send them to support@wealth-lab.com. (Ignore the auto-response email if you get one.)
you:
Are you sure, that you're applying dividends?
me: yes i did.
but today morning APR is 20,86 %.
and if dividends are not checked: 19,1 %
but i have Commission - maybe here is a difference:
% of Trade Value: 0,005
Minimum Commissions (checked): 1
you: Do me a favor, run the strategy with the settings above, right click, copy, and paste the Positions to text or Excel file, and send them to support@wealth-lab.com. (Ignore the auto-response email if you get one.)
me: yes, I did
Are you sure, that you're applying dividends?
me: yes i did.
but today morning APR is 20,86 %.
and if dividends are not checked: 19,1 %
but i have Commission - maybe here is a difference:
% of Trade Value: 0,005
Minimum Commissions (checked): 1
you: Do me a favor, run the strategy with the settings above, right click, copy, and paste the Positions to text or Excel file, and send them to support@wealth-lab.com. (Ignore the auto-response email if you get one.)
me: yes, I did
Thanks. Commissions - good catch. I overlooked that. Mine were set to None.
With None => 21.53%
With 0.005%, $1 Minimum => 21.82%
The result seems counterintuitive, but adding commissions had an effect on trade selection as the average profit % increased from about 0.503% to 0.511% (in dollar terms it was about $20 more per trade on 8,643 trades).
We should have made sure that everything in the Backtest Preferences were the same - maybe you can spot more differences -
I'm guessing that you have some interest or margin setting that's different that mine shown.
As for the trade comparison, about the first 9 months of trades were precisely the same, except for position sizing, where occasionally the sizes differed by 1, and later 2, shares. Ultimately this lead to a different trade about 8 months into the simulation. Eventually, the differences add up, and the trading is different due to different sizing, based on equity and buying power.
Again, this is likely due to a different setting that affects the Equity curve.
With None => 21.53%
With 0.005%, $1 Minimum => 21.82%
The result seems counterintuitive, but adding commissions had an effect on trade selection as the average profit % increased from about 0.503% to 0.511% (in dollar terms it was about $20 more per trade on 8,643 trades).
We should have made sure that everything in the Backtest Preferences were the same - maybe you can spot more differences -
I'm guessing that you have some interest or margin setting that's different that mine shown.
As for the trade comparison, about the first 9 months of trades were precisely the same, except for position sizing, where occasionally the sizes differed by 1, and later 2, shares. Ultimately this lead to a different trade about 8 months into the simulation. Eventually, the differences add up, and the trading is different due to different sizing, based on equity and buying power.
Again, this is likely due to a different setting that affects the Equity curve.
thank you for your answer. But we didn't find the reason of difference in backtests.
here are my settings for backtest.
Signals Sort and Position Matching are different to your settings. Is it important?
here are my settings for backtest.
Signals Sort and Position Matching are different to your settings. Is it important?
additionaly my settings for trading
Whatever it is, it's something very small that causes a tiny change in the equity curve, that affects sizing after 8 months of trading, and finally ends up affecting when more trades are put on and therefore the final result.
1. Your machine is 64-bits? (Do 32-bit machines still exist?)
2. Open Tools > Markets & Symbols and check the Symbols. Are any of the S&P symbols listed there? Remember that # is a wildcard character.
Please right click, copy and send us the Equity/Cash curve data. Comparing that should at least reveal where the first difference occurs.
1. Your machine is 64-bits? (Do 32-bit machines still exist?)
2. Open Tools > Markets & Symbols and check the Symbols. Are any of the S&P symbols listed there? Remember that # is a wildcard character.
Please right click, copy and send us the Equity/Cash curve data. Comparing that should at least reveal where the first difference occurs.
(1. @Cone: WL7 is built for 64-bit only, that's for sure.)
3. Have you refreshed (not updated, but deleted and reloaded) your data?
3. Have you refreshed (not updated, but deleted and reloaded) your data?
to 1. yes, i have a 64-bit machine
to 2. I not 100% sure, what do you mean. Thatswhy I send you another email with a excel (copys and screenshots are in it)
to 3. Yes, I hope I did it right, too. . . right click on "WealthData" -> delete local files . Was it correct?
and are my settings correct (picture below)
to 2. I not 100% sure, what do you mean. Thatswhy I send you another email with a excel (copys and screenshots are in it)
to 3. Yes, I hope I did it right, too. . . right click on "WealthData" -> delete local files . Was it correct?
and are my settings correct (picture below)
Today once again APR : 19,1 % .
Do you have the same?
thank you for your help :)
Do you have the same?
thank you for your help :)
Yes, same 21.82% with the commissions. I haven't had a chance to compare the equity/cash curves yet.
Did you solve the problem?
Marko, have you tried WL8 B4 (or better, B5 soon)?
Hi Eugene,
no, I have not tried WL8. Is it necessary? Does it have the "Extended Score Card" (in metrics report) and the other funktions?
Thank you :) .
Marko
no, I have not tried WL8. Is it necessary? Does it have the "Extended Score Card" (in metrics report) and the other funktions?
Thank you :) .
Marko
Sure, WL8 has everything WL7 has and more every week. WL7 development is done - if there's a fix for anything, you'll only see it in WL8.
You can install WL8 and continue to use WL7 side-by-side.
We weren't able to duplicate what you were seeing and as I recall, we were trying to figure out how we were getting different results with the same settings. Several hundred trades were precisely the same and then some small difference crept in, and the differences begin to snowball, understandably.
See if you can duplicate the issue in WL8. If you can, let's schedule a zoom call and go over it.
You can install WL8 and continue to use WL7 side-by-side.
We weren't able to duplicate what you were seeing and as I recall, we were trying to figure out how we were getting different results with the same settings. Several hundred trades were precisely the same and then some small difference crept in, and the differences begin to snowball, understandably.
See if you can duplicate the issue in WL8. If you can, let's schedule a zoom call and go over it.
Your Response
Post
Edit Post
Login is required