Wondering if something like a Header or Footer Row could be added to the Tabular optimization results tab that displays the simple average for each metric of all the runs conducted in the optimization. I believe it could be a helpful tool to quickly get an idea of the average result of the series of parameters tested, as well as how any given run compares to the average to better measure how realistic it might be to achieve a similar result in the future or how abnormal its past performance might have been.
Rename
I wonder if it's possible to leverage the multitude of existing optimization techniques and 3rd party (finantic) extensions to avoid just adding more options?
https://www.wealth-lab.com/Discussion/KnowHow-Optimizers-10760
https://www.wealth-lab.com/Discussion/KnowHow-Optimizers-10760
QUOTE:
... how realistic it might be to achieve a similar result in the future or how abnormal its past performance might have been
What you are looking for here is called a robustness test.
There are (much) better ways than just display the average of all optimizer runs,
for example show the "relative range" of results of the top N% of all optimization runs.
I am working on a tool that computes robustness of a strategy.
Among other things it calculates the stability of result metrics when the optimization parameters are changed (slightly).
Since DrKoch the optimization expert is working on a solution better than imagined by topic starter, let's mark this request as active.
QUOTE:
it calculates the stability of result metrics when the optimization parameters are changed (slightly).
Right. One wants to avoid "local" maxima or minima in the optimization. One could smooth the surface of the solution vector space to "fill in" (i.e. avoid) these local maxima and minima. But I'm guessing the optimizer is already doing something like that.
"Resampling" (which is what this solution smoothing is about) is a common practice in image processing. But in image processing, we are only working with a 2D image. With a WL optimizer smoother, we are working with a much higher dimension in the solution vector space, so smoothing becomes more CPU expensive. Also, in imagine processing we have many more points to smooth with. That's not the case with the optimizer solution vector space. :(
Truthfully, I've had feature requests rejected in the past for potentially over-complicating things, so I was hoping this would be a simple enough feature request to create and implement that would be also useful as a quick snapshot. Totally understand that it's not thorough by any means.
Love that you're working on a robustness test tool though, DrKoch.
Love that you're working on a robustness test tool though, DrKoch.
Your Response
Post
Edit Post
Login is required