Hello,
I recieved a couple of strange errors when I was trying to run an admittedly large shrinking window WFO. I was trying to get a maximum amount of runs with the day of processing I was willing to devote to the test. Unfortunately after several hours, I got this Exception shown below. This happened two other times after a fresh restart. There were then no OOS shown on the tabular. My machine has a very high amount of RAM: 128GB.
I also recently installed the trial version of the finatic.optimizers. expansion.
I am wondering if it was a problem with what I was attempting or if it is an actual bug.
Here are the relevant screenshots.
I recieved a couple of strange errors when I was trying to run an admittedly large shrinking window WFO. I was trying to get a maximum amount of runs with the day of processing I was willing to devote to the test. Unfortunately after several hours, I got this Exception shown below. This happened two other times after a fresh restart. There were then no OOS shown on the tabular. My machine has a very high amount of RAM: 128GB.
I also recently installed the trial version of the finatic.optimizers. expansion.
I am wondering if it was a problem with what I was attempting or if it is an actual bug.
Here are the relevant screenshots.
Rename
Yep this is a known issue, will be resolved in the next build 👍
I just got this out of memory error running a long exhaustive WFO with 100000 permutations. After about 20 hours of processing it gave up the ghost.
Yes, we let you push your hardware to the very limit, and looks like you found the limit!
If you are using a C# strategy, try using new instead of Series for the indicator creation. Series is going to add to the cache, and using new is probably going to slow things down, but may avoid out-of-memory problems. It can be like walking a tightrope.
Is there any log information I can find somewhere so I can figure out what might have been the problem? I have so much memory on this machine I am surprised it might have used it all. I will have to watch it next time.
@dandude - is it a block strategy or a C# strategy?
Block
If the indicators you use are caching any intermediate results, (e.g. like calling EMA.Series() to get an intermediate result used in calculating the final result), then there is nothing you can much you can do about the caching other than change your strategy, or generate it in C# and the play games with clearing caches. I don't recommend that.
Also, consider using SMAC optimizer...
https://www.wealth-lab.com/Discussion/KnowHow-Optimizers-10760
But, I don't know if that would use less memory. It might cut down on both memory and time. If you don't own it, you could trial it.
https://www.wealth-lab.com/Discussion/KnowHow-Optimizers-10760
But, I don't know if that would use less memory. It might cut down on both memory and time. If you don't own it, you could trial it.
You're probably trying to run too many processor cores at once, so it chokes on all the memory requirements of those parallel threads. I would reduce the number of cores used for optimization down to 3 or 4 so the memory demand is more manageable. You may even find it will run faster with less cores involved. The Windows OS is very inefficient when it comes to process swapping.
"Reduce the number of cores" A very interesting idea. Where do I accomplish this?
QUOTE:
"Reduce the number of cores" A very interesting idea. Where do I accomplish this?
It's easily controllable with Windows 11, but I'm running Windows 10. Ask a Windows 11 user.
You have plenty of memory, so that's not why you're running out of memory. You knew that. But if you're trying to run 16 parallel threads at once, of course you'll run out of memory. So don't do that.
The best solution is to select your processor chip carefully. That's why motherboards come without processor chips. For WL execution, you want a processor chip that maximizes L3 on-chip cache size and core speed while minimizing the number of processor cores. Hardware selection is very important.
@dandude - to set an application's processor affinity in Windows 11 do the following...
Run WealthLab
Right-click on the Windows taskbar
Select Task Manager
In Task Manager, select the details page (highlighted on the left in the image below)
In Task Manager, Right-click on WealthLab.exe and select Set affinity
A dialog will appear. Check/uncheck for the processors you want to use. Note the puny scroll bar on the left you use to scroll for more processors depending on your hardware.
Run WealthLab
Right-click on the Windows taskbar
Select Task Manager
In Task Manager, select the details page (highlighted on the left in the image below)
In Task Manager, Right-click on WealthLab.exe and select Set affinity
A dialog will appear. Check/uncheck for the processors you want to use. Note the puny scroll bar on the left you use to scroll for more processors depending on your hardware.
Your Response
Post
Edit Post
Login is required