I ran a comprehensive optimization for 11 parameters in the range of 100 thousand iterations on a history of 1.5 million minute bars. At first, everything was fine, the estimated time is a couple of days. But after 5 hours, the wheel showing the activity of the process began to slip and then stop at all periodically, until it stopped completely. The task manager showed that the application is not responding, while the memory consumption of the program has increased to 30 gigabytes. In total, the system has 64 gigabytes. I had to close the program.
The conclusion is obvious, somewhere there are memory leaks that accumulate and hang the program after some time. You need to conduct large-scale tests on a large history and eliminate leaks.
The conclusion is obvious, somewhere there are memory leaks that accumulate and hang the program after some time. You need to conduct large-scale tests on a large history and eliminate leaks.
Rename
The primary suspect of a potential memory leak is the user's Strategy code because C# provides him with enough rope to shoot himself in the foot. If you can share the script that demonstrates the anomaly we could look into the case and see if something unexpected takes place. Thanks.
Unfortunately, I can't disclose the code. This is a trade secret. But I can say that exactly the same logic on WL6 is optimized for days and weeks without any problems. Therefore, I would still suggest that you run tests for minutes on history in 7-10 years.
Thanks for your suggestion. I believe the problem is rooted in your code because it might have been converted to WL7 in an inefficient manner. We do not have anything to work with currently. If you wish to get the ball rolling, consider providing us with an "as-small-as-possible" code that demonstrates the anomaly without disclosing your know how. Thanks again.
A memory leak means that some allocated memory is retained after an operation is complete and shut down. While an optimization is still underway, it is expected that memory usage will continue to increase, because the results for each run are retained. It’s not accurate to classify this as a “memory leak”.
Yes, I may not have put it quite right, but I absolutely accurately conveyed what was happening. The program should not hang and go to the "Not responding" state. Maybe you have never tested on so much history data and with so many optimization parameters, I do not know. But it doesn't have to be that way, that's for sure. I just wanted you to check it out. Thanks.
Yes, here is another possible hint. Just compare the memory consumption of WL6 and WL7 after a few hours of operation or a day... I think you can guess that something is wrong here. Even though the results for each iteration are saved as you say. In general, compare for yourself.
Your Response
Post
Edit Post
Login is required