Discussion Starter:
The topic of Resource Usage has surfaced a couple of times in the forum generally in the context of backtesting, or mroe acurately, optimizations. But none that i have found address appropriate expectations or an ideal environment to get the most from WL7 with any specifics.
While some may have significant resources (financial, technological, etc) others may not. So, the questions on my mind are:
At this point I'll reference two other posts that look at the topic of Memory Consumption.
It is important that we all understand and accept that the differences in our development environments will have an impact on our experiences, especially in terms of speed and performance.
For example, my older Dual Logical CPU XEON Workstation with 96GB RAM, large speedy SSD's and an out-of-the-box configuration runs at a factor of 3-5 times faster compared with my trusty notebook (i5, 8GB RAM) on short optimization runs. No doubt this would be greater with some custom tweaking of the XEON environment. The difference in a Long Optimization run is significantly more in the magnitude of 20-30 times faster. I believe this is due to the significantly greater available RAM. But both machines become extremely slow (including periods of unresponsive WL7 GUI) once all available RAM is utilized and paging becomes necessary <-- this is expected and normal.
So, back to my previously stated questions (with an emphasis on Environment and Code)
What can we expect from WL7 (in terms of resource utilization)
1) Are there any benchmark metrics we can use or compare with to help set expectations and identify any introduced memory leaks/excess consumptions within our code. (ie: what is expected increase in RAM consumption per backtest or optimization permutation for a benchmark strategy, settings, dataset combination)
How do we use what we have to get the most from WL7 (system tweaks, upgrades, etc)
1) Is there a "good" hardware configuration that is achievable for the average user?
2) Are there tips, trick, or tweaks we can implement to get more from our current hardware constraints?
3) What would be the most cost effective (Return on Investment) way to increase performance? (Ram, Processor type, scaleable cloud based Virtual Machines)
4) Can we programmatically reduce the Backtester/Optimizer footprint by disabling or clearing properties/objects we believe are unneeded for our objectives?
5) Should we programmatically clear our own objects within strategies, extensions, etc or is .Net garbage collection adequate for resource management?
The above serves as a discussion starting point. My hope is that over time this will become a detailed and actionable list of environment enhancements that enable all users to reach their personal goals through utilizing WL7 in the most speedy and efficient way possible
The topic of Resource Usage has surfaced a couple of times in the forum generally in the context of backtesting, or mroe acurately, optimizations. But none that i have found address appropriate expectations or an ideal environment to get the most from WL7 with any specifics.
While some may have significant resources (financial, technological, etc) others may not. So, the questions on my mind are:
QUOTE:
1) What can we expect from WL7 (in terms of resource utilization), so that we can identify a problem if/when it exists, and...
2) How do we use what we have to get the most from WL7 (system tweaks, upgrades, etc)
At this point I'll reference two other posts that look at the topic of Memory Consumption.
QUOTE:
Intensive Memory and CPU Load on an Exhaustive optimization
https://www.wealth-lab.com/Discussion/Intensive-Memory-and-CPU-Load-on-an-Exhaustive-optimization-6691
Memory leaks in optimization
https://www.wealth-lab.com/Discussion/Memory-leaks-in-optimization-6252
It is important that we all understand and accept that the differences in our development environments will have an impact on our experiences, especially in terms of speed and performance.
For example, my older Dual Logical CPU XEON Workstation with 96GB RAM, large speedy SSD's and an out-of-the-box configuration runs at a factor of 3-5 times faster compared with my trusty notebook (i5, 8GB RAM) on short optimization runs. No doubt this would be greater with some custom tweaking of the XEON environment. The difference in a Long Optimization run is significantly more in the magnitude of 20-30 times faster. I believe this is due to the significantly greater available RAM. But both machines become extremely slow (including periods of unresponsive WL7 GUI) once all available RAM is utilized and paging becomes necessary <-- this is expected and normal.
QUOTE:
Known Significant Factors that influence performance:
---------------------------
- Amount of Processing (calculations, permutations, type of optimization, size of dataset - think bars not timeframe, etc)
- Environment Hardware (CPU type, age, chipset memory, RAM, Drive Type and Speed)
- Power Consumption <-- Notebooks not connected to a power source may reduce processor load as part of their "power management" settings. Generally, a notebook/laptop connected to power will compute several times faster than one running on battery power alone.
- Environment configuration (enabling, disabling cores, VMware, other resource consuming tasks <-- think indexing and Antivirus especially if you are creating or modifying files)
- Code (structure, object utilization, garbage collection and disposal, etc)
So, back to my previously stated questions (with an emphasis on Environment and Code)
What can we expect from WL7 (in terms of resource utilization)
1) Are there any benchmark metrics we can use or compare with to help set expectations and identify any introduced memory leaks/excess consumptions within our code. (ie: what is expected increase in RAM consumption per backtest or optimization permutation for a benchmark strategy, settings, dataset combination)
How do we use what we have to get the most from WL7 (system tweaks, upgrades, etc)
1) Is there a "good" hardware configuration that is achievable for the average user?
2) Are there tips, trick, or tweaks we can implement to get more from our current hardware constraints?
3) What would be the most cost effective (Return on Investment) way to increase performance? (Ram, Processor type, scaleable cloud based Virtual Machines)
4) Can we programmatically reduce the Backtester/Optimizer footprint by disabling or clearing properties/objects we believe are unneeded for our objectives?
5) Should we programmatically clear our own objects within strategies, extensions, etc or is .Net garbage collection adequate for resource management?
The above serves as a discussion starting point. My hope is that over time this will become a detailed and actionable list of environment enhancements that enable all users to reach their personal goals through utilizing WL7 in the most speedy and efficient way possible
Rename
Currently there are no replies yet. Please check back later.
Your Response
Post
Edit Post
Login is required