Data Load for my EOD backtest takes 7 minutes since Build 24. Previously it took under a minute. CPU is near 100% for this period. (224 symbols, 10 years, daily bars)
Log Viewer shows several messages from Yahoo (below). I'm doing a backtest to test strategy modifications. I'm not trading. Why is WL updating data? This makes no sense. I have all the data I need for strategy development.
Log messages...
Timestamp Source Message Exception
"10/5/2021 13:18:59:478" "Yahoo! Finance" "Error getting History (LOGI): The remote server returned an error: (401) Unauthorized." "The remote server returned an error: (401) Unauthorized."
"10/5/2021 13:20:48:396" "Yahoo! Finance" "Error getting History (FLIR): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:22:05:168" "Yahoo! Finance" "Error getting History (GLUU): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:24:23:683" "Yahoo! Finance" "Error getting History (CBB): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:24:24:164" "Yahoo! Finance" "Error getting History (PLT): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:24:51:273" "Yahoo! Finance" "Error getting History (IPHI): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:25:37:507" "Yahoo! Finance" "Error converting company name (^STI.N): Cannot perform runtime binding on a null reference" "Cannot perform runtime binding on a null reference"
Log Viewer shows several messages from Yahoo (below). I'm doing a backtest to test strategy modifications. I'm not trading. Why is WL updating data? This makes no sense. I have all the data I need for strategy development.
Log messages...
Timestamp Source Message Exception
"10/5/2021 13:18:59:478" "Yahoo! Finance" "Error getting History (LOGI): The remote server returned an error: (401) Unauthorized." "The remote server returned an error: (401) Unauthorized."
"10/5/2021 13:20:48:396" "Yahoo! Finance" "Error getting History (FLIR): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:22:05:168" "Yahoo! Finance" "Error getting History (GLUU): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:24:23:683" "Yahoo! Finance" "Error getting History (CBB): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:24:24:164" "Yahoo! Finance" "Error getting History (PLT): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:24:51:273" "Yahoo! Finance" "Error getting History (IPHI): The remote server returned an error: (404) Not Found." "The remote server returned an error: (404) Not Found."
"10/5/2021 13:25:37:507" "Yahoo! Finance" "Error converting company name (^STI.N): Cannot perform runtime binding on a null reference" "Cannot perform runtime binding on a null reference"
Rename
WL7 will always try to update data on-demand, unless you turn on Offline Mode from the File menu. Since I don't personally use Yahoo data I haven't noticed any issue, maybe Eugene can comment on that? I'd also recommend making sure WealthData (and maybe even QData) is above Yahoo in the Historical Providers tab and are both enabled.
Yahoo is being very slow to update for me in WL6 either. Took 45 seconds to get Dow 30 updated there. Then I truncated the last month worth of data and an update finished in nearly half a minute. :/ Y! was definitely much faster before.
Minimal, if any, improvement from Offline Mode. A backtest run takes two minutes. Data load is 5 minutes, 40 seconds. Backtest time was 1 minute, 15 seconds, giving total first backtest time of 6 minutes, 55 seconds. Without the load, a subsequent run took 1 minute, 16 seconds. For comparison, this same backtest, in WL6, takes less than ten seconds
Side note: Offline Mode is not persistent. When I restart WL7, it reverts to unchecked. It should be a Preference, not hidden on the File Menu.
Side note: Offline Mode is not persistent. When I restart WL7, it reverts to unchecked. It should be a Preference, not hidden on the File Menu.
QUOTE:
Side note: Offline Mode is not persistent. When I restart WL7, it reverts to unchecked. It should be a Preference, not hidden on the File Menu.
Right, offline mode should not be persistent. It's to be activated every time on demand.
Do you have Chart Patterns Event Provider, or other Event Provider enabled? To increase data loading speed, disable any Event Providers that you don't absolutely require.
That said, we still support WL6 so if you feel it is serving you better then you can keep using it indefinately.
If you want us to help you with WL7, you might share any relevant information such as the strategy in question, dataset and settings. Or email them to us at support@wealth-lab.com if you don't want to share them publicly. Personally, for me WL7 has been MUCH faster than WL6, so there is some disconnect here.
That said, we still support WL6 so if you feel it is serving you better then you can keep using it indefinately.
If you want us to help you with WL7, you might share any relevant information such as the strategy in question, dataset and settings. Or email them to us at support@wealth-lab.com if you don't want to share them publicly. Personally, for me WL7 has been MUCH faster than WL6, so there is some disconnect here.
QUOTE:
Without the load, a subsequent run took 1 minute, 16 seconds.
Is this new observation? What about other data providers?
Glitch, Chart Patterns Event Provider seems to be the culprit. I disabled it and times are what they were. I can't easily share the strategy because of custom WL6 NeuroLab code.
Topic may be mistitled. Perhaps it should reference "Chart Patterns", not "Build 24" or "Yahoo".
Eugene,
Just verifying that the bulk of the time occurs during the data load. Providers? I have WealthData, Yahoo, and CBOE, in that order. I have no idea which it may be using.
Topic may be mistitled. Perhaps it should reference "Chart Patterns", not "Build 24" or "Yahoo".
Eugene,
QUOTE:
Is this new observation? What about other data providers?
Just verifying that the bulk of the time occurs during the data load. Providers? I have WealthData, Yahoo, and CBOE, in that order. I have no idea which it may be using.
I tried to put a warning in the Event Providers, but yes it takes time to calculate all the selected patterns.
Unless the Strategy specifically uses a pattern, why is every Backtest running all selected chart patterns?
Because the data (including Events) are loaded BEFORE the Strategy is executed. It's up to you to disable any Event Providers you won't need.
That is a bad design. This means that, if a user has multiple pattern strategies, it's up to the user to configure the patterns for each strategy before running it. If he/she forgets, what happens? Can the strategy detect that its required patterns weren't selected? You may want to rethink this, considering the expense of pattern detection.
You're not obligated to use the Event Provider. It's there when you automatically want to view patterns on charts, basically. In your Strategy you can use the ChartPatternEngine class (documented in the QuickRef) to detect just the patterns you're looking for. Or just not use it at all.
In addition to the example in the QuickRef (ChartPatternEngine ProcessPattern method) you can also use the Building Block. It uses the ChartPatternEngine internally, and converting the Building Block Strategy to C# Code is another way to see how it works.
In addition to the example in the QuickRef (ChartPatternEngine ProcessPattern method) you can also use the Building Block. It uses the ChartPatternEngine internally, and converting the Building Block Strategy to C# Code is another way to see how it works.
QUOTE:So you're saying the strategy will cause pattern recognition even when its not generally configured. Is that correct?
Can the strategy detect that its required patterns weren't selected?
No, a Strategy will not cause pattern recognition. The process of loading data with the enabled Chart Pattern Event Provider will cause pattern recognition. If you don't desire this, just disable the Event Provider.
The preferred way for a Strategy to work with the patterns is to use the Buillding Block, or the ChartPatternEngine for C# Strategies.
The preferred way for a Strategy to work with the patterns is to use the Buillding Block, or the ChartPatternEngine for C# Strategies.
Your Response
Post
Edit Post
Login is required