Could-not-load-Backtest-Data-in-this-Scale-Range-8

I have a similar but not the same issue with Norgate data that might be related to this. It happens almost every day if I run a dozens of backtests with a few strategies. The only thing I can do then is to restart WL and it will definingly go away until it happens again. I'm not sure how to reproduce this exactly, but usually it happens with the follow sequence of symptoms.

1. A strategy that uses a Norgate dataset backtests just fine.
2. Somehow when I backtest the strategy again with the same dataset, only a very small number of symbols are loaded (In the screenshot i'm using N100 current & past which has 196 symbols if loaded correctly. Here it only shows 13 symbols are loaded when the problem happens).
3. When the data is reloaded and strategy is rerun, the "can not load dataset in this scale/range" happens. At which point I have to restart WL if i want to use the same dataset in any strategies.

2 does not happen right after 1 and usually there are other operations such as running other strategies, etc. This is the part that seems to be random to me so I do not know the exact steps to reproduce this;
3 usually happens right after 2.

Any thoughts about this cause? It seems to be happening on more and more of my new and modified strategies.

It seems to be something wrong with the Norgate extension, so could also write their support to have a look at this thread.

I don't use Norgate. My problem is with data that is strictly ASCII.

One bit of info...

If I copy the strategy into a new C# template with all of the same strategy parameters that sometimes solves the problem.

@Glitch I'll write an email to Norgate.

Does WL cache data in memory to use across different strategies? If possible could WL have a button to clear cached data and force reload all data from source? Since restart WL solve the issue for me, I feel that force reload all data might help in this case. This happened for me more often recently so I had to restart multiple times each day sometime.

You can force a reload by clicking the message at the bottom of the Strategy window that says "N Symbols Loaded". That will change the message to a red "Need to Load Data" and it will do a fresh reload the next time you run the Strategy.

Thanks @Glitch. I actually use that button a lot if i want to reload data for a single strategy. However once the problem I described here happens that won't work and I keep getting the same error with reload.

My last post was based on my guess that there might be a global cache in WL for datasets for all strategies (e.g. two strategies under backtesting use the same dataset), so whether it's possible to invalidate all data in that cache and reload from disk. I might be totally wrong about WL here.

QUOTE:
... once the problem I described here happens that won't work and I keep getting the same error with reload....

We are talking about two different caches. There's a cache created by Data Manager, and there's a cache created by the strategy itself. You can reload the latter one, but you really can't reload the former one without going into Data Manager and selecting "Delete Local Files" for that particular data provider. But if you do that, then that provider has to re-download all the data for all your stocks, which you don't want to do unless those files (which are saved to disk) got corrupted somehow like during a power failure or an OS lockup.

But what you're saying is that there's something wrong with the Data Manager data provider. So are you using Norgate for your data provider?

@superticker
Yes that's what I meant. Now to force reload data from data providers one has to delete local data files. It would be great if one could reset/clear WL in-memory cached data for all data providers with a click of a button without deleting and downloading the data files from source. Then if sometime a data provider is misbehaving due to bugs, like what I'm experiencing now, one can just click on the button to reload all data.

For data source, I use WD for most live strategies if i can use it, but I do have two strategies for R1000 that I use Norgate. And for strategy development I use both.

No, there's no such global cache, each Strategy maintains its own cache.

QUOTE:
It would be great if one could reset/clear WL in-memory cached data for all data providers with a click of a button without deleting and downloading the data files from source.

Well, what's in RAM memory is cached by the strategy itself, and you can reload that, but you're saying that doesn't fix the problem.

What's cached by the Data Manager is cached on disk--not RAM memory. But restarting WL isn't going to fix what's cached on disk. And the discussion is saying restarting WL fixes the problem initially, so what's cached on disk is okay.

So are we saying the process of fetching the data (by the Norgate provider) from disk to strategy is broken? If so, then why does the strategy execute the first time okay but not subsequent times? And if the strategy is corrupting its own copy of the RAM cached data, then why doesn't reloading that data (from the provider's disk cache) fix it?

More importantly, why am I--and everyone else--not seeing this problem? Are you suggesting this is a memory management problem unique to your hardware? If so, have you flashed your BIOS lately? Remember, the BIOS steers the MMU (Memory Management Unit) of your processor. I think your BIOS is throwing a MMU mapping register in the large memory model. (Applications with smaller memory footprints would remain unaffected.)

I suggest ticking "Obtain data from the selected dataset only" - there's crtiical functionality in the data plugin related to the index constituency within the dataset, and I'm thinking this might have an impact on what's happening.

Please let us know if that solves your issue.

If it doesn't, try and create a simple trading system that demonstrates the issue in a repeatable fashion.

I have continued to explore the source of my problem and have found one intriguing aspect.

If the end test date when running a script is before the start of the data for a symbol data the fail occurs, but only for some scripts. In fact, for the vast majority of scripts using the some exact test parameters there is no issue and the run proceeds normally.

I hope this bit of info helps track down the source of my problem.

QUOTE:
the fail occurs, but only for some scripts.

But does the failure occur reproducibly for some scripts? That's a key question.

If the failure is reproducible, then it's probably software. If the failure is intermittent for a given script, then it's probably hardware.

What I would do is try to reproduce the problem with another computer. If you can, then all of us should be able to reproduce it reliably and the application bug can be located.

QUOTE:
But does the failure occur reproducibly for some scripts?

Yes, it always fails on particular scripts now but I never had this problem occur previously with these very same scripts. I had not had a reason to use these scripts again until recently. In the interim Glitch has released 10-15 Builds of WL8 which might be the problem, but I don't know. If I had a computer with Build 125 to test those scripts today I could then be certain, but I don't have that.

I will be leaving my current location and will see if the problem persists when I use a computer that has a Build of WL that is 2-3 months old. That will occur in a couple of weeks.

QUOTE:
Yes, it always fails on particular scripts now but I never had this problem occur previously with these very same scripts.

Then I would say it's a WL problem with ASCII datasets. If I remember right, the ASCII provider was rewritten to be faster by preserving more of its existing binaries; see the release notes for Build 136. (Did you follow those release notes and put all your ASCII files in the same folder?) Sounds like it maybe failing at "appending" incoming ASCII data to existing binaries.

Could the existing binaries be corrupted? Why don't you try deleting those binaries, then try recreating them from the original ASCII data to see if the problem remains reproducible.

Yes, I noticed that item re: ASCII datasets with Build 136 and I deleted all of the data in the ASCII files for all of the Symbols and recreated those files from scratch. They were already in the same folder so that is not an issue. That had no impact on resolving the problem. The same scripts that exhibited the problem prior to that action still had a loading issue afterwards.

I found another strange behavior also. I modified an existing script and tested it and there was no issue loading and running it. I saved it and then loaded that saved copy into that current open copy of WL with the original copy right next to it. The just-saved copy of the script failed to load the Watchlist. Then I proceeded to clone the original copy of the script (I now had 3 copies loaded) and cloned copy ran flawlessly. 😣

So there's inconsistent behavior with 3 copies of the same script and data. And the only difference is their location in virtual address space. So are you trying to tell me you have a mapping problem between physical and virtual memory as well--a memory management problem? If so, have you flashed your BIOS lately as I suggested in Post #12 for similar reasons?

If you really think it's a hardware problem (such as a memory management problem with the MMU BIOS code), then it's also time to try to reproduce it on another machine.

I have the latest update to my BIOS, so that may not be an issue. There is always a possibility that the BIOS Firmware has a coding bug, but that I feel is less likely.

QUOTE:
I have the latest update to my BIOS, so that may not be an issue.

It's possible you may have other hardware problems (RAM memory problems), but based on your description of exact copies of the same script and data failing occasionally (with the only difference being their virtual memory location), I doubt that's the case.

Software problems would be reproducible unless you're dealing with asynchronous I/O and external events (such as a client and server on different clocks). But in your case, everything is under one clock and there are no external asynchronous events coming in from outside of your WL process--or at least there shouldn't be.

If you think there are external asynchronous events that are interrupting your WL process (and creating a nondeterministic race condition), can you suggest what those might be? Enlighten me. And explain why the rest of us aren't seeing those external events and having the same problem?

We are at a crossroads where we need to determine if this is a software or hardware problem. As I said earlier, I would try reproducing it on another computer. You could also run a memory checkerboard test on all your RAM memory. But if it were the RAM, I would typically expect more unpredictable behavior.

---
Maybe the problem is more reproducible than you're suggesting?

If you’d like to email a strategy that is exhibited the behavior as well as the required ascii data in a zip file I can investigate, support@wealth-lab.com

I am planning to do that test on another computer with an 2-3 month old version of WL in the next couple of weeks. That may shed more light on the issue.

I think Post #22 is "implying" that the software bug is reproducible and was introduced in Build 136 when the ASCII provider was revamped. However, if you don't think the problem is reproducible enough--as yet--then hold off forwarding it to the developers until you think it is.

The trick to finding any software bug is tracing the exact steps that makes it reproducible and that can be tricky.

Glitch,

Thanks for the offer. Just sent you a Script which exhibits the problem along with the Watchlist. Any help would be appreciated! Thanks!!

superticker,

I was able to create a stripped down script which reproducibly exhibits the problem and sent it to Glitch. I hope that he can identify and resolve the issue.