Extremely-slow-performance-on-first-data-load

QUOTE:
It will take 1-2 minutes with 100% load of 1-2 CPU cores just to load the data, instead of regular 2-3 seconds (so, around 50 times slower). Any subsequent runs / symbol loads will be as fast as usual.

That's likely because you're using one of the free Event Providers that's scraping fundamental data from some slow website. What I do is uncheck all the free event providers before downloading any morning data. Then initial data downloads go fast. (You can re-enable them later if you want to examine individual stocks for their fundamentals.)

If this is a real irritation, then your should consider subscribing to one of the paid fundamental event providers, which are much faster.

Personally I'm not seeing this. Checking which Event Providers you have enabled is a good idea.

Thank you for the suggestions.

I've never installed any free providers, so only default were selected. I unchecked all (Historical Providers and Event Providers tabs) except for WealthData - same stuff, extremely slow loading.

One strange thing is that I can't uncheck several specific ASCII files. When I uncheck - the app hangs for several minutes. Subsequent check/uncheck are instant (before the app is restarted). Some other ASCII files can be checked/unchecked instantly. Some cache corruption?

As I hadn't changed neither settings, nor providers before update to B135, can you please give me a link to B134, to make sure if it's the new build or not.

Addition: I isolated the problem, not all ETFs are equal, please try any daily strategy on SMH (WealthData), or Saylor Shiff on BTC.USD (Cryptocompare)

Thank you

QUOTE:
I unchecked all (... Providers tabs) except for WealthData - same stuff, extremely slow loading.

So you're saying it's a WealthData problem with build 135. Interesting.

QUOTE:
So you're saying it's a WealthData problem with build 135. Interesting.

I can't be sure, as I don't have B134 installer. It can be a very simple and effective (not to waste your time) check if someone gives me the previous build.

I've been using WL for 1.5 years ~5 hours a day on average. Never seen anything like that. Disabled antivirus, checked HDD health...

Even simpler. Open Wealth-Lab -> new Chart -> enter SMH and press Enter, CPU starts to work hard, in 2 mins the data is loaded

Purged all SMH.QF / SMH.QX files, no luck

It doesn't happen at my end, but I don't doubt you since I witnessed other strange and (so far) unexplained behavior on your computer running WL8. Anyone else seeing this?

I donot see SMH from WealthData, only from Norgate.

Right, SMH isn't covered, but it is available from QData.

QUOTE:
I witnessed other strange and (so far) unexplained behavior on your computer running WL8.

I think that there may be some connection between the issues.

Some additional observations.

When I enter SMH in the Chart symbol name, it shows me a single option - Yahoo:


If I click on the (Yahoo:)SMH in the dropdown the data is fetched/depicted instantly. And if I get back to the editbox and enter "SMH", then press Enter, the same CPU load starts again:


Here is how it looks in the Task Manager:


After several minutes of high CPU load it finishes with Yahoo data:


Any consequent attempts to enter "SMH" and press Enter, or open another Chart window complete instantly.

So I believe there is some check of available data sources which consumes a LOT of CPU.

I could see the following error:


I was wondered to see this message, as my ASCII datasources don't contain SMH. So I disabled all ASCII datasets. After that everything works fine. I found a dataset that causes the problem, it's 15m set for SP500 (obviously no SMH there). Any attempt to turn it on or off in DataManger hangs the app for 5-7 mins. It's 6Gb of data. Nice to have a workaround, but the strange thing is that this data set was turned on for at least half a year, I never disabled any ASCII datasets after I added each of them.

So I think the problem is caused by a big ASCII data set that is turned on and doesn't contain SMH. The app tries to find SMH and opens each file that takes a lot of time. It seems that the algo doesn't rely on ASCII file names (no SMH there), and reads/parses full 6Gb content. Seems to be a bit too meticulous.

The only unclear question is why it worked before. I don't remember the exact order of data sources before, but currently I see Yahoo as the last element in the list. So if the app looks for the first match, it first analyzes the big 15min data source with all those delays.

Ok I can work with that! Let’s see if I can resolve this for the next Build cycle.

I managed to find B133, same thing there. So I apologize for blaming B135.

As I didn't change the order of providers, the only possible reason for the behaviour was that I somehow disabled Q-Data which was right after WealthData. And as Yahoo was the last in the list, the app started to load all ASCII data.

Great that you can resolve this, thank you very much!

If you need some large sample data I can send it to you.

That would help, zip it up and send it over via email.

I sent an email to support@wealth-lab.com with the Topic "Slow ASCII market data". The file is ~700Mb, so I uploaded it to Google drive

Thanks

Could this be an issue?

That looks unrelated, is this something you’re just recently experiencing?

Not sure. When I am not having issues I don't go looking. ;)

But I am having the same issue as the OP.

I seems that the issue only occurs with ASCII datasets on my system.

Just tried the latest build, put the Cryptocompare data set in the end of the list (after all ASCII datasets), and run Saylor Schiff's strategy. Loaded the dataset very fast!

Thanks very much for the fixes!