YCharts-errors-on-WL8-startup

Hmm, I'm getting the fundamental data from YCharts, I wonder if you're running into some throttling issues here.

YCharts is known for systematically retracting the fundamental items from "public domain" (i.e. when their partial history is visible on the website). No wonder if it happened to the cash flow.

Otherwise we haven't made any changes to YCharts since May so nothing could affect it on our end:
https://www.wealth-lab.com/extension/detail/Fundamental#changeLog

Here is some info from YCharts about their rate limitting:

https://ycharts.com/api/docs/rate_limits.html

It doesn't look like our provider enforces this rate limit, so I'll add this throttling to the next release.

OK but just to make the context clear, @superticker has not been using the paid YCharts API. The 'free' data comes from a different endpoint.

QUOTE:
Here is some info from YCharts about their rate limitting:

Is that for the paid subscription or free subscription? I'm using the latter.

Again, this is happening on startup with no strategies loaded. So does the Fundamental provider load all the event data before any strategies get started? If so, is that a good idea?

I found the text of the log error message (with the YCHarts typo) and applied the rate limiting there. Is this not hitting YCharts?

And no, it should not need to make a request like that at startup, not sure what’s it’s doing but will investigate.

QUOTE:
it should not need to make a request like that at startup, not sure what’s it’s doing but will investigate.

I'm guessing it has something to do with the timing of when WL8 B26 loads the Fundamental provider.

Good and bad news. The good news is WL8 B27 and the Fundamental Ext B6 fixed the SSL errors that occurred on startup. I did need to clear local files for some of these event providers to bring them back to life.

The bad news is that YCharts has stopped working altogether. This is the "free" version of the YCharts subscription, not the paid REST endpoint. I did try clearing local files for YCharts, but that didn't help.

I do get earnings from Zacks okay and dividends for WealthData okay. But nothing from the free subscription of YCharts. Is anyone else seeing this?

Oops, I spoke too soon. I'm still getting SSL connection errors for YCharts and this event provider isn't working at all. This is for the "free" scraper version, not the paid REST endpoint version. Running WL8 B27 and Fundamental B6.

I wonder if YCharts black listed your IP? We’re unable to duplicate this error!

Well, I don't have a VPN or laptop, so changing my IP address would be difficult.

So you're saying WealthLab was doing bad things earlier to YCharts, so the black listed my IP address? What's the web address of the YCharts page the provider scrapes? Perhaps I can try going there and seeing if I get anything.

And I can wait a few days to see if the black listing clears.

It’s only a guess. I don’t know for sure.

QUOTE:
It’s only a guess.

Well, I went to the YCharts website and it loads just fine. Is there a specific page I should try to load for testing my IP?

What other tests can I do at my end to narrow down the problem?

I’m about to head to the airport and will be traveling all day, but hopefully Eugene can pick up with the answers.

We may speculate about YC blacklisting you due to making way too many requests. They do have means for this but in my experience it gets cleared up pretty quickly so I'm not a supporter of the theory.

The (unreproducible) error message itself is the key: establishing an SSL connection in 2023 looks really odd. SSL has long been deprecated, now it's TLS all over. That's why your browser opens the YC website ordinarily.

It's an issue purely at your end, something is misconfigured in your legacy Windows 7 system to let .NET prefer SSL which gets refused by web server then. To say VERY politely, the hardware make and firmware date are not present-day. We may look into ways to disable this behavior in a future build of WL but since your case is the only, consequences of this should be weighed.

P.S. We will not reveal endpoints of services that our providers hit to scrape - that's our edge, please don't even ask going forward.

QUOTE:
It's an issue purely at your end, something is misconfigured in your legacy Windows 7 system to let .NET prefer SSL which gets refused by web server then. To say VERY politely, the hardware make and firmware date are not present-day. We may look into ways to disable this behavior in a future build of WL

So why does Firefox use TLS and WL use SSL then? And how do I configure WL so it connects via TLS like every other application on my system?

Shouldn't .NET negotiate the connection protocol with the web server?

QUOTE:
Shouldn't .NET negotiate the connection protocol with the web server?

Yes and that's how it happens for everyone. But your system is far from being modern and we simply do not have an idea what settings you might have applied somewhere which may be affecting WL's HttpClient.

Check your system registry and IE settings to make sure nothing overrides or disables TLS 1.3, 1.2 etc.

BTW, Windows 7 is unsupported.

@superticker: I think that this should help:
https://learn.microsoft.com/en-us/mem/configmgr/core/plan-design/security/enable-tls-1-2-client

Apparently, you need to install an update for Windows 7:
https://support.microsoft.com/en-gb/topic/update-to-enable-tls-1-1-and-tls-1-2-as-default-secure-protocols-in-winhttp-in-windows-c4bd73d2-31d7-761e-0178-11268bb10392

The Windows update discussed in the second link in Reply# 18 was already installed as part of the Windows Update "Recommended" fixes. However, none of the registry patches were installed. The "Easy Fix" regedit installer installed some of those patches, but not the ones for SCHANNEL, so I added those. That took care of all the requirements for the second link.

The first link has regedit patches for the .NET framework. Those need to be created manually. But after doing all this, it still doesn't work; same problem. I'm wondering if the regedit patches below (from the first link for .NET) are just for .NET 4.6.2 thru 4.8? Would they also work for .NET 6.0?

I'm assuming the "free" YCharts provider is employing WinHTTP to connect with YCharts in the first place, which is what these patches target. Is this correct? Or is the free YCharts provider using an ODBC connection, which these patches don't target?


At any rate, all applications work fine on my system except WL8. Why is WL8 the only application with this problem? Why is WL8 special? All the web browsers can open YChart pages just fine. I think we are missing something.

It's important to keep the older security protocols, SSL 3.0 and TLS 1.0, enabled for WinHTTP to work with the WL6 providers, so the second "A" in the DefaultSecureProtocols value below is critical.

CODE:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\WinHttp\
      DefaultSecureProtocols = (DWORD): 0x00000AA0
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Internet Settings\WinHttp\
      DefaultSecureProtocols = (DWORD): 0x00000AA0

QUOTE:
Would they also work for .NET 6.0?

I don't know that but since you're running an operating system officially unsupported by Microsoft, anything can happen there. 🤷‍♂️

How about if I use the Concierge Support Service to fix the problem with the YCharts provider. I can pay for it. Certainly it should be too hard to force using TLS at the code level.

Hi superticker, some research is indicating that this is out of our control, and we can't accept a Concierge Support item on an unsupported OS (Windows 7). It may be time to finally bite the bullet and upgrade your system?

https://stackoverflow.com/questions/62301938/c-sharp-could-not-create-ssl-tls-secure-channel-on-windows-7-windows-server-usi

The reason the stack overflow report fails is because the poster is trying to use .NET 4.5 and the Windows Update patch clearly states one needs to be running .NET 4.6.2 or higher. So his .NET framework is too old for the said patch to work.

Thanks for citing that link. I just updated that answer on stack overflow to clarify why his attempt to enable TLS 1.1 and 1.2 failed on the older .NET framework.

Not sure if this will help, but I found this too:

https://stackoverflow.com/questions/70674832/windows-7-could-not-create-ssl-tls-secure-channel-system-net-webexception

I cross checked all the registry keys with the link in Reply# 24. They are all okay, but the problem persists. I'm afraid the "free" version of the YCharts provider is not connecting with an https channel, so patching WinHTTP is not going to fix this problem. Part of the reason I say this is because none of the other data providers (event or price data) have this problem because they are connecting via an https channel.

Whatever the protocol the "free" YCharts provider is using to connect with, the easiest solution is to patch that specific code that's making that weird connection unique to the free version. And I'm willing to pay to have that code modified.

But patching WinHTTP isn't going to fix this problem since https isn't involved. I suppose using the paid REST endpoint for YCharts would work since that has to use https (otherwise, it wouldn't be REST). But I was hoping to get some free event data via YCharts.

I’m sorry but there’s no weird connection at our end, it’s simply using the standard .NET HttpClient class and the connection is working correctly at our end.