hi all,

I am playing around with the fundamental data. mainly YCharts and InsiderCow.
with InsiderCow (InsiderTransactions), i ran into a download quota limit. and i did not downloaded a lot of data...
when i checked this download quota problem a bit more, i have seen that there is also on YCharts quiet a low limit (25M for Pro, Advanced subscription limit unclear?!).
so i begin to ask myself, are these data providers really usable in a professional way in WL?

I would be happy to pay for reliable data providers where there are no download limit problems.
for me, having reliable fundamental data providers is a key point in using WL!

so the basic questions would be:

1)are the existing fundamental data providers really usable? which are ok, which not?
2)are there other providers out there which could be integrated in WL?


thanks
coppola

How did you run into InsiderCow download limit? How did you determine that?

1) The existing data feeds (except YCharts) are free. With free feeds, you get what you paid for. Temporary outage, download limit, service termination - all this happened in the past and may happen again.

2) This question I have already answered:

QUOTE:
c) You're welcome to suggest better sources of data.

In other words, we need some input from you.

hi,

as mentioned, i would be happy to pay for a good provider! i plan to use WL in a professional way.
therefore I need a professional environment, including data providers!
if this is not the case today, it must be changed, for example by integrating good data providers.

QUOTE:
1) The existing data feeds (except YCharts) are free. With free feeds, you get what you paid for. Temporary outage, download limit, service termination - all this happened in the past and may happen again.

my question was, which ones are really usable. is YCharts really usable now??

QUOTE:
2) This question I have already answered:

this question was more related to users, which maybe have a good idea about possible other good providers.

QUOTE:
How did you run into InsiderCow download limit? How did you determine that?

updating with the data manager was not working. i finally went to their site (with the browser) and got a HHTP Response Status with this quota message.

As a WL Dev customer without the fidelity data, other sources of good data must be available for using the platform.
on the non fundamental data side(OHLC), it looks that yahoo is good. and there are also other possibilities (for example IQ).
but on the fundamental side i have some doubts now.

regards
coppola

QUOTE:
i finally went to their site (with the browser) and got a HHTP Response Status with this quota message.

How exactly did the message look (HTTP error code e.g. 403, 429 or 509 and its text)? How many symbols were you able to update before running into the quota? For how long did the limit last?

QUOTE:
this question was more related to users, which maybe have a good idea about possible other good providers.

When demand exists, our users suggest their candidates. Recent example: Multpl.com data provider request

QUOTE:
How exactly did the message look (HTTP error code e.g. 403, 429 or 509 and its text)?

HTTP Response Status 1001
and later Error:
404
with text:
You have exceeded InsiderCow's daily usage quota per user. Please contact support@insidercow.com for details. Please include this message in the email. Thank you

QUOTE:
How many symbols were you able to update before running into the quota? For how long did the limit last?

since about one day, i can't download anything.

QUOTE:
When demand exists, our users suggest their candidates. Recent example: Multpl.com data provider request

as mentioned, this is for me a key issue. I need good providers for fundamental stock data, insider sales and probably also shorting information (shorting ratio etc). so my demand exists.

questions for you would be:

1)when checking possible new data providers, what would be the technical requirement, that they can be integrated in WL. must they provide a web service?
2)if such a provider is found, how much time would this normally take for being implemented in WL?

thanks

1) There's no formal requirement, it really depends. A WebService or formal API is a plus but is not required: most free feeds offer web-based data that can be grabbed and parsed.

Note: it's extremely unlikely that we would even consider implementing against an outdated (e.g. COM/ActiveX) or complicated (e.g. C++) API.

2) It depends on task's complexity, business value, demand, current workload, deadlines for other tasks, and inspiration in the end. My extension release cycle is monthly. A rare project would exceed that term provided I'm working on it full time (but there's the need to satisfy earlier requests).

Note: a low demand and high complexity would mean that the request is likely to get rejected, because as technicians we have to support many users daily. Another candidate for rejection would be a new feed which duplicates one of the existing providers that is known to be stable and/or comprehensive.

QUOTE:
You have exceeded InsiderCow's daily usage quota per user. Please contact support@insidercow.com for details.

Interesting. Have you contacted them? How much symbols have you updated before running into this?

QUOTE:
my question was, which ones are really usable. is YCharts really usable now??

The question is. how reliable is YCharts. During its four years of existence as a Wealth-Lab provider extension, there were about a dozen of breaking changes, partially or completely. The thing is, YCharts has never managed to provide a formal API despite the fact that their customers pledged about it since the beginning. A paid customer clicking "Export data" on their website in a browser will never notice that a "breaking change" has happened, whereas the users of 3rd party solutions like the YCharts provider will notice when a website change has been made.

Nonetheless, with a little help from YCharts users we were able to react to those changes, fixing the provider on demand.

P.S. Free users are more affected but the free mode isn't suitable for any serious task because it's very limited in terms of data depth and item availability. I'd rather call it a demo.

hi eugene,

QUOTE:
Interesting. Have you contacted them? How much symbols have you updated before running into this?

yes, i contacted them. no answer!
and i think (not completly sure) i used maybe 600 symbols.

so i am still blocked here ...

QUOTE:
The question is, how reliable is YCharts

yes, true. but also the question about the download limit is key. are there YCharts customer here which can use it with lots of data? this is still an open point for me.

QUOTE:
Nonetheless, with a little help from YCharts users we were able to react to those changes, fixing the provider on demand.

so there is hope.

thanks
coppola

QUOTE:
and i think (not completly sure) i used maybe 600 symbols.

Wow, that's too much. Another website that I know limits anonymous users to just 50 requests per day.

QUOTE:
so there is hope.

Also, be warned that EPS release dates reported by YCharts are unprecise:

YCharts Fundamental Data for Securities and Economic Data provider (see posts #17, 46, and 95)

QUOTE:
are there YCharts customer here which can use it with lots of data?

I know that users were hitting the 25MB quota but some of them asked and YCharts upped the limit a bit (maybe by 5MB).

QUOTE:
Wow, that's too much. Another website that I know limits anonymous users to just 50 requests per day.

too much? so this service would then be useless to me. so some of these providers are obiously in WL without much practical value for professional use...
ok. i will try to find an alternative for the insider data then.

QUOTE:
I know that users were hitting the 25MB quota but some of them asked and YCharts upped the limit a bit (maybe by 5MB)

i intend to use YCharts for thousands of stocks. does this work with this 25/30MB limit?

and (see previous message) i just rechecked the download limits on the YCharts site.
they say 25MB for the lite subscription and 500 MB for pro.

but the pro price is very steep, 3600 usd!

QUOTE:
i intend to use YCharts for thousands of stocks. does this work with this 25/30MB limit?

A data request for a fundamental item for a stock symbol (in Pro/Lite mode) may take from 1KB (typically) to a hundred KB (certain items). On average, it's about several Kilobytes. Multiply that by the number of stocks you're following, then by frequency of requests you're making in a month.

QUOTE:
but the pro price is very steep, 3600 usd!

Aforementioned data issues aside, the price is "adequate" for professional use. Some institutional/professional users are used to pay Bloomberg $2.5K (?) a month.

QUOTE:
A data request for a fundamental item for a stock symbol (in Pro/Lite mode) may take from 1KB (typically) to a hundred KB (certain items). On average, it's several Kilobytes. Multiply that by the number of stocks per frequency of requests a month.

thanks for that info!
several kilobytes on what timeframe (years of stock data)?

and i assume (hope), that when an update is made in the data manager, only the delta (difference between the last stored data and the newest incoming) is downloaded from ycharts. so not the whole data is downloaded from scratch.

right?

QUOTE:
Aforementioned data issues aside, the price is "adequate" for professional use. Some institutional/professional users are used to pay Bloomberg $2.5K (?) a month.

i meant using the WL software professionally.

QUOTE:
several kilobytes on what timeframe (years of stock data)?

The data for most fundamental items is quarterly, explaning the low download size per unit. But some economic (and maybe fundamental, I don't remember) items at YCharts may be daily/weekly with the respective increase in the file size.

QUOTE:
and i assume (hope), that when an update is made in the data manager, only the delta (difference between the last stored data and the newest incoming) is downloaded from ycharts. so not the whole data is downloaded from scratch.

This assumption is incorrect. Unlike static/streaming providers, fundamental providers always reload their data from scratch. Fundamental data update isn't required every day. This is by design and is not subject to change.

QUOTE:
i meant using the WL software professionally.

WL has nothing to do with the 3rd party vendor's quirks and/or policies.

QUOTE:
This assumption is wrong. Unlike static/streaming providers, fundamental providers always reload all data from scratch. This is by design.

by design. why?

QUOTE:
Fundamental data update isn't required every day.

if ycharts reports quarter data at, or shortly after the actual reporting(?), then updating is required very often, i think daily.
infact since the fiscal quarter can be at any month, you have reporting dates during the whole year!

QUOTE:
by design. why?

I could say it's a decision taken in early stages of development but most importantly, rare a fundamental feed has an API and always returns the entire history for that reason. Especially that many of them return the last value on a rolling basis and adjust it later. It's much easier and less error-prone to refresh all than to expose the user to a risk of incorrect data. Data traffic is not a concern.

hi,

but data traffic is obviously, from a WL user point of view, a major concern for using YCharts(and other providers probably too).

so is there no API which could be called to receive only the delta?
then the amount of downloaded data would be drastically lower!
this would ease these download limit problems a lot ...

what do you think, eugene?

I think that "upping" one's posts on this forum (especially within mere hours) has always been considered quite pushy. A technician's time must not be monopolized by a single customer since we're supporting many other customers. I may get back to your question if I have something to reply. Thank you for your understanding.

since i am new to this actual WL software (but was a longtime user of older versions) quite a lot of very important questions (to me) came up.
these points are for me crucial to make a final decision to stay (or not) with WL and invest a lot more time in developing the actual trading systems.

i was just not sure, that my last post was lost or not.

but i understand your point.
peace

YCharts doesn't have an API, but even if it had, refreshing fundamental items was a global design decision. It's too much trouble to rewrite the design pattern which has been working for years. As a rationale, traffic savings would be weak if compared to workload required and potential new bugs. Fundamental data accuracy (i.e. staying free of adjusting problems) has a precedence over potential data traffic savings.

An effort was already made in v2014.05 to leave only the important items, alleviating the quota problem:

QUOTE:
* Change: (breaking) provider consumes considerably less traffic, saving download quota and updating faster after removal of some fundamental (secondary, annual/TTM, proprietary) and economic items (discontinued, secondary, growth/price)

For all practical purposes, YCharts "Lite" users may choose to update less symbols and/or update them less frequently. Professional traders have the option to subscribe to YCharts "Pro".