Fidelity static data stops getting updated automatically in WLP 6.9.17
Author: Ray60
Creation Date: 4/28/2017 5:35 PM
profile picture

Ray60

#1
I am sending this message to Wealth Lab and Fidelity.

I have been using Wealth Lab Version 6.9.12 due to the Orders hanging up as “Submitted” problem that has existed since 2010. Yesterday, April 28,2017, Fidelity forced an upgrade to 6.9.17. I immediately experienced a problem four times in one day with the new version.

After running for about one hour the static WL data files stop getting updated automatically. The streaming data continues to come in and the chart appears to be ok but if you reload the chart the last data displayed is what is in the WL files and the additional data is added from the streaming data.

The attached WLproblem1.png shows the same symbol with 5-minute candles that ran from the open today. The numbers below the candles are the intraday bar numbers. The plots are identical up to bar 11(10:30am). At that point the WL file stopped updating. When bar 13(10:40am) was reached I reloaded the Left window. As you see the left window bar 13 is actually bar 15(10:50am) in the right window. The static WL file was never updated.

After exiting and restarting the program and the same thing happened about one hour later. I added the time stamp in red below the bars. The attached WLproblem2.png shows that the static WL file stopped getting updated on bar 38(12:45pm). After reloading the Left window the next bar to appear is 13:05 or the same as bar 42 in the right window.

I called Fidelity about this yesterday and was told to (1) log off and log on, and (2) uninstall and reinstall the program. I uninstalled the program and the chart is from today after reinstallation. Logging off does not work because the program will not let you log back on. You have to exit and restart.

It appears that the server that updates the static WL file goes into a delayed 15-minute mode after one hour. I hope Fidelity and Wealth Lab will talk to each other about this problem and not just say it is the other party’s problem.
profile picture

Cone

#2
I'm very sorry to hear about the Stuck Submitted. It's extremely frustrating not being able to duplicate a problem like this while customers experience it rather frequently. :(

Re: Data
I'll report this to right guys Fidelity. I think you're right about 15-minute delay. Something must be at work on the back end that's dropping the real-time entitlement.

Re: "At that point the WL file stopped updating"
While Streaming, the static WL data file is not expected to update. But sure, it should update upon reloading the chart (Go, F5, etc.) as it requests the most recent historical bars up to an including the partially-formed bar in progress.

profile picture

Ray60

#3
Cone,

I don't know if this helps but I found the following in the wlp.txt file.

2017-04-28 13:39:26,456 [Managed End Point thread 1] ERROR WealthLab.DataProviders.FidelityACTIVStreamingProvider [(null)] - OnBreak() - Connection to Activ server is broken. Reconnecting using asynchronous login.

Thanks

profile picture

Cone

#4
Without context, it's hard to make anything out of those debug logs. It seems that this issue is somewhat easily duplicated and once we figure out the cause, I'll let you know.
profile picture

MohammadRashid

#5
I have the same 15 minutes delay problem on Thursday but I didn't experience it on Friday. It caused automated orders to fill twice with 15 minutes delay. This problem already costed me thousands of dollars. It seems Fidelity does not do enough testing on a real environment.

For Example, this very simple 5-minute interval strategy on VXX (to overcome the 'submit' issue) fires on 00:14 and 00:45 instead of 00:30 and 00:00
here is the text:
CODE:
Please log in to see this code.

profile picture

kcbars

#6
Just wanted to add that I, too, have been experiencing the stuck submitted problem, probably for the last 3 years or so, and have reported the issue multiple times to Fidelity. I also had the same sort of issue on Friday with the data not updating correctly. The problems make it virtually impossible to use the software intraday on a consistent and reliable basis. The only thing I can think of is that Fidelity's pool of Wealth-Lab users is too small for them to worry about. I wish they would either fix the issues or drop the product, and I think at this point it's safe to say they are not interested in the former.
profile picture

KGo

#7
QUOTE:
While Streaming, the static WL data file is not expected to update.

I know this has been the design. However I can positively say the static data did update in prior versions as:
1. I have watched in prior versions in Windows File Explorer the date ,time and file size update every 5 min. It was regular in that I could calculate file size at a future time. See this data below for today.
2. A system that only ran streaming strategies would have up to date data noted by the Data Manager for streaming symbols during the following morning download. At most one bar would be updated.

Today, the numbers for SPY suggest data updates are occurring:

Time File Size(bytes) Size change
1035 12,229,689
1040 12,229,737 48
1045 12,229,785 48
1050 12,229,833 48
1055 12,229,881 48
1100 12,229,929 48

Is file update with data added the reason the file size is changing when streaming? Another possibility?

Lastly, I did not start tracking this at session open so the data could still be changing with 15 min delay. Know that the 15 min delay for static intraday strategies is a serious restriction for me which will require some workaround if not quickly fixed.
profile picture

Cone

#8
QUOTE:
Is file update with data added the reason the file size is changing when streaming?
It's an interesting observation to throw into the list of clues. I'm just extremely surprised that anything changed in the data area, because nothing was targeted here.

QUOTE:
is a serious restriction
Absolutely. Fidelity is looking into it as "speak".
profile picture

KGo

#9
Here is the AM tracking for SPY file size changes while streaming.
Time File Size(bytes) Change
4pm 12,232,377
935 12,232,425 48
940 12,232,473 48
945 12,232,521 48
950 12,232,569 48
955 12,232,617 48
1000 12,232,665 48
1005 12,232,713 48
1010 12,232,713 0
1015 12,232,713 0
1020 12,232,713 0
1025 12,232,761 48
1030 12,232,809 48
1035 12,232,857 48
1040 12,232,905 48

In prior WL versions the update never did a 15 min pause. It wrote the file at every bar except the 4pm bar.

When this stops showing the pause the problem will be fixed. Start in in 5 min streaming before market open while SPY is streaming in another 5 min strategy.
CODE:
Please log in to see this code.

profile picture

KGo

#10
Last point. This AM the Data Manager updated 4 bars of SPY data. Prior WLP version this was typically 1 bar. It's just further confirmation that after the 1010-1020 gap demonstrated above the system continues to wrote SPY data without the last 3 bars.
profile picture

Ray60

#11
Cone,

I hope this will help. I called Fidelity today and asked them to ask the systems people for a target date that the data delay problem would be solved. After waiting I was told that there was no target date. I have used Wealth Lab since 2008. I have a lot of coding time invested and do not want to move to another platform.

I wondered if another data provider would solve the problem. I deleted the Fidelity Static Files and used Google Finance for data. The attached WLproblem3.png shows the problem is still there despite the new provider. Every thing worked fine until an hour after login. When the windows are reloaded the delay appears. The left window is the static (streaming off) Google data with the last candle at 2:45 pm. The right window is streaming and shows the last complete candle at 2:30 pm and a partial candle being formed. This screen capture was made at 1:47pm CT. I notice that the Left Title is the Google “NUGT 5 Minute” while the Right Title is from Fidelity.

Back on April 27 I immediately assumed that it was a problem with Fidelity servers but I am hoping Wealth Lab will look at their coding of 6.9.17 vs 6.9.12 and see if there is something that is triggered after an hour and causing the delay.

Thanks
profile picture

Eugene

#12
Roy,

Sorry to rain on your parade but the Google provider is not to blame. You're still using Fidelity streaming data because it's activated in Wealth-Lab's Preferences (Streaming Data section). To perform a real test, switch to Google in that dialog.
profile picture

Ray60

#13
Eugene,

Thanks. I will see what happens Monday.
profile picture

Ray60

#14
Cone,

The problem with Fidelity data getting delayed after one hour remains. I have called Fidelity twice since I opened this thread. Today I asked them if it was even on their list to look at. The rep found a note that it had been escalated and the action taken was to notify Wealth Lab of the problem. This seems to be the normal response of two organizations to say simply that it is the other party’s responsibility.

I believe that the problem is with Fidelity but they don’t seem to want to spend the time to look into it. If this is indeed the case then this will never get fixed.

Please get back with them and try to get their attention.

Thanks

profile picture

Cone

#15
Rest easy, the problem is being worked and there's already a fix identified for it even if the Fidelity support team isn't aware of it.

That said, I cannot tell you (because I don't know) when a fix will go out to production. I'm also helping on the Stuck Submitted bug and making some progress. I've got my fingers crossed that this can finally be nailed down.
profile picture

MohammadRashid

#16
The above issue causes my 5-minute interval’s strategy to execute orders twice. I.e. If I keep WL running for more than one hour and if it happens the strategy executes an order then the order is duplicated 5 minutes later. So I have to set in front of the screen all the time and reset the app every 45 minutes or so. Imagine the pain!

Any update on this issue is appreciated.

Thanks.
profile picture

Cone

#17
Don´t quote me on it, but we´re looking at some time next week for an update.
profile picture

MohammadRashid

#18
The week passed and the month passed without an update! Is there ETA for this issue to get fixed or at least a way to go back to the older version?
profile picture

Cone

#19
I've heard the update will be ready 6 July in the evening. Just a few more sessions :/
profile picture

Ray60

#20
Cone,

Still no WLP 6.9.18

Do you have a revised date?
profile picture

Cone

#21
Not yet.
profile picture

Cone

#22
6.9.18 is ready! However, things on the fidelity back end have changed a bit, so upgrading from 6.9.17 will not work. You must:

1. Uninstall WLP 6.9.17 from the Windows Control Panel, and,
2. Download and install Wealth-Lab Pro 6.9.18 from https://www.fidelity.com/wlp
profile picture

Ray60

#23
Cone,

Got it. Ran all day without problems.

Thanks
profile picture

Ray60

#24
Cone,

The problem is back this morning. After one hour the data gets delayed 15 minutes.

The good news is I can log out and log back in without exiting the program and the static data is updated.

I really hope Fidelity / WealthLab will make a patch available quickly. The workaround of not touching anything to keep the chart updated by the streaming data is not realistic.

Thanks
profile picture

Cone

#25
QUOTE:
After one hour the data gets delayed 15 minutes.
I need the emoticon with the big eyes and a surprised expression. Please make sure that you're running 6.9.18 by checking the dialog at Help > About Fidelity Wealth-Lab Pro.
profile picture

Ray60

#26
Cone,

I am running 6.9.18(see attached).

Every thing has worked for the past three hours.

"superticker" #105 comment on the Fidelity Server problem in the "Fidelity WLP Orders don't get placed properly (Stuck Submitted)" thread must have been the reason.

Thanks
profile picture

Cone

#27
QUOTE:
Every thing has worked for the past three hours.
I'll take that to mean that you're no longer seeing delayed charts. That's good. We're pretty confident that that issue is behind us.

However, another customer has reported a delay in Trade Ticket quotes and Paper Trading fills. Does anyone else see that?
profile picture

kcbars

#28
I have not noticed a delay in paper trading fills, but I have noticed that the basis for paper trades is incorrect. It seems to be delayed by approximately 15 minutes. The entries on the charts are correct, but account balances and positions doesn't show the correct basis. Thanks.
profile picture

superticker

#29
There still remains a bar-update problem in WLP 6.9.19.0 when you open a new 1-minute bar chart early in the trading day. The bars from ~11 minutes earlier never appear. See attachment. What's weird is that I only notice it for 1-minute charts. All other charts seem to update fine when they are opened.
profile picture

Cone

#30
Thanks for the info. I'll take a look at it on Monday.
profile picture

Cone

#31
I forgot that this delay always occurs in the QA environment, so I can't test this myself.

Is it consistently a problem for 1-minute charts for you? For now, this is this the only report that I'm aware of.

Does anyone else see an issue with backfill data in a 1-minute Streaming chart?
profile picture

KGo

#32
Using WLP 6.9.19.0 I have periodic problems but way less than 6.9.18 so gave it a try about 15 min into the session with a new streaming strategy. It opened in a 5 min chart looking good. Then it appeared to update bars properly on a 1 min chart. I closed it down and later saw the 5 min streaming strategies data had became corrupted and failed. For 3 symbols all static backfill data was gone the only data left was for today. I reloaded data from backup and ran fine the rest of the day.

The workspace runs flawlessly if uninterrupted. Just reporting since you asked. No action needed.
profile picture

superticker

#33
QUOTE:
Is it consistently a problem for 1-minute charts for you?
I thought it was, but ...

... I just reloaded the 1-minute bar chart for VRTU for Friday (11/10/2017, two trading days ago), and its chart is "almost identical" (except for a few time corrections) to my original screenshot above for Friday. In other words, there were hardly any ticks for VRTU between 9:32am EDT and 9:43am as far as the static data reflects. The behavior above is normal and expected.

I'll continue to monitor the issue. But when monitoring, one needs to be sure to test this problem with a high volume stock that has ticks for every minute; otherwise, you could get fooled. :(