CandlePattern thread safety issue in Streaming mode
Author: gregChen
Creation Date: 5/19/2020 4:09 PM
profile picture

gregChen

#1
Add the following code and turn stream on

CODE:
Please log in to see this code.



Will see exception if run the strategy.
profile picture

Eugene

#2
1. Why does this thread's title mention "thread safety"?
2. What exception message are you seeing?
3. How do we reproduce it (symbols, bar scale, data loading range)? I'm not seeing an exception on enabling Yahoo! streaming in a Daily chart (500 bars) of various symbols.
profile picture

gregChen

#3
CODE:
Please log in to see this code.
profile picture

gregChen

#4
TQQQ, 1min

wait for data coming and then run.
profile picture

Eugene

#5
1. So if I gather correctly, you click "Run strategy" in an already Streaming strategy and get this exception? What's the point?

2. If you don't do #1, the script executes correctly. Right?

P.S. Tried #1, can't repro.
profile picture

superticker

#6
The error message is suggesting that the ".Series" cast below fails internally.
CODE:
Please log in to see this code.

I see no reason why the above line would throw that error. But rather than argue with the run-time system, why not just avoid the DataSeries ".Series" cast altogether as shown in the line below.
CODE:
Please log in to see this code.

You might even try (below), but that solution is also a DataSeries cast--but without the indicator cache involved.
CODE:
Please log in to see this code.

I don't know if that will fix the problem, but you should at least get a "different" error message, which might shed more light on what's really going on.

I "think" what's really going on has to do with the indicator cache "not settling" (i.e. thread safety issue with the indicator cache?) before casting to ".Series" occurs inside the ATR code. This cache settling problem never occurs with static data, but can occasionally occur with on-demand data. I haven't seen it with streaming data. With on-demand data, if you just repaint (not reload) the chart, there will be no cache race condition on the second try, so it will always clear up.

Because this error maybe related to a race condition (with the indicator cache), you're not going to be able to reproduce it on another machine. I wouldn't even try.
profile picture

Eugene

#7
Internally, the CandlePattern class uses a 10-period ATR. There may be a race condition since OP's code uses the same series.

But it boils down to this old joke: Patient says, "Doctor, it hurts when I do this Doctor says, "Then don't do that!" To rephrase, don't click on "Run strategy" in an already Streaming strategy since it makes no sense to do.
profile picture

gregChen

#8
Reloaded symbol data file from data manager seems resolved the issue. Thanks
profile picture

Eugene

#9
You never know. Good to hear it's that simple.
profile picture

superticker

#10
QUOTE:
Internally, the CandlePattern class uses a 10-period ATR. There may be a race condition since OP's code uses the same series.
Thanks for that clarification; interesting. I do appreciate using the .Series version for CandlePattern so the ATR is cached for speed.

QUOTE:
Reloaded symbol data file from data manager seems resolved the issue.
Right. So if you force the data to be static rather than on-demand, you avoid the caching race condition. Sometimes if you miss a couple consecutive days of updating the Fidelity static data cache, static data updating gets messed up. (You could look into fixing that.)

Currently, a DataSeries (and its cache) are inherited from the "List" .NET datatype, which is not thread safe. You could switch from List to one of the thread safe types .NET has, but that would tie the hands of the Windows thread scheduler with many scheduling semaphores. I don't like that overhead unless it's really a problem. Personally, it hasn't been a problem for me unless the Fidelity on-demand server is really slow that day.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).