Maximum symbols to update in real time in SM ?
Author: bobydesi123
Creation Date: 5/19/2010 1:48 PM
profile picture

bobydesi123

#1
Hi,
This is a two part question.
1.
I have strategy in the WLP moniter that runs on nasdaq100 stocks, every 5 minutes.
Sometimes , while executing, it cant update all the 100 symbols. Does any one have idea of the maximum numbers of symbols that the fidelity server lets u update in 1minute and 5min scans (assume that Internet/cpu/io is NOT the limiting factor)?
2.
Is there a way to set trigger in quote manager using strategy?
Thxs.
profile picture

Cone

#2
1. There's no artificial limit. It's troubling, though, that you can't update 100 symbols in under 5 minutes. Fidelity updates 10-symbols at a time, and generally the round trip is under 3 seconds; implying a 30 second update for 100 symbols.

Updates can appear to take longer if one or more of the symbol's hasn't traded. It keeps trying to update until 30 seconds before the end of interval. That can hardly be the case with Nasdaq 100 symbols. Make sure that your DataSet symbols are current and that you don't have any "bad tickers".

2. Sure! That's the whole point of the Quote Manager. See User Guide?
profile picture

bobydesi123

#3
Cone,
Thank u for ur quick response.
QUOTE:
Make sure that your DataSet symbols are current and that you don't have any "bad tickers".

All my symbols are upto date and i have 5min dataset for NDX100 that gets updated every night. I am loading 500 bars for the strategy on startup. Using the same strategy while streaming, i manually clicked through the entire 5 minute NDX100 dataset (one symbol at a time), it worked fined. I am going to try dataset with a fewer stocks and give it a try tomorrow (need streaming)
2. thxs, i will look at it.
profile picture

bobydesi123

#4
Cone,
A wrote a simple strategy that(plots EMA 10,20 in the pricepane) loops through 5min NDX100 dataset by using "SetContext()" and "RestoreContext()" . I am able to loop through a few symbols and then it hangs! Works fine while not streaming.
profile picture

Eugene

#5
It probably doesn't hang but is requesting data. Have you disabled on demand data update?
profile picture

bobydesi123

#6
Eugene,
QUOTE:
It probably doesn't hang but is requesting data. Have you disabled on demand data update?

Yes data on demand is disabled. However, i should mention that i had four workspaces open that is streaming 1/5 min data for spx,ixic and ndx indices and aapl. Anyways, cant do much trouble shooting without streaming.
Thxs as always.
btw,do you, cone and glitch, all work for same company (Just curious because all three are the most helpful/knowledgable)? No need to answer if u deem the question too personal.
profile picture

Cone

#7
I guess you missed the memo :) Eugene and I work for MS123. Glitch is Vice President of Wealth-Lab Product Development at Fidelity.

It's hard to say without all the details, but the hang could be a memory problem or a slow down due to series synchronization of many intraday symbols/bars.
profile picture

bobydesi123

#8
Cone,
Thxs for ur info.
Sorry i could not respond to this post quickly. I had to wait for the streaming to start to test it again and got too busy with trading. Anyways, today for a brief moment i tried str. Monitor (SM) again. The first few 5min runs of NDX100 went through in about 20-30 seconds (as u had mentioned). Then i started getting bunch of "server returned error 500" errors and i stopped it. I guess yesterday, there might have been some server/internet isssues.
Other question is why the following simple script in 5min scale hangs for NDX100(by this i mean it cant finish executing the strategy in 5min, before the next bar arrives). If i change "DataSetSymbols.Count" to 5 or 6 it works more than that i get "busy circle" and hangs. I know that WLP/fidelity has problem when u stream more than 6-7 charts. Does the following code open up as many streaming as there are symbols in the dataset (DataSetSymbols.Count)

CODE:
Please log in to see this code.
profile picture

Eugene

#9
QUOTE:
Then i started getting bunch of "server returned error 500" errors and i stopped it. I guess yesterday, there might have been some server/internet isssues.

Re: "Error 500" - this is purely a server issue. (To those who don't know, here's list of server errors from Wikipedia.) If you had an internet issue, it's like nothing was returned at all. ;)
profile picture

bobydesi123

#10
Eugene,
Thxs for ur reply.
Re: "Error 500" - this is purely a server issue
I am aware of this.
Why does the code above will not work for more than 5-6 symbols? Howerever, i can insert an alert (like buy @ mkt) and run this strategy in strategy monitor (SM) and it will executes over entire NDX100 in 30-40 minutes (that is when fidelity server is in good mood :) )
How does SM differs when monitoring NDX100 vs. Loop through NDX100 using SetContext(), in RT streaming?
profile picture

Cone

#11
QUOTE:
Why does the code above will not work for more than 5-6 symbols?
Are you running it on the DataSet or for just one symbol in the DataSet?

You should almost never run a script with a DataSetSymbols loop on the entire DataSet - it just doesn't make sense to do that. You'll end up getting DataSetSymbols * DataSetSymbols requests.
profile picture

bobydesi123

#12
Cone,
QUOTE:
Are you running it on the DataSet or for just one symbol in the DataSet?

I am running this strategy by clicking a symbol in dataset "NDX100" containing NDX100 components.
Thxs.
profile picture

Eugene

#13
QUOTE:
I am running this strategy by clicking a symbol in dataset "NDX100" containing NDX100 components.

Is this also true when you run it in SM?

Hopefully WLP isn't running in debug mode with some extra libraries (like before)? Running an innocuous strategy in SM for 30-40 minutes is certainly kind of strange. Is there anything else specific that could help with this?
profile picture

bobydesi123

#14
Eugene Cone,
I am really sorry, there was a major typo in my previous post.
Eugene,
QUOTE:
Thxs for ur reply.
Re: "Error 500" - this is purely a server issue
I am aware of this.
Why does the code above will not work for more than 5-6 symbols? Howerever, i can insert an alert (like buy @ mkt) and run this strategy in strategy monitor (SM) and it will executes over entire NDX100 in 30-40 minutes (that is when fidelity server is in good mood :) )
How does SM differs when monitoring NDX100 vs. Loop through NDX100 using SetContext(), in RT streaming?

The text in bold should read 30-40 Seconds .
Thxs.

profile picture

bobydesi123

#15
Eugene,
Sorry to re-open this thread after a long time, i have been away. Beware! its a long one :smiley:
I have few questions, assume that i am refering to .NDX index streaming in RT,strategy takes 50ms to run.
1. when i have a strategy running in RT 1min.
-On the 1st run the WLP backfills the data from the static dataset and loads the bars into "virtual memory".
- on subsequent run as the new data arrives, the WLP "appends" the new data to it.Static Dataset is not used
- is the above assumptions correct?
2. If i open the same strategy with same symbol(.ndx),same scale(1min) etc in a different workspace(WS).
- Does the WLP refer to the same "virtual memory"(VM1) or does it backfill from static dataset and start loading data into another "virtual memory" (VM2)?
3. The reason i am asking is, if i run the same strategy in differnt WS, i see a "circle" which indicates that the strategy is waiting for the end of the execution, this takes about 4-5 seconds.
-if i re-run strategy manually between the time interval(1min) by hittin "F5", i see the same wait "circle". (This happens regardless of using same WS or a new WS)
-why does this happen? If the data is already in the memory it should not take more than 50+ ms to execute.( Does it have something to do with ghostbars?)
4. How does 1,2,3 different from when i use same strategy in the strategy monitor (SM)?
- does the SM create new bars for each of the symbols in memory after the initial backfill (assume time scale <= 5min), even if the WS have the same symbols/scale being reffered to? i.e, even if the WS and SM are executing same strategy/symbols/scale they refere to the different memory location?
- The reason i am asking this question is:
- the SM is able to run the strategy over the entire 100 symbols of .NDX(5min scale)in about 20-30 seconds but if i try to programmaticaly loop through the symbols, the strategy takes for ever, i am able to loop through only for 4-5 symbols vs. SM going through 100 symbols in 20-30 seconds!
Thank u in advance.
profile picture

Cone

#16
1. Yes, but Wealth-Lab doesn't determine if your O.S. uses physical or virtual memory - but hopefully never the latter!

2. Almost for sure (but I don't know) it re-loads the data. But why would you do it anyway?

3. Not sure what the "circle" is, but it's likely that you have on-demand requests on, so you have to wait for the data loaded from disk and then a round-trip data request.

QUOTE:
programmaticaly loop through the symbols,
That's probably because you're trying to synch all the intraday symbols in the primary context. The S. Monitor doesn't do that. It runs each symbol separately in its own context. How it should be done do it depends on how you want to use and apply the results.
profile picture

bobydesi123

#17
cone,
thxs for ur response. Data on demand is off.
QUOTE:
Not sure what the "circle" is, but it's likely that you have on-demand requests on, so you have to wait for the data loaded from disk and then a round-trip data request.

This a large spinning circle (gray-blue), that idicates that strategy is busy. sorry. i dont know how else to describe it( circle in lieu of the usual hour glass?)
If i have WS with with 2 chartwindow running the same strategy on .ndx /1min & 5min/streaming. I have separate static data for 1min & 5 min .ndx. As the new data arrives the charts get updated in less than second.
- If i hit "F5" in between the data arrival period, i.e. 1min, the strategy takes 5-10 second to finish vs. < 1sec, when executed by the WLP heartbeat. Shouldnt the WLP refer to data already memory? it seems like on every "F5" the WLP waits. Seems like there is a difference between manual execution and the heartbeat execution.
- As far as looping through dataset programmaticaly is concerned.
-I strat the execution of the startegy by clicking on 1min .NDX
- then i have code that is described in my post on 5/20/2010 7:13 PM.
- does SetContext( DataSetSymbols[n], false ) refer to the dataset that was clicked, i.e. 1min .ndx or does it default to daily scale? Shouldn't data be synched already because all data in the 1min dataset is upto date?



profile picture

Cone

#18
When you hit Go or F5, it reloads the data completely from disk file. Yes, there's a difference.

QUOTE:
Shouldn't data be synched already because all data in the 1min dataset is upto date?
No, especially not for intraday data. Synchronization means that there's exactly one bar in the secondary symbol for every symbol in the primary context, and no others. See WealthScript Guide: DataSeries > Accessing Secondary Symbols > Secondary Series Synchronization

QUOTE:
SetContext( DataSetSymbols[n], false )
false means that you're not requesting synch, so there would be no delay due to synch. As for scale, it should default to the scale of the DataSet. If you're loading "All Data" for 1 Minute bars for 1000 symbols, this operation could take some time.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).