IQFeed-daily-volatility-indices-were-repeating-yesterday-s-bar

The next time it would help that it happened to me, then I could investigate it. Post the piece of your script that synchronizes and plots those BarHistories.

QUOTE:
I have a daily strategy showing VIX.XO, VXN.XO, and VXD.XO (IQFeed symbols) on the chart windows.

To be clear, you dragged a strategy into a chart for which symbol?

And then around 8:30, what action happened to change the chart? Did you refresh? Change the symbol? Was Stream enabled?

QUOTE:
Post the piece of your script that synchronizes and plots those BarHistories.

This is the piece of code snipped from my strategy.
The symbol is QQQ and it runs in Single Symbol mode with Daily scale and All Data range.

CODE:
using WealthLab.Backtest;
using System;
using WealthLab.Core;
using WealthLab.Data;
using WealthLab.Indicators;
using System.Collections.Generic;

namespace WealthScript1 
{
    public class MyStrategy : UserStrategyBase
    {
      BarHistory VX_D;

      public override void Initialize(BarHistory bars)
        {
         ChartDisplaySettings cds = new ChartDisplaySettings();
         cds.ShowVolumePane = false;
         cds.PaneSeparatorWidth = 1;
         cds.ColorPaneSeparator = WLColor.Black;
         cds.ShowEventIcons = false;
         cds.DisplayTradeArrows = false;
         SetChartDrawingOptions(cds);

         SetPaneDrawingOptions("Price", 100, 100);

         VX_D = GetHistoryUnsynched("VXN.XO", HistoryScale.Daily);
         VX_D = BarHistorySynchronizer.Synchronize(VX_D, bars);
         SetPaneDrawingOptions("Volatility", 100, 0);
         PlotBarHistory(VX_D, "Volatility");
      }

        public override void Execute(BarHistory bars, int idx)
        {
        }
    }
}

QUOTE:
To be clear, you dragged a strategy into a chart for which symbol?

Yes, I dragged the strategy into a chart.

Actually, I have three chart windows and the Strategy Monitor in the workspace.

Those three chart windows are all running the same strategy.

The three chart windows are with SPY, QQQ, and DIA symbols, and display the second symbols, VIX.XO, VXN.XO, and VXD.XO, respectively on the charts. The chart windows are not Stream enabled.

The Strategy Monitor runs the same strategy with the dataset consisting of the same symbols (SPY, QQQ, DIA, VIX.XO, VXN.XO, VXD.XO) with the polling mode (as suggested for daily strategy).

QUOTE:
And then around 8:30, what action happened to change the chart? Did you refresh? Change the symbol? Was Stream enabled?

Refresh, changing symbol, deleting the data, restarting WL8, enabling/disabling Stream, all tried but none of them fixed the problem.

The last thing was that I ran the sample code above to duplicate the problem so that I could report the problem in the discussion forum.

When I ran it at around 8:30 pm EST, the problem had gone away.
So immediately I ran the strategy window that I had opened, and then the problem had gone away there as well.

This is so mysterious.

It was two volatility indexes (VXN.XO and VXD.XO) that were duplicating yesterday's bar.

All the symbols (SPY, QQQ, DIA) and one volatility index (VIX.XO) were all showing today's bar.

It may be a coincidence but the problem happened today for the first time since I changed the Strategy Monitor to use Polling instead of Streaming Data, based on yesterday's discussion on the other thread that Polling should be used for daily strategy.

It must be due to returning those bars from cache, but why it didn't update is going to be a mystery until we can duplicate it. Our dev subscription doesn't have U.S. indices, so we'll have to keep an eye on it with any other external symbol. In the end, it's going to come down to a procedural scenario to find the cause.

I have never had anything like this issue since I started this strategy more than 10 years ago on WLP 6.4. Since my strategy heavily relies on the volatility indexes, currently I cannot use WL8 for real trading until this issue is understood.

Is there any way to flush or purge the cache on the setting or the code?

Sorry for the growing pains. I know you can't use WealthLab 8 if all the issues that affect you aren't resolved. That reminder isn't required for every post. For sure won't be able to use WealthLab 6 after next June.

I'm just hypothesizing about the cache, but it seems likely. Restarting would certainly clear a cached bar. Programmatically could be a problem.

There are a great many things that WL8 does differently than WL6 and there are reasons for them. One is not adding the Daily bar to a BarHistory before the market closes. For that reason, it's important for a provider to know the market hours, which always defaults "US Stocks" if not assigned. IQFeed pretty much always uses "US Market", so really, it's difficult to know what happened to cause it.

QUOTE:
I'm just hypothesizing about the cache, but it seems likely. Restarting would certainly clear a cached bar.

Then it may not be a cache issue as the issue remained after restarting.

Do you have any idea of what possibly would cause the difference in the way how the bar data is used differently between the chart window and the strategy window?

The volatility indexes repeating yesterday's bar on the strategy window looked fine on the chart window.
Both the strategy window and the chart window were streaming disabled.

The only difference I can think of would be the strategy code using BarHistorySynchronizer to synchronize the unsynched volatility index bars to the symbol bars.

I wasn't able to duplicate the problem today but I found something.

My conclusion is that I should not have any strategy chart running a daily strategy that ends up updating the volatility index data at 4:00 pm (the market close).

Otherwise, the volatility index which keeps moving until 4:15 pm won't be reflected in the data due to the daily bar once completed at 4:00 pm.

So the strategies running at 4:30+ pm in the Strategy Monitor will end up with getting the volatility index from 4:00 pm instead of the last bar at 4:15 pm.

Updating the dataset manually from Data Manager won't be able to refresh the data but I have to manually delete them.

I tried disabling the "Filter out Pre/Post intraday market data" option in the Data Preferences and the "Regular Session Only" option in the IQFeed Data Provider setting but neither works for daily data.

The original problem remains mysterious but the conflict explained above in the daily bar like the volatility index which closes at 4:15 pm instead of 4:00 pm may ring something.

Remember what I said about "Market Hours".
I wasn't aware that those indices had a different closing time, but you can define those hours in Tools > Market & Symbols.

1. First Create a Market and name it. Let's call it "VX Hours". The market will be M-F from 09:30 to 16:15 EST, fore example.
2. Now on the Symbols tab, create a record for each symbol required and assign it to the "VX Hours". Market.

Now the Daily bar for those symbols will not "close" until after the hours defined by VX Hours.

Yes, you said about "Market Hours" but I didn't understand it very well.
Thank you for explaining it with the example, now I think I understand it.

Now I think I can make the VX symbols to be closed at 16:15 EST (VX Hours).

However, these are secondary symbols to be used in the strategy that runs with the primary symbols (e.g. SPY, QQQ) that close at 16:00 EST (US Stocks).

In this case, the strategy chart running with the primary symbols will get the bar update at 16:00 EST not 16:15 EST so I have to manually refresh the chart to get the VX symbols to be updated after 16:00 EST?

The daily bar for symbols assigned to the "VX Hours" Market will not be available until after 16:15.

If you want data for those indices that "closes" at 16:00, you should not use the VX Hours Market. Instead you'll need to use 15 minute bars and compress/sync them (BarHistoryCompressor/BarHistorySynchronizer) for a bar history that covers the US Market hours.

To get the update for the 15-minute bar for a secondary symbol at the end of its interval, make sure to use GetPairHistory().

Thank you for your suggestion.

Although the 15-minute bar idea sounds promising, it appears slightly prone to go wrong.

I guess I would rather stop streaming for the strategy chart and stick to the Strategy Monitor solely for updating the data.

By the way, how about using the "VX Hours" Market idea in the Strategy Monitor?

It doesn't matter as long as polling is used and the strategy is scheduled to run after 16:15?

Or any advantage using the "VX Hours" Market idea still in the Strategy Monitor?

If you don't need to run precisely at 4pm and can wait to 4:15 for an EOD strategy, then for sure assign VX Hours to those symbols.

RE: it appears slightly prone to go wrong.
Don't know what you're talking about. It would work perfectly, creating bars from the 15 minute data from 09:30 to 16:00, if that's what you need. Streaming isn't required, and you can do that in the S. Monitor too. (For this option you do not assign those symbols to VX hours... although I'm not sure if it matters since you'll be synchronizing to a US Stocks market. You'd have to test that.)

QUOTE:
By the way, how about using the "VX Hours" Market idea in the Strategy Monitor?

If it's an EOD strategy, it doesn't matter because you'll schedule the run after 4:15, right? But if required otherwise ...

QUOTE:
If you don't need to run precisely at 4pm and can wait to 4:15 for an EOD strategy, then for sure assign VX Hours to those symbols.

By those symbols you mean the primary symbols (SPY, QQQ), so that EOD strategy will be executed with the updated secondary symbols (VIX.XO, VXN.XO) at 16:15 on the strategy chart, correct?

QUOTE:
Don't know what you're talking about.

I just meant my preference that simple logic would be less troublesome.

QUOTE:
If it's an EOD strategy, it doesn't matter because you'll schedule the run after 4:15, right?

That's right. It's an EOD strategy to schedule the run after 16:15.

QUOTE:
By those symbols you mean the primary symbols (SPY, QQQ), so that EOD strategy will be executed with the updated secondary symbols (VIX.XO, VXN.XO) at 16:15 on the strategy chart, correct?

Not correct. Assign the secondary symbols (VIX.XO, VXN.XO) to the VX Hours market because that's their market.

This is a general statement:
Any time you requeste a daily bar before the end of the day's session, you'll get the daily bar for the previous day. You'll get "today's" daily bar when that market ends.

The same applies to ANY interval.
If you request a 15-minute bar, say at 10:40, you'll get the 10:30 bar. You won't get the 10:45 bar until after 10:45.

You don't need to change anything except assign the VX Hours market to the correct symbols and schedule when you want the S. Monitor to run.

QUOTE:
You don't need to change anything except assign the VX Hours market to the correct symbols and schedule when you want the S. Monitor to run.

Sorry for the confusion but my previous comment was about the strategy chart where the chart runs with the primary symbols which close at 16:00 and the secondary symbols which close at 16:15.

I was wondering if I enforced the secondary symbol's close time to the primary symbol's close time so that the strategy chart would update the last daily bar (EOD bar) at 16:15 instead of 16:00.

There won't be a streaming bar update for the primary symbols after 16:00, if that is the case the strategy chart may not update the last daily bar automatically, in which case I would manually kick off the "Reload Chart Data from Provider" from the chart menu.

Imho, these are mistakes:
1. using Streaming for a EOD strategy that doesn't need to be run immediately at the market close, and,
2. changing the market for symbols that don't require it. (It will affect something else you do later and then you'll make us investigate something that looks like a bug that doesn't exist.)

You have all the information for how it works. Do it however you like.

I think I'm better off not doing those hacky things then, otherwise, I would likely run into some issues, but I don't want to bother you guys.

I would rather do the "Reload Chart Data from Provider" on the strategy chart after 16:15, it is just to verify the trade visually anyway.

Hopefully, the original repeating yesterday's bar issue is going away with your VX Hours suggestion.

At this point, my migration task from WL6 to WL8 is pretty much completed.
I greatly appreciate all your assistance on all the issues.