Using-Medved-Trader-for-streaming-in-Strategy-Monitor

Streaming bars is an option (not required) that has to be built into the Streaming provider, but only if the source provider supports some version of it. For example, IB supports 5-second bars, so the IB Provider has to accumulate the 5-second bars and build the bar for the full interval selected.

For more info and potential difference with "Streaming", see the paragraph on Streaming Bars in the blog:
All About WealthLab Intraday and Realtime Data Providers

I tested the Streaming Data option in SM (1-minute) and it did not return correct OHLC for the bar. I was using Fidelity data as the streaming source in MT. Next I am going to check Schwab data.

I am not sure if this is something that WL team can fix since the data is coming directly from MT, but just wanted to report that so far Streaming Data does not seem to be able to generate correct Streaming Bars.

QUOTE:
I tested the Streaming Data option in SM (1-minute) and it did not return correct OHLC for the bar.

After reading the Streaming Bar discussion cited/linked in Post #1, and the comment in Post #2, I would say the Wealth-Lab team can't fix this because the MT API simply won't support building Streaming Bars on the Medved Trader server side. And if WL tries to build its own streaming bars from MT Streaming Data, it will be in error because some of the ticks won't make it (and be counted) through the WL websocket connection.

If accurate Streaming Bars is important to you, I would use some other data provider than Medved Trader. I use IQFeed, and it supports Streaming Bars accurately.

ST, your understanding of the difference between Streaming and Streaming Bars needs some help still.

Streaming builds bars from the ticks/trades - this is what you see in the Streaming Charts, and they're the same bars when "Streaming" in the S. Monitor.

Streaming bars, available only in the S. Monitor, are OHLC/V bars built on the server end and pushed to WealthLab.

To Shakker's point on Streaming, if the trades are not tick-by-tick, then there will be a difference between the result and historical bars. This is covered ad-nauseum in sections of the F1 Help and here: All About WealthLab Intraday and Realtime Data Providers

That said, when we had Fidelity streaming integrated years ago, it was perfect. So my question is if you're analyzing the right bars. WealthLab timestamps bars at the END of bar, whereas the same bar at Fidelity will have the time from the START of bar.

QUOTE:
when we had Fidelity streaming integrated years ago, it was perfect.

So is WL connecting directly to a Fidelity socket or are ticks being relayed by Medved servers? Perhaps some ticks are being dropped (not counted) between servers.

QUOTE:
So my question is if you're analyzing the right bars. WealthLab timestamps bars at the END of bar, whereas the same bar at Fidelity will have the time from the START of bar.

So you think the discrepancy has to do with time borderline cases? Interesting. In that case, the discrepancy should be very small. Who would care?

QUOTE:
So you think the discrepancy has to do with time borderline cases?

Not borderline. I'm asking if even the same bars are being compared.
If you're not aware of the difference in timetamps, every bar you compare will be different by 1 interval.

update:

In general, I found the MT connection with a linked Schwab account to be completely unreliable, experiencing intermittent problems (and this is not even about using it for SM). For example, there were times when I could update datasets in WL, but at the same time I was not able to place any trades from a strategy window (Order Manager received and showed the trades as "placed" but none were going through to MT) But worse that this, most of the time, even data updates were not working. And all these were happening when the MD Windows app was showing all the connection were up, linked, and running.

QUOTE:
That said, when we had Fidelity streaming integrated years ago, it was perfect. So my question is if you're analyzing the right bars. WealthLab timestamps bars at the END of bar, whereas the same bar at Fidelity will have the time from the START of bar.

I remember that too when SM in WLP could receive perfect 1-minute data from the Fidelity, and I was expecting/hoping that it to be the same through MT. Also the timestamps point that you mentioned does not seem to be at work here. I have just included one example below where you can see the open bar (updated by the SM run) is reported as 170.56 vs. the correct open which is 169.08. In fact, none of OHLC values for that bar are correct.


Lastly, I would like to ask if you happen to know whether the 1-minute Tradier Streaming Data also has the same problem or not?

Get all the WealthLab updates available now (and Medved if one's available).

Tradier Streaming didn't (and won't) have a problem. We think the Tradier account socket - which was failing intermittently - should be solid too with the latest change. We'll be anxious to hear reports about it this week.

You can't expect the Medved Trader extension to have the same responsiveness as the native Fidelity data interface we had in WL6. Regarding the intraday data, it's correct for me. Here is a side by side compare of Medved one-minute data compared to IQFeed.

QUOTE:
Tradier Streaming didn't (and won't) have a problem. We think the Tradier account socket - which was failing intermittently - should be solid too with the latest change. We'll be anxious to hear reports about it this week.

Thanks. I will give it a try.

QUOTE:
You can't expect the Medved Trader extension to have the same responsiveness as the native Fidelity data interface we had in WL6. Regarding the intraday data, it's correct for me.

The issue I was reporting was not about responsiveness, but the accuracy of the Streaming Bars values.

I am not sure how the MT chart shown in your example was updated - through the Strategy Monitor runs or the Data Manager updates? I should reiterate that this problem is only about 1-minute bars received through SM runs/updates. Otherwise, charts/datasets updated by the Data Manager return correct bar values.

Are you using Streaming or Polling in the Strategy Monitor? I would imagine the streaming might be conflated which could lead to differences in closing prices of the generated bars.

I have been only using Streaming Data since Polling is not recommended for 1-minute scale. I will try Polling.

Polling's ok for 1-minute too as long as you're not trading too many symbols - more than 6 is probably too many for 1-minute trading. If you have to poll one by one for 100 symbols and the round trip time is 2 per second, placing orders may be delayed by up to 50 seconds.

Streaming updates everything immediately when the interval closes, so there's minimal delay. But if you don't have a tick-by-tick provider, i.e., a conflated "trades" streaming, then the prices will frequently not match the historical chart.

fwiw, we changed the streaming feed in the Tradier build 28 update from a "conflated trades" feed to a "timesale" feed, which we found was a full-lot, tick=by-tick stream. The result almost always matches the historical chart; the exception is when timesale data is transmitted out of order. (WealthLab's chart will be correct, but Tradier's won't be until after they apply corrections later.)

Read also about Streaming Bars -
All About WealthLab Intraday and Realtime Data Providers

Just providing an update that Polling did not work out with 1-minute scale data (using a Fidelity linked account.) It was very slow even for a set just 4 symbols (shown below), but more importantly OHLC values were still reported incorrect.

Multiple passes that return 0 symbols in the Strategy Monitor is a symptom of a delayed feed. Activate the strategy during market hours and check the timestamp(s) shown in the Log. For a 1-minute chart, the time should match the time of the last minute interval.

Another possibility is that the provider is returning data with start-of-interval timestamps. The test above would reveal that too. i.e., if you activated the strategy at 09:45:15, check if the last bar shown is 09:44.

Our Medved provider is correctly returning end-of-bar timestamps.

My MT also reports end-of-bar timestamps.

So in conclusion, MT 1-minute live data stream in SM is not tick-by-tick and the reported OHLC values do not match historical values regardless of selecting Streaming Data or Polling. This can be easily verified by running this code in SM on 1-minute data:

CODE:
public override void Execute(BarHistory bars, int idx)
      {

         if (bars.IsFirstBarOfDay(idx))
            startBarToday = idx;
                  
         todayOpen = bars.Open[startBarToday];
         
         if (idx == bars.Count - 1)
         {
            string msg = String.Format("{0:f2} - {1:G}", todayOpen, bars.DateTimes[startBarToday]);
            Transaction t = PlaceTrade(bars, TransactionType.Buy, OrderType.Limit, todayOpen, msg);
            t.Quantity = 1;
         }
      }

Polling must match a historical chart from the same provider. We'll have to see that one for ourselves.

Meanwhile, here's a script that I use to test the data created/used by streaming or polling. It will create directories ..\Data\StrategyMonitorTesting\[scale] in your user data folder and write the last bar for each update. Let it run a while and then compare it to historical chart data. This will be a much better test.

CODE:
using WealthLab.Backtest;
using System;
using WealthLab.Core;
using WealthLab.Indicators;
using System.Drawing;
using System.Collections.Generic;
using System.IO;

namespace WealthScript5 
{
    public class LastBarToFile : UserStrategyBase
    {     
        public override void Initialize(BarHistory bars)
        {
         string subfolder = bars.Scale.IsIntraday ? bars.Scale.Interval + "-MIN" : "Daily";
         string path = Path.Combine(WLHost.Instance.DataFolder, "Data\\StrategyMonitorTesting\\" + subfolder);

         if (!Directory.Exists(path))
         {
            Directory.CreateDirectory(path);
         }

         string file = bars.Symbol.Replace('/', '_') + ".txt";
         string filename = Path.Combine(path, file);

         System.IO.StreamWriter csv = new System.IO.StreamWriter(filename, true);

         /* write the BarCount and OHLC/V of the last bar to the file */             
         int bar = bars.Count - 1;

         string line = $"{bars.Count}\t{DateTime.Now:HH:mm:ss.fff}\t{bars.DateTimes[bar]:yyyyMMdd HHmm}\t";
         line += $"{bars.Open[bar]:N4}\t{bars.High[bar]:N4}\t{bars.Low[bar]:N4}\t{bars.Close[bar]:N4}\t{bars.Volume[bar]}";
         
         csv.WriteLine(line);
         csv.Close();
         csv = null;
        }

        public override void Execute(BarHistory bars, int idx)
        {   }
    }
}

Just to clear up something about MT and Fidelity data - though Fidelity has tick data available in their ATP app, their website streaming data is NOT tick-by-tick. it is updated about once per second. That is what Medved Trader has access to since Fidelity does not provide an API for 3rd parties. So as was stated before, building bar data from that will not result in 100% accurate data.

As for Schwab, that has been very reliable, at least in MT and via API. if you have some issues, then
1) if you see the problem in MT, send us a report.
2) if OK in MT, but problem from WL, may want to let both our support and WL support know - I would be glad to help track down the problem.