MIH8
- ago
Perhaps you can find a better name for the topic (at least "live data" seems to be better than "streaming data"). Today I found the following issue.





In the first image you can see the updated historical data for IB from yesterday's close. The next image of IB shows a completely different closing price from yesterday directly from the IB chart window.

Why do you think the two prices differ?

I can easily understand why prices from different sources differ, but here we are talking about the same source. The problem is that these prices are used in calculations and lead to completely different actions The price difference is enormous.

Is there anyone who has also made these observations or can give me a plausible reason why prices from the same provider can differ so much? Is there anyone who has also made these observations?
0
550
Solved
14 Replies

Reply

Bookmark

Sort
Cone8
 ( 21.52% )
- ago
#1
Of course we expect minor differences in streaming, and, we recommend that you do not use IB Streaming in the Strategy Monitor in favor of Streaming Bars.

However, this difference is so great it makes me wonder if,
1. the stream stopped, or,
2. the market or the market's closing time is not properly set up in the item's activation settings.
0
Cone8
 ( 21.52% )
- ago
#2
Sorry, I didn't look closely enough. Since you mentioned Streaming data I thought we were talking about the Strategy Monitor.

Streaming data has nothing to do with it. We're talking about yesterday's closing price, right?

For me, it's correct.


When did you update the data? It seems that a partial bar found it's way through.
0
MIH8
- ago
#3
I already must apologize because the situation was different. I did not stream today, which means i did not use the strategy monitor.

I took the signals for today from a strategy window (backtest). IB was the provider, so i would assume the data is from IB. There "should" no other data source be involved as the provider was choosen explicitly (but not exclusivly).

1. Which number would you expect? The "12.42" value or the historical close from the previous day 12.09? (guess 12.09)

Now in this scenario another question came up?



The order price in the quotes tool is based on a 4% dip on the close, but none of the above prices can be the source for the order price. Any idea?
0
MIH8
- ago
#4
We are writing at the same time :-).

Well, I think I did the update quite soon after yesterday's session? I don't normally do that. But look at your close.
0
Cone8
 ( 21.52% )
- ago
#5
I have no idea with the information given.
0
MIH8
- ago
#6
Your close (12.8) is also different from the IB chart (12.42)
0
Cone8
 ( 21.52% )
- ago
#7
I take that back 11.61 / 0.96 = 12.09, which was your close

Probably you updated too soon and the daily bar wasn't ready with the settled close.. It's a question for IB.

My IB Chart shows 12.80 as yesterday's close.

0
Best Answer
MIH8
- ago
#8
Thanks for the screenshot. What the hell ...

Well, your IB shows you 12.8 and your historical data is 12.8 too, i see everything is fine for you. On my side i have a different historical price than you, a different price in the IB chart than you, and different closes between historical and live data. Damn.

And a "hidden" price that was used for the order price calculation. None of the prices is the result of close * 0.96.
0
Cone8
 ( 21.52% )
- ago
#9
It turns out that 12.42 is the correct price. My chart and TWS is returning 12.80 because the chart is configured to return data outside of market hours. There was an aftermarket tick at 12.80. I had no idea that would affect the Daily bars for the API.
0
MIH8
- ago
#10
Ok, if I had updated at a later time, there is a chance that I would have got the correct 12.42. So the first question of #3 can be ignored until I have a reason to look for another cause.

EDIT: OK, the order price "11.61" is of course the result of the 12.09 close of my (wrong) close. Thanks a lot for the help. I might delay IB updates a bit in the future.

I already had two issues with IB updates. One was within the pre-market hours (i reported it) and now direclty after the session. On the fly we saw a side effect for you too. Need to find a good time for IB updates.
0
MIH8
- ago
#11
Can you tell me if there is a way to clean\correct the IB data. I am not sure how often i did an update in the past after the session or during pre-markes ... ?

I would like to remove such erroneous data. Do i need to delete and reload it?
Data integrity tab won't help i guess ?
0
Cone8
 ( 21.52% )
- ago
#12
As you know, IB is slow to load, so I wouldn't recommend a refresh unless absolutely required. Probably you can truncate the last several days or weeks, but you need to do it by DataSet and Scale. Select the DataSet/Scale and use Data Truncation at the bottom.
1
MIH8
- ago
#13
Hello Cone,
Hello Eugene,

I find it time consuming to keep the historical data for IB clean. For this reason, I mainly use WD, QD, EODHistorical, Firstrate data for the historical data. Until the data provider for EODHistorical data is ready, I use ASCII imports for this data.

For the streaming data I am using IB, but I might consider EODH once it is available. So now my question.

When using the strategy monitor, the data range can be set in the settings.
Do the bars that are loaded come from the local files or are they freshly downloaded from the provider (before the streaming starts)?

For EOD trading, I only use WD+QD (EODH) to generate signals. But for intraday trading, I don't want to use historical IB data to generate signals based on the previous day's close (unless they are based on a fresh download). I see two options for me.

1. using local intraday data in the SM. If I have my ascii provider before IB in the data manager, will the SM load the data from the ASCII files or from the local IB source in this special context?

2. if the bars are loaded from IB (not from local files), I just use IB.
0
Cone8
 ( 21.52% )
- ago
#14
QUOTE:
I find it time consuming to keep the historical data for IB clean.
Why is that? What are you doing to clean it? (After reading further, I suspect you're updating Daily data too soon before the settled close and corrections are applied. You can avoid that by scheduling updates or EOD Strategy runs for the next morning.)

QUOTE:
Do the bars that are loaded come from the local files or are they freshly downloaded from the provider (before the streaming starts)?
They're combined. Cached data are loaded and then new bars are requested from the end of that local data (and before if required) and combined with what comes back. If it's not up-to-date, it goes through the Historical Data Provider priority, as usual.

1. Unless you're dynamically updating ASCII data, it's not an option (read: don't use it) for intraday strategies in the Strategy Monitor. Data aren't combined from different providers.... at least I don't think so!

2. Yes, and Streaming Bars or Polling is recommended so that the result (the new bars added) matches the historical charts that you will request at some later date.
0

Reply

Bookmark

Sort