TD-Ameritrade-streaming-aborts

As for streaming, I watched 5 active charts for the last hour with TDA streaming and didn't see a problem. It could help if you identified the symbols (and how many) and to lesser extent, the chart interval. I don't know what the TD limits are for number of streams, but my feeling is that you may have trouble if requesting more than 20 or 30 symbols.

Re: 2) Are we able to discard the most obvious possibility? i.e., the FIlter Pre/Post checkbox at the bottom of the chart? Also, you should make sure the correct Market is selected there too. I haven't looked Pre-market, but TD streaming is working Post-market right now.

Hello Cone,

I'm streaming only 2 symbols ARKG and INTC. With IBKR, I don't face any issues with streaming.

Yes, I do have Filter Pre/Post checked at the bottom of the chart and US Stocks selected as Market. Again with IBKR streaming, I have the Filter Pre/Post filter checked and there are no issues.



Also, I noticed that when I try to Run Backtest with Most Recent N months set to 6 in Strategy Settings then the historical 1-minute data downloaded from TD Ameritrade before 11/8/2021 is in CST time zone instead of New York EST time zone. The data after 11/8/2021 till today (11/22/2021) is in EST time zone and was downloaded by updating TD dataset from Data Manager. I'm not sure if that's the reason why TD Ameritrade doesn't start streaming when Market opens. How do I fix the time zone issue in TD Ameritrade?

QUOTE:
the historical 1-minute data downloaded from TD Ameritrade before 11/8/2021 is in CST time zone instead of New York EST time zone.

That sounds like a problem that we need to investigate.

What is your computer's local time zone?

Are you certain of the source of the data?
Since you have IB too, and we don't know the priority of your Historical Data Providers, be sure check the Data Source in the chart window's status bar.

QUOTE:
..why TD Ameritrade doesn't start streaming when Market opens

I'll check that today. Any particular tool where you noticed that? Quotes? Charts? Strategy Monitor?

I tested with a Quotes and 5 streaming charts in a Workspace set up before the open. Everything started fine for first 7 minutes until another team member logged in and started using the Stream from a different location. That killed my connection with Streaming. You're not using more than one connection, are you?

I'll set up again an eye on it until something happens.

My computer local time zone is in CST. In TWS, I have set it to EST zone so I always get streaming data in EST zone. For TD Ameritrade, I'm not sure how time zone can be set. In strategy settings, I made sure to select TD Ameritrade dataset. With IBKR I cannot get more older data than 11/8/2021. IBKR data is always in EST zone as its been set in TWS. I'm sure the incorrect time zone is coming from TD Ameritrade.

Today I noticed that TD Ameritrade did start streaming on market open but it lasted only for 56 minutes. At 9:24AM local time (market opens 8:30AM local time), the connection to TD Ameritrade was disrupted and it couldn't be recovered. Whereas with IBKR, the connection broke as well at the same time but the next minute the connection was restored automatically. Is there something we can do automatically restore TD Ameritrade connection?

QUOTE:
With IBKR I cannot get more older data than 11/8/2021.

Sure you can, but "deep" initial historical data requests from IB will take a while.

Preferences > Data > Intraday Minute-based Days Multiplier controls the data loaded when you request an intraday chart. Especially for IB, I recommend keeping that number as small as possible. However, if you need more history for a backtest, just change the range and Wealth-Lab will request more back fill. NOTE - for IB this will take a LONG time if you're looking for even just 1 year of 1-minute data for 1 symbol - (on the order of 5 to 10 minutes).

Try this - Put IB high in the list of Historical Providers and perform an intraday backtest for 1 symbol with an extended history. Again, give it a lot of time to load the data the first time you request a symbol's history from IB.

Although TD is a lot faster at returning history, the depth is actually way more limited - like only 6 weeks it seems. Of course, Wealth-Lab will cache more history if you keep your data updated.

QUOTE:
At 9:24AM local time (market opens 8:30AM local time), the connection to TD Ameritrade was disrupted

Yup, I can see that in the Log.

QUOTE:
Is there something we can do automatically restore TD Ameritrade connection?

No, it's something the data adapter need to detect and take action on. We'll put it in the queue.

Another update today. Bars streamed well past market open without any issues. But after 1PM EST, the streaming stopped without any errors.

Here's the bar chart of ARKG from TD Ameritrade in WL7


The same chart from TWS at the same time:


WL7 shows not issues logged.


Luckily, I was long and price went up otherwise I would have made a loss. I had to disable and enable streaming again. But then streaming was getting aborted multiple times after manually enabling streaming. Even loading up the save workspace didn't resolve the issue. I had to close and restart WL7 and load the saved workspace again. Now streaming is stable.

Please look into this instability in the connection as well. It's kind of dangerous as no errors were logged and streaming chart show green status even though charts were not updating.

We'll definitely keep looking at it. As you can imagine, these types of intermittent issues are difficult to isolate.

I noticed the new TD extension build # 13 release notes mentioned about capability to re-connect. I tried it out and it did attempt to reconnect but it never was able to restore the connection. Whereas IBKR extension was able to restore the connection.

Here's the snapshot:


Both IBKR and TD lost connection at 2:01 AM and at 2:02:50 AM, IBKR extension was able to restore the connection but TD extension could never restore the connection.

Here are more details about reconnect attempt fail by TD extension

It was reconnecting when we developed and tested this capability, but all configurations and situations are different, and I'm sorry it's not reconnecting at your end. I will see what else I can come up with based on those log messages, thanks for posting!

The "No such Host is Known" error makes this sound like a DNS error, is it possibly you have sporadic internet issues in general?

Some days the internet can get interrupted by 1-2 minutes. I think it will be good for the tool to attempt reconnect every 30s just like TWS does. If it tries to reconnect once then it may not work if internet is out for more than a minute.

Today, it looks like reconnect was not even attempted by TD extension.

At 8:44AM local time (9:43AM EST), internet disrupted 3-4 secs and got restored. It can be seen from IB logs but TD extension didn't show anything about connection being disrupted and reconnect being done. But from streaming chart it's evident that streaming stopped as last bar was 9:43AM EST whereas time shown in the window is 9:50AM EST. The streaming status is also green.

Snapshot of log:


Snapshot of streaming chart:


I think the internet disruption was too short (3-4 secs) for TD extension to detect it. It would be ideal if extension can check connection every 10 secs and attempt to reconnect to ensure there is no issue.

Does the latest TD Ameritrade extension build improves re-connection attempt similar to IB?

The latest TD Build does support auto-reconnect, yes. If you're on the latest build of the TD extension and still are not seeing any reconnect, let us know!