TD-Ameritrade-pre-market-intraday-data-gaps

paul19868

2024-03-01T13:11:05Z - 2024-03-01T13:11:05Z ago

TD Ameritrade pre-market intraday data gaps

*** First, I believe this IS NOT a Wealth-Lab issue ***

This is to let TD Ameritrade users know about the following intraday data issue...

I noticed this morning (2024/03/01) that at least two stocks I checked, CLSK and MARA, were missing premarket one-minute bars (at least) in Wealth-Lab when compared to the TD Ameritrade ThinkOrSwim app.

In Wealth-Lab the bars stopped at 2024/02/29 8:00 PM for the day before (Thursday), as expected. But, for Friday the bars started at approximately 7 AM. However, in ThinkOrSwim there were a lot of bars between 8 PM Thursday and 7 AM Friday.

Glitch has noted in other posts that Wealth-Lab is not the issue. WL is simply requesting the data via the appropriate TDA API and storing the results.

If you want to scan for these incidents here's a simple strategy. However, you should check that the symbol is typically active premarket.

CODE:
using System;
using System.Collections.Generic;
using WealthLab.Backtest;
using WealthLab.Core;
using WealthLab.Data;
using WealthLab.Indicators;

namespace WealthScript1 
{
   /// <summary>
   /// Scan for apparently missing premarket intraday bars with data sourced from TD Ameritrade.
   /// Note that the symbol must be typically active between 8 PM (one trading day back) and 7 AM.
   /// </summary>
   public class MyStrategy : UserStrategyBase
   {
      //create indicators and other objects here, this is executed prior to the main trading loop
      public override void Initialize(BarHistory bars)
      {
         if (!bars.IsIntraday)
         {
            return;
         }

         WriteToDebugLog($"Scale: {bars.Scale}");
         
         for (int idx = 1; idx < bars.Count; idx++)
         {
            var oneBack = bars.DateTimes[idx - 1];
            var current = bars.DateTimes[idx];
            // At the intraday scale, did we just start a new day and the hour >= 7?
            if (oneBack.Date != current.Date && current.Hour >= 7)
            {
               WriteToDebugLog($"{bars.Symbol}: seems to be missing some premarket bars from {oneBack} to {current}");
            }
         }
      }

      //execute the strategy rules here, this is executed once for each bar in the backtest history
      public override void Execute(BarHistory bars, int idx)
      {
      }
   }
}

152

2 Replies

Bookmark

Notify

Sort

MedvedTrader8

2024-03-01T13:19:06Z - 2024-03-01T13:19:06Z ago

TD Ameritrade datafeed cannot return equity historical data before 7 AM. Think of Swim uses an entirely different datafeed.

paul19868

2024-03-01T13:56:47Z - 2024-03-01T13:56:47Z ago

QUOTE:
TD Ameritrade datafeed cannot return equity historical data before 7 AM. Think of Swim uses an entirely different datafeed.

But, in some cases it does. I understand that ThinkOrSwim uses an entirely different data feed.

Here's a modified version of the above code. Using only TD Ameritrade as an intraday data source run the strategy on symbol ADIL, one minute scale.

CODE:
using System;
using System.Collections.Generic;
using WealthLab.Backtest;
using WealthLab.Core;
using WealthLab.Data;
using WealthLab.Indicators;

namespace WealthScript1 
{
   /// <summary>
   /// Scan for apparently missing premarket intraday bars with data sourced from TD Ameritrade.
   /// Note that the symbol must be typically active between 8 PM (one trading day back) and 7 AM.
   /// </summary>
   public class MyStrategy : UserStrategyBase
   {
      //create indicators and other objects here, this is executed prior to the main trading loop
      public override void Initialize(BarHistory bars)
      {
         if (!bars.IsIntraday)
         {
            return;
         }

         WriteToDebugLog($"Scale: {bars.Scale}");
         
         for (int idx = 1; idx < bars.Count; idx++)
         {
            var oneBack = bars.DateTimes[idx - 1];
            var current = bars.DateTimes[idx];
            // At the intraday scale, did we just start a new day and the hour >= 7?
            if (oneBack.Date != current.Date && current.Hour >= 7)
            {
               WriteToDebugLog($"{bars.Symbol}: seems to be missing some premarket bars from {oneBack} to {current}");
            }
            else if (oneBack.Date != current.Date && current.Hour < 7)
            {
               // we just started a new trading day at idx, so count the bars up until 7 AM
               var count = 0;
               var atOrAfter7AMBar = idx;
               for (int i = idx; i < bars.Count; i++)
               {
                  // count the bars until we hit 7 AM
                  if (bars.DateTimes[i].Hour >= 7)
                  {
                     atOrAfter7AMBar = i;
                     break;
                  }
                  count++;
               }
               WriteToDebugLog($"{bars.Symbol}: has {count} bars from {oneBack} to {bars.DateTimes[atOrAfter7AMBar - 1]}");
            }
         }
      }

      //execute the strategy rules here, this is executed once for each bar in the backtest history
      public override void Execute(BarHistory bars, int idx)
      {
      }
   }
}

Results from ADIL scan...

ADIL: seems to be missing some premarket bars from 2/20/2024 7:45:00 PM to 2/21/2024 7:02:00 AM
ADIL: has 1 bars from 2/21/2024 7:58:00 PM to 2/22/2024 5:42:00 AM
ADIL: seems to be missing some premarket bars from 2/22/2024 6:44:00 PM to 2/23/2024 7:50:00 AM
ADIL: has 1 bars from 2/23/2024 5:30:00 PM to 2/26/2024 5:15:00 AM
ADIL: seems to be missing some premarket bars from 2/26/2024 7:46:00 PM to 2/27/2024 8:16:00 AM
ADIL: has 1 bars from 2/27/2024 6:47:00 PM to 2/28/2024 5:46:00 AM
ADIL: has 98 bars from 2/28/2024 8:00:00 PM to 2/29/2024 6:57:00 AM
ADIL: seems to be missing some premarket bars from 2/29/2024 8:00:00 PM to 3/1/2024 7:01:00 AM

ADIL chart showing bars for premarket 2024/02/29...

Bookmark

Notify

Sort