How-I-can-override-Limit-Order-logic

2023-09-23T13:20:58Z - 2023-09-23T13:20:58Z ago

I employ a 1-minute interval strategy, which typically takes 5-10 seconds to execute. This means that by the time the execution is completed, the initial open price and the first 10 seconds of the 1-minute bar have already passed. Consequently, I intend to revise the entry price to consistently align with the worst-case scenario, which is the low of the bar, as opposed to the current approach of selecting the best price between the open or limit.
I use this logic after the position is created, but it doesn't seem to work well:

CODE:

            foreach (Position pos in OpenPositionsAllSymbols)
            {
                if (pos.EntryBar == bar)
                {
                    pos.EntryPrice = pos.PositionType == PositionType.Long ? pos.Bars.High[entryBar] :  pos.Bars.Low[entryBar];
                }
}

Is this correct or is there a better ways to go around it?

Thanks.

486

Solved

7 Replies

Bookmark

Notify

Sort

Glitch8

( 12.10% )

2023-09-23T14:30:57Z - 2023-09-23T14:30:57Z ago

I think you should just use the slippage settings. Since this is a limit order it cannot execute below the order price. Either it fills or the price moves higher on that 10 seconds causing the limit order to not get filled. This is all modeled by the Limit Order Slippage Backtest Preference.

And why assume the worst case? Isn’t it also possible that the price moves lower in that 10 seconds and you actually get a better entry price, lower than the limit price?

MohammadRashid8

2023-09-23T15:57:03Z - 2023-09-23T15:57:03Z ago

Background: My trading strategy involves frequent trading of QQQ and SPY, equivalent to 10 ES futures contracts. While the slippage is slightly higher for QQQ compared to SPY, on average, both securities trade within a few cents of the last minute's closing price. However, during morning hours or periods of high volatility, they can experience larger price fluctuations. To mitigate potential losses, I consistently use limit orders and account for slippage.

Despite implementing slippage adjustments and fees in my trading strategy, my live trades consistently underperform compared to backtesting results, often by 10-20% on a daily basis. I've noticed that most of the time, I end up getting the open price during backtesting, which is different than live trades.

I'm currently unsure how to address this disparity, and I'm considering scheduling a consultation session to seek guidance and potential solutions from one of you.

Cone8

( 6.32% )

2023-09-23T17:28:17Z - 2023-09-23T17:28:17Z ago

QUOTE:
takes 5-10 seconds to execute

That's a ridiculous amount of time. Are you designing a new optimal digital filter with non-negative Fourier transform, or something?

My guess is that you're trading with too much data.
Don't load more data than you need. A 1-minute strategy is likely to need no more than about 2 weeks of data, and probably even much less.

QUOTE:
I'm currently unsure how to address this disparity,

Analyzing 1 trade that you didn't get should be sufficient to explain it.
If you're using limit orders, you can't do "worse" than the backtest - except by not participating in profitable trades because the "train left without you".

MohammadRashid8

2023-09-23T18:52:51Z - 2023-09-23T18:52:51Z ago

I have a number of indicators and most of the time is spent on initialization.
loading 1-minute data from 8/30/23 until now takes about 5-seconds.
See code below ( IQFeed is slower on weekends. so it might be faster on weekdays). my computer is Dell 32 GB 11th Gen Intel, I9 @ 3.5GHz 8 Cores

CODE:

using WealthLab.Backtest;
using System;
using WealthLab.Core;
using WealthLab.Data;
using WealthLab.Indicators;
using System.Collections.Generic;
using WealthLab.AdvancedSmoothers;

namespace WealthScript1 
{
    public class MyStrategy : UserStrategyBase
    {
      DateTime startTime;
      VWAP spyVwap, qqqVwap = null;
      IndicatorBase vidyaF, vidyaS;
      IndicatorBase highest = null, lowest;
      BarHistory qqqBars;

      private IndicatorBase lower;

      private IndicatorBase upper;
      private IndicatorBase emaBb;



      //create indicators and other objects here, this is executed prior to the main trading loop
      public override void Initialize(BarHistory bars)
        {
         startTime = DateTime.Now;

         spyVwap = new VWAP(bars, 0930);

         vidyaF = new VIDYA(bars.Close, 2);
         vidyaS = new VIDYA(bars.Close, 7);

         highest = new Highest(bars.Close, 7);
         lowest = new Lowest(bars.Close, 8);

            PlotIndicator(spyVwap, WLColor.Blue);

            //PlotIndicatorLine(vidyaF, WLColor.Black, 1, LineStyle.Solid, false);
            PlotIndicatorLine(vidyaS, WLColor.Red, 2, LineStyle.Solid, false);
         
         qqqBars = GetHistory(bars, "QQQ");
         qqqVwap = new VWAP(qqqBars, 930);
         lower = new BBLower(qqqBars.Close, 9, 1.81);
         upper = new BBUpper(qqqBars.Close, 10, 2.0);
         emaBb = new EMA(qqqBars.Close, 9);
         
         
         DateTime endTime = DateTime.Now;
         double executionTime = (endTime - startTime).TotalSeconds;
         DrawHeaderText("Execution Time: " + Math.Round(executionTime, 2));


      }

        //execute the strategy rules here, this is executed once for each bar in the backtest history
        public override void Execute(BarHistory bars, int idx)
        {
            if (!HasOpenPosition(bars, PositionType.Long))
            {
                //code your buy conditions here
            }
            else
            {
                //code your sell conditions here
            }
        }

        //declare private variables below

    }
}

Cone8

( 6.32% )

2023-09-23T19:25:21Z - 2023-09-23T19:25:21Z ago

This explains a lot. The call for the external symbol is the problem. And, it's likely the cause of variability in live trading.

Here's the solution -

Set up this script to run the Strategy Monitor with IQFeed: Streaming or Streaming Bars, 1-Minute bars. Load the history required and it will update the QQQ bars in Global memory.

CODE:
using WealthLab.Backtest;
using System;
using WealthLab.Core;

namespace WealthScript4 
{
    public class BarsToGlobal : UserStrategyBase
    {   
        public override void Initialize(BarHistory bars)
        {
         string gkey = bars.Symbol + "," + bars.Scale.ToString();
         SetGlobal(gkey, bars); 
        }
        
        public override void Execute(BarHistory bars, int idx)
        {  }
    }
}

Now, add this GetHistoryGlobal() routine to your strategy and use that. This will load the QQQ bars as soon as they're available with the same "last date/time" as the source bars. Normally, it should happen in less that 1/2 second.

CODE:
using WealthLab.Backtest;
using System;
using WealthLab.Core;
using WealthLab.Data;
using WealthLab.Indicators;
using System.Collections.Generic;
using WealthLab.AdvancedSmoothers;
using System.Diagnostics; 

namespace WealthScript1
{
   public class MyStrategy : UserStrategyBase
   {
      VWAP spyVwap, qqqVwap = null;
      IndicatorBase vidyaF, vidyaS;
      IndicatorBase highest = null, lowest;
      BarHistory qqqBars;

      private IndicatorBase lower;
      private IndicatorBase upper;
      private IndicatorBase emaBb;

      Stopwatch _sw = new Stopwatch(); 

      public override void Initialize(BarHistory bars)
      {
         _sw.Reset();
         _sw.Start();

         spyVwap = new VWAP(bars, 0930);

         vidyaF = new VIDYA(bars.Close, 2);
         vidyaS = new VIDYA(bars.Close, 7);

         highest = new Highest(bars.Close, 7);
         lowest = new Lowest(bars.Close, 8);

         PlotIndicator(spyVwap, WLColor.Blue);

         //PlotIndicatorLine(vidyaF, WLColor.Black, 1, LineStyle.Solid, false);
         PlotIndicatorLine(vidyaS, WLColor.Red, 2, LineStyle.Solid, false);

         qqqBars = GetHistoryGlobal("QQQ", bars, true, 3);
         
         if (qqqBars == null)
            qqqBars = GetHistory(bars, "QQQ");   // backup only
         qqqVwap = new VWAP(qqqBars, 930);
         lower = new BBLower(qqqBars.Close, 9, 1.81);
         upper = new BBUpper(qqqBars.Close, 10, 2.0);
         emaBb = new EMA(qqqBars.Close, 9);


         _sw.Stop();
         double executionTime = _sw.Elapsed.TotalMilliseconds / 1000.0;
         DrawHeaderText($"Execution Time: {executionTime:N2}", WLColor.NeonOrange, 14);
      }

      
      public override void Execute(BarHistory bars, int idx)
      {
         if (!HasOpenPosition(bars, PositionType.Long))
         {
            //code your buy conditions here
         }
         else
         {
            //code your sell conditions here
         }
      }


      // returns the synched or unsynched BarHistory for the symbol-scale if found in global memory
      // allows up to 5 seconds to return
      private BarHistory GetHistoryGlobal(string symbol, BarHistory sourceBars, bool syncToSource = true, double maxSeconds = 5)
      {
         BarHistory symBarsRaw = null;
         DateTime symLastDate = DateTime.MinValue;
         string gkey = symbol + "," + sourceBars.Scale.ToString();

         DateTime start = DateTime.Now;
         do
         {
            if (HasGlobal(gkey))
            {
               symBarsRaw = (BarHistory)GetGlobal(gkey);
               if (symBarsRaw?.Count > 0)
               {
                  if (symBarsRaw.DateTimes[symBarsRaw.Count - 1] == sourceBars.DateTimes[sourceBars.Count - 1])
                     break;
               }               
            }
            System.Threading.Thread.Sleep(10);
         } 
         while ((DateTime.Now - start).TotalSeconds < maxSeconds);     // 5 seconds max

         if (syncToSource)
            return BarHistorySynchronizer.Synchronize(symBarsRaw, sourceBars);
         
         return symBarsRaw;                                              // these bars are unsynched
      }
   }
}

You can test it right now...
Just run the first script on QQQ 1-minute bars, and make sure to use the market filter so that it ends at 4pm on Friday.

Then run your VIDYA script on anything else with 1-minute bars, with the U.S. stocks market filter. Should finish in under 0.5 seconds.

Best Answer

MohammadRashid8

2023-09-23T21:36:20Z - 2023-09-23T21:36:20Z ago

Magic! the whole strategy executed in 0.07 seconds.
Thank you for your help.

Cone8

( 6.32% )

2023-09-24T06:06:48Z - 2023-09-24T06:06:48Z ago

The problem you were having was because each time the Strategy calls GetHistory for 1-Min data, WealthLab has to open, read - and during Market hours - write to the ~100 MB file cache of the 1-Min bars. This takes time.

Furthermore, when trading live, the GetHistory call takes place momentarily after the 1-minute mark, and unless you delay the call by a second or two, it's unlikely that the history data will be available on the history server. If not, you end up using a duplicate bar from the previous minute.

Now you'll be able to avoid all that by streaming the external symbol data and putting it in Global memory where GetHistoryGlobal will wait maybe 10 or 20 msec (but up to a few seconds, if required) for the latest bar to become available.

Bookmark

Notify

Sort