Profit per bar changes significantly following market close
Author: superticker
Creation Date: 11/13/2019 7:08 AM
profile picture

superticker

#1
Could someone please explain why the profit-per-bar changes dramatically immediately after the US market closes (see screenshot)? Is this by design? Some comments:

1) Yes, on-demand pricing is turned on, which means after the market closes, on-demand pricing automatically loads the new bar just created for all 33 stocks in this dataset. But one new bar shouldn't affect the profit-per-bar calculation that much ... right? Why does it?

2) One very minor change was made to the strategy in the screenshot between 4:30pm EST and 11:30 EST.

3) WLP 6.9.20.7 is running in the CST time zone (GMT-6). Does switching between Daylight Saving Time and Standard Time affect the profit-per-bar calculation? I would think Market Manager would automatically compensate for the time changes relative to US market closings.
profile picture

Eugene

#2
Two bars. 800 bars (your figure) vs. 802 bars (as reported by WL) in the first place, then significant differences in the average/net profit, and the fact that unknown system has underwent some changes - I see no reason why PPB should remain the same.
profile picture

superticker

#3
QUOTE:
Two bars. 800 bars (your figure) vs. 802 bars (as reported by WL) in the first place, then significant differences
But wouldn't the calculation in all three "time cases" be based on 800 bars, and not 802 bars? Or is the 4:30pm calculation special (802 bars)?

Just in case you're wondering, this behavior happens all the time day-after-day. It's not unique to today. And the Net Profit difference is (506K-679K)/679K*100 = 25% difference for a 2/800*100 = 0.25% bar change. Something is really wrong here.

Explain why all the numbers for 3:00pm and 11:30pm are roughly the same--as expected, but the Net Profit numbers for 4:30pm have this 25% difference? What's special about the 4:30pm calculation?

I think the 3:00pm and 11:30pm calculations are correct, but the 4:30pm calculation has a problem. And this is the case for all days immediately after the market closes. I've known about this problem for two years, but only mentioned it now. I "think" it has something to do with the time zone WLP is operating in. The problem does correct itself before the next trading day.
profile picture

Eugene

#4
QUOTE:
But wouldn't the calculation in all three "time cases" be based on 800 bars, and not 802 bars? Or is the 4:30pm calculation special (802 bars)?

New bar is appended to your backtest as on demand update is enabled. In case of an intraday backtest, it would happen all day long.

QUOTE:
Explain why all the numbers for 3:00pm and 11:30pm are roughly the same--as expected, but the Net Profit numbers for 4:30pm have this 25% difference?

Not only your code undergoes modifications, it's been unknown. Unless you're willing to provide a complete test case, I'm afraid there's nothing for me to explain in this context.

A good test case should include:

1. Strategy, public downloadable Strategy or sample code used to demonstrate the anomaly.
2. Single symbol or Multi-symbol mode, Position Sizing, and Data Loading settings
3. Preferences (F12) > Backtest settings / Slippage and Round Lots
4. Symbol(s) and Data provider
profile picture

superticker

#5
My strategy only operates on "daily bars" and there's nothing in the strategy that samples time. Let me try reproducing it with a simple public strategy and see what happens. If it's a Market Manager problem, it should show up with all strategies running in the CST time zone trading US stocks. Let me verify that's the case.

Honestly, I find it difficult to imagine what would cause a 25% difference in Net Profit for a couple hours each day. That's just weird. But it does correct itself with time--that's weird too.

I wonder if it's a WL DataSeries caching update problem after the new (daily) bars enter the chart? I'll try restarting WL and see if that fixes it with refreshed cache entries. (I'm thinking simply closing the strategy, then reopening it won't update the daily DataSeries caches.) I'm sure WL must automatically update the caches for intraday strategies, but I'm not sure that applies to daily-bar strategies when there's an on-demand update.

WL was not restarted for the 4:30pm calculation, but it was restarted for the 11:30pm calculation. So results could be a function of restarts (cache refreshes) rather than time. I do like the caching architecture of WL because it operates invisibly--a feature of good design.
profile picture

Eugene

#6
QUOTE:
If it's a Market Manager problem, it should show up with all strategies running in the CST time zone trading US stocks.

Market Manager does not apply altogether if you're working with Fidelity static data.

QUOTE:
Let me try reproducing it with a simple public strategy and see what happens.

Sounds like a plan.
profile picture

Cone

#7
Bizarre. Avg. Profit % is within 0.01% and with the same number of trades and average bars held. We have to get to the bottom of this one!
profile picture

KGo

#8
It looks like 4:30 PM has about a 25% reduction in trade size. Perhaps copy the trade list for each time and compare shares purchased.
profile picture

superticker

#9
QUOTE:
Market Manager does not apply altogether if you're working with Fidelity static data.
I only work with Fidelity data. And I only work with static data at the beginning of the next trading day.

Concerning the "stale caching" theory, I declare my indicators two ways:

1) Non-cached indicators are declared with the "new" operator prior to entering the trading loop. This represents 90% of the indicators I use. I favor this approach because it reduces overhead (somewhat).

2) Cached indicators are declared with the Indicator.Series member function typically prior to the trading loop. Only about 10% are declared this way, but many WL indicators (including some of my own) do use the .Series member function internally.

So the question remains, after on-demand updates pull in the new daily bars (following the market close), does that leave the non-cached indicators misaligned with the cached indicators? If so, would that misalignment cause a 25% difference in the Net Profit calculation?

For the first question, I'm not sure. For the second question, I don't know. But I'll try to restart WL after the market close and on-demand update complete and see if that clears the problem. I can say the problem always clears itself before the next trading day (static updates employed by then).
---
UPDATE: A DataSeries has a date stamp correct? If restarting WL fixes the problem today, then I'll run a little test comparing the date stamp of a cached DataSeries with a non-cached DataSeries tomorrow. If they come up with an offset of one trading day after the market closes, then we have our answer to the first question. (No guesswork.)
profile picture

Eugene

#10
Please see post #4 above. High level programming languages give enough rope to shoot oneself in the foot. Let's not waste time on unproductive guesswork.
profile picture

superticker

#11
I sampled the Performance of the same 33 stocks today, once at 3:50 EST, and again immediately after market close and the on-demand updates completed at 4:02 EST. Everything worked as expected. My conclusion is that it's not an outright DataSeries caching problem.

I then went manually through my usual stock stop/limit sells for the day, which the exchange does in the background, just to evaluate any possible behavioral optimizations with the strategy I might consider. When I was done, I checked the 33-stock dataset again, and the profit-per-bar problem appeared.

This is when things get weird. I closed my strategy and, without restarting WL, I re-opened my strategy again. The problem vanished! I'm perplex how that's possible. Let's drop this issue until I can determine what causes this problem, and why restarting the strategy fixes it.
profile picture

Eugene

#12
Static variables often mask problems in strategies that may be fixed by restarting them.
profile picture

Cone

#13
@Eugene +1
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).