Could-not-load-Benchmark-Symbol-Data-for-WFO-Interval-1

How far back does your SPY 1-min data go?

I'd recommended waiting for Build 33 before continuing with the "Por fin" strategy. The equity curve looks like it could be the result one of those evolver stop and limit exit strategies that should exit on the open but that gives you credit for limit order when it should not.

More information to help you helping me: I backtested the original 1-minute based strategy that uses SPY, using ASCII data (I was stuck in the process but I moved on thanks to your kind support: https://www.wealth-lab.com/Discussion/Could-Not-Load-Benchmark-Symbol-Data-For-WFO-Interval-1-9459). The problems I am having now, are probably related, is that it's not loading the benchmark during the backtests, also when I want to do a WF Optimization using dates I know there is available data. An additional problem I am experiencing is when I clone the strategy to evaluate the same strategy using using SPY symbol instead of its ASCII dataset, it only backtests a couple of days and in this case there is benchmark data, even tough I specify to do it for 4 years. Now look at the image above: it looks like the there is a benchmark indeed, for the last couple of days. So there is a relation between these weird things:

A) When using the ASCII dataset (cached) it shows the backtest results for several years but the benchmark only for a couple of days.
B) When using the SPY symbol (not the ASCII dataset) and I do a 4 years backtest it only does it for a couple of days.
C) When using the ASCII dataset for WFO it is bringing a "Could not load benchmark" error, but I can do a standard optimization.

Note: I was able to clone it and backtesting it with the online SPY symbol for this week, the only period it shows when I want to run the 4 years back again.
If I change the symbol for QQQ or anything else it does backtest the entire 4 years, I thought it could be that I am using the same symbol, SPY, for benchmark and for the backtest but the symbol name in the ASCII dataset is SPY_1MIN

The strategy logic is buying at market when a bullish white candle pattern is recognized and selling at limit, it's very simple.

QUOTE:
How far back does your SPY 1-min data go?

I'd recommended waiting for Build 33 before continuing with the "Por fin" strategy. The equity curve looks like it could be the result one of those evolver stop and limit exit strategies that should exit on the open but that gives you credit for limit order when it should not.

Hi Cone, good to read you!
The SPY 1-min ASCII data (now cached) goes from January 3, 2005.

The strategy is a Buy At Market then Sell At Limit, it's being traded real-time right now in my IB paper trading with live Market Data account with a reasonable win rate.

Is your ASCII DataSet's Provider enabled (checked on) in the Data Manager, Historical Providers tab?

QUOTE:
Is your ASCII DataSet's Provider enabled (checked on) in the Data Manager, Historical Providers tab?

Hi Glitch, it is enabled indeed.

Here is the resulting image if I use QQQ instead of SPY as a benchmark.

QUOTE:
The strategy is a Buy At Market then Sell At Limit,

Nice! Good trading!

It looks to me like you're testing on ASCII data - SPY (Offline) - but the benchmark is coming from another source higher in the Historical Data Providers list.

1. Close your Strategy window to start over.
2. Drag your SPY (Offline) DataSet to the top of the Historical Data Providers

Now it should get the benchmark from SPY (Offline) too.

I closed the strategy, dragged SPY (Offline), to top as Historical Provider, then ran again, it did the same flat benchmark.

Both Online and Offline modes, same same.

I also tried changing the benchmark for SPY (offline) then I don't get an error message, I get a Loading SPY_1M progress bar that freezes near 100% and never ends.

Whatever I do, even if I create a different brand new strategy with any rule, let's say: Buy At Market when SPY closes above the SMA20, whatever the time I set for running a backtest it runs only from March 6th 2023.

Same thing if I change the symbol. Something related to when I loaded and used the offline dataset has messed up my WL8 backtest and optimization processes.

I will try deleting the imported ASCII data

It didn't work. I deleted the datasets and also the ascii cache folder in WL folder and it keeps running for March 2023 only. It looks like somehow, something related to the use of this ASCII datafile corrupted WL8 defaults. I will do a clean new installation.

QUOTE:
I'm not sure how you conclude that the "datafile corrupted WL8 defaults"

Me either :D :D

Thank you very much, it was just in time!! I will do as you say, it will be much easier and it will allow me to try again with the use of ASCII data

I renamed the user data folder and did all the Data Manager setup again adding IB and Alpaca as historical and streaming providers...

I am still having problems, I create a 1-minute SPY buy sell next bar strategy and it backtests for one year, I change the time to two years and it keeps backtesting for one year. :(

I change the time to 3 years and it keeps backtesting one year.

Can you email us the data file in question to support@wealth-lab.com? You mention IB, Alpaca, ASCII so a bit confused at this point.

QUOTE:
Can you email us the data file in question to support@wealth-lab.com? You mention IB, Alpaca, ASCII so a bit confused at this point.

Yes I can. Indeed it's so confuse at this point. I deleted the datafile from the user folder and then the entire WL user data folder but the problem persists:

I create any SPY 1-minute strategy, I backtest it for a year, then no matter what time I select, it backtest it only for a year!
I select, 2, 3, 4 or 5 years but it keeps backtesting only for one year, and with the default user datafolder because I deleted the previous one and restarted WL. The only change I did was adding IB and Alpaca as historical and stream providers, nothing else, it's a default WL.

Maybe the registry is corrupt, I don't know, I am just speculating as I have no idea of what causes this problem.

By the way: if I change double the data range so then and only then it starts to load new data (in a slow batch process, one by one). It seems like if I increase data range in small steps it does the same with the data cached from previous runs (I am speculating, sorry for my lack of technical knowledge), but if I significantly increase the data range then it starts to load new data.

Please try to see if the error occurs with you too or if it's only me: A) create a Buy At Market, Sell At Market, 1-minute, SPY symbol strategy; B) backtest it for a year; C) backtest it for 2 years and check if it did the 2 years backtest

An update on this:

I made a new clean installation of WL8 and ran any 1-minute based strategy using UVXY, for one year, no ASCII data, just my historical providers Alpaca and IB, it downloaded the data and performed the process, then I made a second run with a two years data range and performed only a year, then I switched to 3 years and the process bar looks like frozen (never ends). It looked like Alpaca historical data processer went on strike. I moved IB on top as an Historical Data provider in the Data Manager, then I ran a 3 years backtest period but it only tested one. :(

It looks like the problem is not related with the datafile I loaded before, it's something related to the handling of new vs existing historic data.

I hope this update helps to find the problem and to improve this process in particular, because WL8 capabilities are amazing in short intraday terms too!

Luckily I solved my problem by comparing my WL8 installation with my second computer where all backtests load any data range of historic data and performs okay for all the symbols in the dataset: the only difference was that in the version that's working fine there was no Alpaca and Norgate extension either. I am not sure which extension was causing the problem (I think Alpaca because I wasn't using Norgate), but I made a new clean installation without these extensions and now it works perfectly.

Although I solved it I would like if you could please check if you can reproduce the problem, in case it was indeed somehow caused by Alpaca extension (or Norgate?).

I requested access to the google drive file but you haven't given it to me. It would be easier if you use the option to "give access to anyone with the link".

Oh, I am very sorry. I never use Google Drive. I will give you access, anywise the file is a false clue ==> In my second computer backtests and optimizations are okay in any time range and on every symbol. That machine has not Alpaca and Norgate extension installed. I made a clean installation without Alpaca and Norgate on my main computer and everything worked perfect. I think the problem is related with Alpaca or its extension because I don't use Norgate yet.

Hello.

I made a clean installation of WL8 without Alpaca and Norgate extensions. I worked ok with intra-day data, no need to load any ASCII file, so the problem is not related to the data file. Today it started to work bad again: it performs backtest only for 4 years no matter if I specify 6, 8 or ten.

Where you, please, be able to analyze the problem? It is not related with the data file.

After running this one time, it immediately identified the problem. Your symbol name.

The name of the ASCII file is the symbol symbol name. If the filename is SPY_1MIN, then the symbol is SPY_1MIN.

Now look at your Strategy Settings in the first post. You specified "SPY" as the benchmark. That's not the right symbol for your large 1-minute datafile.

Probably it's a good idea in this case to leave that file named SPY_1MIN. That way, when you enter SPY_1MIN for the benchmark symbol, it will always find the data in that file and nowhere else.

By the way, your ASCII data's timestamps are "start-of-bar", so you should have checked this option when creating the DataSet. (You can check/change the Configuration by right-clicking the DataSet > Configure...)

Thank you Cone.

This is a useful answer and helps to manage ASCII data.

I analyze the Walk-Forward strategy on Data Scale: Daily.

Optimization is successful.

I analyze Walk-Forward strategy on Data Scale: 60 Minute or 1 Minute.

An error is thrown.

https://drive.google.com/file/d/1YsTF4NUrs9YUMO3Pq1hlUx-sfj4x91Dh/view?usp=sharing

There is no such error in WL6. Everything works there.

Can you please tell me how to perform Walk-Forward analysis on Data Scale: 60 Minute, 1 Minute?

It's because you don't have an intraday provider for SPY, the benchmark.

I thought we had implemented a work-around for that scenario, but there could be an issue with it for optimizations.

For now, just use the same symbol as your benchmark, or another one that has intraday data.

WL8 could not load data for the benchmark symbol, SPY, in 60-minute data.

The solution is to go to the Data Manager, Historical Providers tab, and turn on a data provider that can deliver 60-minute data for SPY.

Or, change the benchmark symbol to one that you know you have 60-minute data for.

Cone, hello!

I set the "Benchmark" field as you advised, I get the same error.

I can't make that happen using some of the data you've sent before (SPFB.RTS_YFL_20200508).

With those settings, just run a backtest with 60 minute bars and show us the Equity curve with the Benchmark checked (to see the Benchmark equity too).

Make sure your DataSet is enabled in the Data Manager, Historical Providers tab.

Cone, hello!

Couldn't reproduce the error. If I run into it again, I'll let you know.

Ran into another critical bug today with a different error message, but couldn't reproduce either.

Glitch, hello!

My DataSet is enabled in the Data Manager.