I've read several posts on this topic here, and the recurring issue is the problem of finding a data provider that can supply market capitalization data.
... I wanted to try a different approach:
I loaded the issued shares for each symbol from another software program into a text file:
Examples of symbols and the list structure:
A 1228504232
AAMI 35709120
AAOI 68279888
AAP 60022245
AAPL 14776353000
AARD 21773272
AAT 61152542
ABAT 129969958
ABBV 1767384632
The list contains approximately 2500 symbols with the corresponding number of shares, which I have saved on my computer:
C:\Trading\Aktienanzahl.txt
I wrote the following code:
When I run the code, it compiles cleanly. The debug log clearly shows that everything is being calculated correctly:
Excerpt:
---Symbol by Symbol Debug Logs---
---CRM---
1. INIT: 2476 stocks loaded.
2. RANKING: Top 5 are AAPL, NVDA, MSFT, AMZN, WMT
---AMZN---
1. INIT: 2476 stocks loaded.
--GS---
1. INIT: 2476 stocks loaded.
The calculation is always correct at different start times, so the 5 stocks with the highest market capitalization are always found.
As you can see in the script, a daily comparison is supposed to take place, and as mentioned, this works flawlessly.
According to the code, the 5 stocks with the highest market capitalization should be bought in quantities of 100, and then sold when they are no longer among the top 5 by market capitalization.
This is precisely where the problem lies. Not a single stock is being bought.
Could one of the C# experts please take a look and see if the problem can be solved?
In my opinion, the biggest hurdle was getting Wealth-Lab to access a list that calculates the correct market capitalization based on the stock symbol and the number of shares issued. As you can see, this works. Why the system isn't buying or selling anything in the backtest is a mystery to me.
Thanks!
... I wanted to try a different approach:
I loaded the issued shares for each symbol from another software program into a text file:
Examples of symbols and the list structure:
A 1228504232
AAMI 35709120
AAOI 68279888
AAP 60022245
AAPL 14776353000
AARD 21773272
AAT 61152542
ABAT 129969958
ABBV 1767384632
The list contains approximately 2500 symbols with the corresponding number of shares, which I have saved on my computer:
C:\Trading\Aktienanzahl.txt
I wrote the following code:
CODE:
using WealthLab.Backtest; using System; using WealthLab.Core; using System.Collections.Generic; using System.IO; using System.Globalization; using System.Linq; namespace WealthScript1 { public class MarketCapRotationFinal : UserStrategyBase { private Dictionary<string, double> _sharesMap = new Dictionary<string, double>(); private List<string> _top5 = new List<string>(); public override void Initialize(BarHistory bars) { try { _sharesMap.Clear(); string filePath = @"C:\Trading\Number of shares.txt"; if (File.Exists(filePath)) { string[] lines = File.ReadAllLines(filePath); foreach (string line in lines) { string[] parts = line.Split(new char[] { ';', ',', '\t' }, StringSplitOptions.RemoveEmptyEntries); if (parts.Length >= 2) { string sym = parts[0].Trim().ToUpper(); if (Double.TryParse(parts[parts.Length - 1], NumberStyles.Any, CultureInfo.InvariantCulture, out double qty)) _sharesMap[sym] = qty; } } } WriteToDebugLog("1. INIT: " + _sharesMap.Count + " shares loaded."); } catch (Exception ex) { WriteToDebugLog("ERROR INIT: " + ex.Message); } } public override void PreExecute(DateTime dt, List<BarHistory> participants) { try { List<KeyValuePair<string, double>> ranking = new List<KeyValuePair<string, double>>(); foreach (BarHistory bh in participants) { if (_sharesMap.TryGetValue(bh.Symbol.ToUpper(), out double shares)) { int tIdx = GetCurrentIndex(bh); if (tIdx >= 0) ranking.Add(new KeyValuePair<string, double>(bh.Symbol.ToUpper(), bh.Close[tIdx] * shares)); } } ranking.Sort((x, y) => y.Value.CompareTo(x.Value)); _top5 = ranking.Take(5).Select(r => r.Key).ToList(); // We only log on the very first bar of the entire backtest if (dt == participants[0].DateTimes[0]) WriteToDebugLog("2. RANKING: Top 5 are " + string.Join(",", _top5)); } catch (Exception ex) { WriteToDebugLog("ERROR PRE: " + ex.Message); } } public override void Execute(BarHistory bars, int idx) { string sym = bars.Symbol.ToUpper(); bool isInTop5 = _top5.Contains(sym); Position pos = FindOpenPosition(bars); if (pos != null && !isInTop5) { ClosePosition(pos, OrderType.Market, 0, "Out"); } else if (pos == null && isInTop5) { / SAFETY: Initially set to 100 units to rule out sizing errors Transaction t = PlaceTrade(bars, TransactionType.Buy, OrderType.Market, 0); if (t != null) t.Quantity = 100; } } private Position FindOpenPosition(BarHistory bh) { foreach (Position p in OpenPositions) if (p.Symbol.Equals(bh.Symbol, StringComparison.OrdinalIgnoreCase)) return p; return null; } }
When I run the code, it compiles cleanly. The debug log clearly shows that everything is being calculated correctly:
Excerpt:
---Symbol by Symbol Debug Logs---
---CRM---
1. INIT: 2476 stocks loaded.
2. RANKING: Top 5 are AAPL, NVDA, MSFT, AMZN, WMT
---AMZN---
1. INIT: 2476 stocks loaded.
--GS---
1. INIT: 2476 stocks loaded.
The calculation is always correct at different start times, so the 5 stocks with the highest market capitalization are always found.
As you can see in the script, a daily comparison is supposed to take place, and as mentioned, this works flawlessly.
According to the code, the 5 stocks with the highest market capitalization should be bought in quantities of 100, and then sold when they are no longer among the top 5 by market capitalization.
This is precisely where the problem lies. Not a single stock is being bought.
Could one of the C# experts please take a look and see if the problem can be solved?
In my opinion, the biggest hurdle was getting Wealth-Lab to access a list that calculates the correct market capitalization based on the stock symbol and the number of shares issued. As you can see, this works. Why the system isn't buying or selling anything in the backtest is a mystery to me.
Thanks!
Rename
Sorry, the code was faulty; here is the correct code:
@Glitch, Could you take a look at the code and see why no symbol is being purchased?
Can I also send the list of all symbols and their corresponding number of shares to an email address, or is it possible to attach a text file here?
CODE:
using WealthLab.Backtest; using System; using WealthLab.Core; using System.Collections.Generic; using System.IO; using System.Globalization; using System.Linq; namespace WealthScript2 { public class MarketCapRotationFinal : UserStrategyBase { private Dictionary<string, double> _sharesMap = new Dictionary<string, double>(); private List<string> _top5 = new List<string>(); public override void Initialize(BarHistory bars) { try { _sharesMap.Clear(); string filePath = @"C:\Trading\Aktienanzahl.txt"; if (File.Exists(filePath)) { string[] lines = File.ReadAllLines(filePath); foreach (string line in lines) { string[] parts = line.Split(new char[] { ';', ',', '\t' }, StringSplitOptions.RemoveEmptyEntries); if (parts.Length >= 2) { string sym = parts[0].Trim().ToUpper(); if (Double.TryParse(parts[parts.Length - 1], NumberStyles.Any, CultureInfo.InvariantCulture, out double qty)) _sharesMap[sym] = qty; } } } WriteToDebugLog("1. INIT: " + _sharesMap.Count + " Aktien geladen."); } catch (Exception ex) { WriteToDebugLog("FEHLER INIT: " + ex.Message); } } public override void PreExecute(DateTime dt, List<BarHistory> participants) { try { List<KeyValuePair<string, double>> ranking = new List<KeyValuePair<string, double>>(); foreach (BarHistory bh in participants) { if (_sharesMap.TryGetValue(bh.Symbol.ToUpper(), out double shares)) { int tIdx = GetCurrentIndex(bh); if (tIdx >= 0) ranking.Add(new KeyValuePair<string, double>(bh.Symbol.ToUpper(), bh.Close[tIdx] * shares)); } } ranking.Sort((x, y) => y.Value.CompareTo(x.Value)); _top5 = ranking.Take(5).Select(r => r.Key).ToList(); // Wir loggen nur beim allerersten Balken des gesamten Backtests if (dt == participants[0].DateTimes[0]) WriteToDebugLog("2. RANKING: Top 5 sind " + string.Join(",", _top5)); } catch (Exception ex) { WriteToDebugLog("FEHLER PRE: " + ex.Message); } } public override void Execute(BarHistory bars, int idx) { string sym = bars.Symbol.ToUpper(); bool isInTop5 = _top5.Contains(sym); Position pos = FindOpenPosition(bars); if (pos != null && !isInTop5) { ClosePosition(pos, OrderType.Market, 0, "Out"); } else if (pos == null && isInTop5) { // SICHERHEIT: Erstmal starr 100 Stück, um Sizing-Fehler auszuschließen Transaction t = PlaceTrade(bars, TransactionType.Buy, OrderType.Market, 0); if (t != null) t.Quantity = 100; } } private Position FindOpenPosition(BarHistory bh) { foreach (Position p in OpenPositions) if (p.Symbol.Equals(bh.Symbol, StringComparison.OrdinalIgnoreCase)) return p; return null; } } }
@Glitch, Could you take a look at the code and see why no symbol is being purchased?
Can I also send the list of all symbols and their corresponding number of shares to an email address, or is it possible to attach a text file here?
Forum Tip!
Click the Code button and paste your code between the tags.
Click the Code button and paste your code between the tags.
Check if the separators in your txt file are matching the set in this line of code:
Perhaps you need to include a space, I'm not sure.
Also, check that the dataset that you are using includes all of the symbols listed in the .txt file.
I ran your latest code with a minor modification so that a space is included in the Split call. And, the dataset I used consisted of symbols that match the symbols in the .txt file. For ten years of daily data, many trades occurred in the backtest.
CODE:
string[] parts = line.Split(new char[] { ';', ',', '\t' }, StringSplitOptions.RemoveEmptyEntries);
Perhaps you need to include a space, I'm not sure.
Also, check that the dataset that you are using includes all of the symbols listed in the .txt file.
I ran your latest code with a minor modification so that a space is included in the Split call. And, the dataset I used consisted of symbols that match the symbols in the .txt file. For ten years of daily data, many trades occurred in the backtest.
Why are you trying to determine market capitalization from a text file? I would subscribe to a WL fundamental data provider and get it from there.
There are free and there are paid fundamental data providers. The free ones are slow (and I disable them when updating many symbols for the day). The paid are are very fast, so you can leave those enabled.
YCharts has both free and paid options. Install the WL Fundamental extension, enable (checkbox) YCharts, and be sure to check that you want "shares_outstanding"; you can calculated market capitalization from that. Don't check more fundamentals than you need because things will get very slow.
I'm not suggesting YCharts is the best fundamental data provider. You should read the WL blog article that compares the different data providers first. For paid fundamental providers, EODHD is very good for $60/month; there are cheaper ones.
There are free and there are paid fundamental data providers. The free ones are slow (and I disable them when updating many symbols for the day). The paid are are very fast, so you can leave those enabled.
YCharts has both free and paid options. Install the WL Fundamental extension, enable (checkbox) YCharts, and be sure to check that you want "shares_outstanding"; you can calculated market capitalization from that. Don't check more fundamentals than you need because things will get very slow.
I'm not suggesting YCharts is the best fundamental data provider. You should read the WL blog article that compares the different data providers first. For paid fundamental providers, EODHD is very good for $60/month; there are cheaper ones.
@Paul1986
Thank you, it's working now after your change.
Unfortunately, I made a fundamental error in my thinking. I only created my list using the number of shares currently outstanding. This means that splits and share buybacks aren't included. For example, the number of shares in Amazon today is completely different from what it was in 2015. Therefore, this list is only of limited use to me.
I've also written another code snippet where the market cap is pulled directly from WL. The code compiles, but unfortunately, it only buys once at time X (the start of the backtest). The positions remain open indefinitely; no symbol is changed. Do you have an explanation for this?
Thank you, it's working now after your change.
Unfortunately, I made a fundamental error in my thinking. I only created my list using the number of shares currently outstanding. This means that splits and share buybacks aren't included. For example, the number of shares in Amazon today is completely different from what it was in 2015. Therefore, this list is only of limited use to me.
I've also written another code snippet where the market cap is pulled directly from WL. The code compiles, but unfortunately, it only buys once at time X (the start of the backtest). The positions remain open indefinitely; no symbol is changed. Do you have an explanation for this?
CODE:
using WealthLab.Backtest; using System; using WealthLab.Core; using System.Collections.Generic; using WealthLab.Indicators; namespace WealthScript8 { public class MarketCapRotation : UserStrategyBase { public override void Execute(BarHistory bars, int idx) { // 1. Marktkapitalisierung als Serie berechnen (für das Plotten wichtig) TimeSeries sharesSeries = Fundamental.Series(bars, "shares_outstanding", true); TimeSeries mCapSeries = bars.Close * sharesSeries; // Fallback-Logik: Falls MarketCap 0 ist, nutzen wir Volumen if (Double.IsNaN(mCapSeries[idx]) || mCapSeries[idx] <= 0) { // Wir füllen den Wert manuell, falls die Fundamentaldaten fehlen mCapSeries[idx] = bars.Close[idx] * bars.Volume[idx]; } // 2. Visualisierung (Syntax für WL8: TimeSeries, Name, Farbe, Pane) // Wir plotten nur einmal am Ende der Historie oder laufend PlotTimeSeries(mCapSeries, "MarketCap", "MCapPane", WLColor.Blue); double currentMCap = mCapSeries[idx]; if (Double.IsNaN(currentMCap) || currentMCap <= 0) currentMCap = 0.001; Position pos = LastPosition; // Handelslogik if (pos == null || !pos.IsOpen) { Transaction t = PlaceTrade(bars, TransactionType.Buy, OrderType.Market, 0, "Entry"); t.Weight = currentMCap; } else { // 1% Hurdle zur Stabilisierung Transaction t = PlaceTrade(bars, TransactionType.Buy, OrderType.Market, 0, "Hold"); t.Weight = currentMCap * 1.01; } } } }
@superticker
As far as I know, Y-Charts doesn't offer a permanently free account, so what do you mean by a free option?
As far as I know, Y-Charts doesn't offer a permanently free account, so what do you mean by a free option?
My father speaks German, but I don't.
YCharts offers both a free and paid option. Download the WL Fundamental extension and leave off the API key because that extension scrapes the YCharts website without the API key to get the data. That's why the "free option" is very slow, but it's okay for one or two fundamental items. Yes, the paid option--with an API key--is much faster and reliable because it employs a REST endpoint.
When we talk about a "WL data provider" are talking about a WL code extension that employs the data broker's API to bring its data into WL via its interface. Therefore, if a WL data provider extension is available, then you simply download that code extension and you're done. For YCharts, you need to download the WL Fundamental Extension. For Tiingo, you need to download the WL Data Extensions extension.
Now if there is not a WL data provider extension for the datafeed you want, then yes you'll need to develop your own WL provider extension and you'll need the data broker's API to do that with. It's good to see you're a programmer. You'll like the WL framework. It's very powerful.
YCharts offers both a free and paid option. Download the WL Fundamental extension and leave off the API key because that extension scrapes the YCharts website without the API key to get the data. That's why the "free option" is very slow, but it's okay for one or two fundamental items. Yes, the paid option--with an API key--is much faster and reliable because it employs a REST endpoint.
When we talk about a "WL data provider" are talking about a WL code extension that employs the data broker's API to bring its data into WL via its interface. Therefore, if a WL data provider extension is available, then you simply download that code extension and you're done. For YCharts, you need to download the WL Fundamental Extension. For Tiingo, you need to download the WL Data Extensions extension.
Now if there is not a WL data provider extension for the datafeed you want, then yes you'll need to develop your own WL provider extension and you'll need the data broker's API to do that with. It's good to see you're a programmer. You'll like the WL framework. It's very powerful.
I have installed the "Fundamental" extension, but I still can't get a usable result.
Well, reading is "fundamental" (like the details in your post) and apparently today I don't have reading fundamentals right now. :)
I'm running your code now, I'll let you know what I find.
In your else clause, do you have the correct TransactionType? Its not clear to me what you're trying to achieve.
Also, you're requerying the fundamentals on every bar in Execute(). That's not good. Move that code to the Initialize method. Here's the updated code (but with my namespace):
Also, you're requerying the fundamentals on every bar in Execute(). That's not good. Move that code to the Initialize method. Here's the updated code (but with my namespace):
CODE:
using WealthLab.Backtest; using WealthLab.Core; using WealthLab.Indicators; namespace WealthLabStrategies.PriceAction { public class MarketCapRotation : UserStrategyBase { private TimeSeries _mCapSeries; public override void Initialize(BarHistory bars) { // 1. Marktkapitalisierung als Serie berechnen (für das Plotten wichtig) var sharesSeries = Fundamental.Series(bars, "shares_outstanding", true); _mCapSeries = bars.Close * sharesSeries; } public override void Execute(BarHistory bars, int idx) { // Fallback-Logik: Falls MarketCap 0 ist, nutzen wir Volumen if (double.IsNaN(_mCapSeries[idx]) || _mCapSeries[idx] <= 0) { // Wir füllen den Wert manuell, falls die Fundamentaldaten fehlen _mCapSeries[idx] = bars.Close[idx] * bars.Volume[idx]; } // 2. Visualisierung (Syntax für WL8: TimeSeries, Name, Farbe, Pane) // Wir plotten nur einmal am Ende der Historie oder laufend PlotTimeSeries(_mCapSeries, "MarketCap", "MCapPane", WLColor.Blue); var currentMCap = _mCapSeries[idx]; if (double.IsNaN(currentMCap) || currentMCap <= 0) { currentMCap = 0.001; } var pos = LastPosition; // Handelslogik if (pos is not {IsOpen: true}) { var t = PlaceTrade(bars, TransactionType.Buy, OrderType.Market, 0, "Entry"); t.Weight = currentMCap; } else { // 1% Hurdle zur Stabilisierung var t = PlaceTrade(bars, TransactionType.Buy, OrderType.Market, 0, "Hold"); t.Weight = currentMCap * 1.01; } } } }
QUOTE:
I have installed the "Fundamental" extension, but I still can't get a usable result.
And can you tell us why that is? Where are you stuck?
You also need to go into the indicators list and drag the Fundamental indicator into your Chart window. Right click and choose convert-to-code to see the C# result. Now you should have Shares Outstanding, so write some code in the Initialize{block} to convert that to market capitalization.
UPDATE: Looks like Post #11 has already done that for you.
You need to read the Fundamental section in the Help docs.
@superticker

I've now tried a rotation strategy, and as you can see from the image, it can't be mathematically wrong. The closing price is multiplied by the fundamental (shares outstanding), and the result is the market capitalization.
Now, here's what happens: when I apply the strategy to the Dow 30 starting January 1, 2022, everything works fine. However, if I start the strategy on January 2, 2015, the system doesn't even begin buying until February 2021. If I apply the strategy to a different dataset like the S&P 500 or Nasdaq 100, I get no results at all, regardless of the starting date.
This is all very strange. What could be going wrong?
I've now tried a rotation strategy, and as you can see from the image, it can't be mathematically wrong. The closing price is multiplied by the fundamental (shares outstanding), and the result is the market capitalization.
Now, here's what happens: when I apply the strategy to the Dow 30 starting January 1, 2022, everything works fine. However, if I start the strategy on January 2, 2015, the system doesn't even begin buying until February 2021. If I apply the strategy to a different dataset like the S&P 500 or Nasdaq 100, I get no results at all, regardless of the starting date.
This is all very strange. What could be going wrong?
@paul1986
So, when I apply your last code to the Dow30, I get the following result: see image
Backtest started in 2010, but I only bought it in 2020.
I'm starting to get desperate, whatever.
So, when I apply your last code to the Dow30, I get the following result: see image
Backtest started in 2010, but I only bought it in 2020.
I'm starting to get desperate, whatever.
@paul1986
QUOTE:
However, if I start the strategy on January 2, 2015, the system doesn't even begin buying until February 2021.
And why is that? You tell us.
I'm guessing the "free" YCharts provider won't get shares_outstanding that far back. Can you confirm? Sounds like you need a paid fundamental provider for this. Read over the WL blog post about fundamental providers and pick a paid provider. https://www.wealth-lab.com/blog/wealthlab-data-providers You can use a non-WL datafeed provider, but you'll need to write a WL provider extension to interface with their API endpoint. You would be writing a WL "Event Provider" extension for a fundamental datafeed.
I think your WL strategy code must be correct; otherwise, it wouldn't work at all.
@Superticker
It will be exactly as you say; Y-Charts probably only goes back a maximum of 5 years. Therefore, I've now solved it with my own list by incorporating the splits for the 30 largest stocks so that the market cap is calculated accurately.
Thank you for your support.
It will be exactly as you say; Y-Charts probably only goes back a maximum of 5 years. Therefore, I've now solved it with my own list by incorporating the splits for the 30 largest stocks so that the market cap is calculated accurately.
Thank you for your support.
Your Response
Post
Edit Post
Login is required