I'm simply trying to scrape a webpage with HttpClient. Please tell me why this simple code doesn't work?
I would prefer to do this with a "static" httpClient instance, but I would also like to pass parameters back from the scraping task. If this requires a non-static implementation (Does it?), then so be it.
I would prefer to do this with a "static" httpClient instance, but I would also like to pass parameters back from the scraping task. If this requires a non-static implementation (Does it?), then so be it.
CODE:
using System; using System.Net.Http; using System.Threading.Tasks; using WealthLab.Backtest; using WealthLab.Core; namespace WealthScript3 { public class MyStrategy : UserStrategyBase { static readonly HttpClient httpClient = new HttpClient(); private static async void DownloadPage(UserStrategyBase usb, string symbol) //private static async Task<string> DownloadPage(UserStrategyBase usb, string symbol) { string responseString = null; try { HttpResponseMessage response = new HttpResponseMessage(); response = await httpClient.GetAsync("<a href="https://digital.fidelity.com/prgw/digital/research/quote/dashboard/summary?symbol=" target="_blank">https://digital.fidelity.com/prgw/digital/research/quote/dashboard/summary?symbol=</a>" + symbol); //response.EnsureSuccessStatusCode(); responseString = await response.Content.ReadAsStringAsync(); } catch (Exception ex) { usb.WriteToDebugLog(ex.Message.ToString()); } usb.WriteToDebugLog(responseString); usb.WriteToDebugLog("hello world"); //doesn't work //return responseString; } public override void Initialize(BarHistory bars) { //bool isMutualFund; //Task.Run(new Action(DownloadPage)); Task.Run(() => DownloadPage(this as UserStrategyBase, bars.Symbol)); } public override void Execute(BarHistory bars, int idx) { } } }
Rename
You’re calling the download method in a new Task, and then immediately exiting the method. So the Task isn’t even completed before the main code line hits your WriteToDebug method.
CODE:
using System; using System.Net.Http; using System.Threading.Tasks; using WealthLab.Backtest; using WealthLab.Core; namespace WealthScript3 { public class MyStrategy : UserStrategyBase { static readonly HttpClient httpClient = new HttpClient(); private string DownloadPage(UserStrategyBase usb, string symbol) { string uri = "https://" + $"digital.fidelity.com/prgw/digital/research/quote/dashboard/summary?symbol=" + symbol; string responseString = null; try { HttpResponseMessage response = new HttpResponseMessage(); response = httpClient.GetAsync(uri).Result; responseString = response.Content.ReadAsStringAsync().Result; } catch (Exception ex) { usb.WriteToDebugLog(ex.Message.ToString()); } return responseString; } public override void Initialize(BarHistory bars) { WriteToDebugLog( DownloadPage(this as UserStrategyBase, bars.Symbol) ); } public override void Execute(BarHistory bars, int idx) { } } }
The forum automatically recognizes URLs and wraps them into "a href", including CODE blocks.
The forum website shouldn't adulterate CODE blocks under any circumstance. It simply breaks the code.
It's too troublesome to fix that. Here's a workaround that would be way easier (amended my code above):
CODE:
string uri = "https://" + $"digital.fidelity.com/prgw/digital/research/quote/dashboard/summary?symbol=" + symbol;
Your Response
Post
Edit Post
Login is required