Below is the log for streaming data on my machine:
First message says data farm connection OK to Interactive Brokers.
Alas I cannot get data for Australian securities. "No security definition has been found for A1N.au"
I used the data streaming testbed in the image below to try to find the correct mappings. Tried *.au and *.ASX.
I also found the symbol mappings dialog which is also in the image below.
This is almost certainly a symbol mapping issue, but I cannot find the correct mappings. Googling not coming up with relevant results.
First message says data farm connection OK to Interactive Brokers.
Alas I cannot get data for Australian securities. "No security definition has been found for A1N.au"
I used the data streaming testbed in the image below to try to find the correct mappings. Tried *.au and *.ASX.
I also found the symbol mappings dialog which is also in the image below.
This is almost certainly a symbol mapping issue, but I cannot find the correct mappings. Googling not coming up with relevant results.
Rename
A few more diagnostics to report.
1. Tried *.ax also same as yahoo.
2. Tried upper and lower case, but symbols forced to uppercase
3. Tried streaming in chart with same result (was not expecting different really).
4. IB in on Australian website has tickers with no extension. Tried that, but get not subscribed error, most likely since SMART routing to US exchange, to which I am not subscribed. Only subscribed to ASX at this time.
1. Tried *.ax also same as yahoo.
2. Tried upper and lower case, but symbols forced to uppercase
3. Tried streaming in chart with same result (was not expecting different really).
4. IB in on Australian website has tickers with no extension. Tried that, but get not subscribed error, most likely since SMART routing to US exchange, to which I am not subscribed. Only subscribed to ASX at this time.
What symbol mappings have you tried?
*.ASX, *.AX, *.AU
Using the Streaming data test bed.
Using the Streaming data test bed.
These are just suffixes. What I'm asking for is how exactly you defined the symbol mapping patterns in your case?
No symbol mapping patterns used yet.
I am typing the security code and suffix into the streaming data testbed part of the data manager tool in Wealthlab 8 and looking at the results in the Log Viewer tool. The log viewer tool is an extremely useful tool.
Once I get the format correct, I will enter regex into symbol mapping.
Some more information. I have also tried the following since I found a weak lead in the IB API help:
I have also found that after a 2-5 unsuccessful tries, the API must reject requests from that user for a while.
I am typing the security code and suffix into the streaming data testbed part of the data manager tool in Wealthlab 8 and looking at the results in the Log Viewer tool. The log viewer tool is an extremely useful tool.
Once I get the format correct, I will enter regex into symbol mapping.
Some more information. I have also tried the following since I found a weak lead in the IB API help:
I have also found that after a 2-5 unsuccessful tries, the API must reject requests from that user for a while.
Symbol mappings have nothing to do with defining the contract at IB.
Please see the Help (F1) > Interactive Brokers for how to do it.
Please see the Help (F1) > Interactive Brokers for how to do it.
Got it working. Thanks Cone. I read the IB extension help file to connect to IB and did not read to the bottom where it defines the required additions to the IBContracts.txt file.
Have now!
For new players:
1. Read ALL the Interactive Brokers Extension help file!
2. Check out this thread mentioned in the help file: https://www.wealth-lab.com/Discussion/9074
3. Add lines like this for all stocks to the file IBContracts.txt file (location is mentioned in the help).
A2M.AU=A2M|AUD|STK|SMART|ASX
29M.AU=29M|AUD|STK|SMART|ASX
...
The ASX may not be required since the AUD currency is defined.
In my case I am using Norgate data, so *.AU is required to map the Norgate data used in my strategies to the IB data being streamed and the orders being placed with IB.
4. Cone has code in the help and at the post referenced above that can create the required lines for IBContracts.txt. In my case I used text editor with column editing mode to cut and paste the data out of the bottom left datasets window and create the required additions. Worked for me because there will be no duplicate symbols as the Norgate data is distinct because of the *.AU suffix.
5. The streaming data testbed in the data manager and Log Viewer are useful debugging tools when going through this process.
Have now!
For new players:
1. Read ALL the Interactive Brokers Extension help file!
2. Check out this thread mentioned in the help file: https://www.wealth-lab.com/Discussion/9074
3. Add lines like this for all stocks to the file IBContracts.txt file (location is mentioned in the help).
A2M.AU=A2M|AUD|STK|SMART|ASX
29M.AU=29M|AUD|STK|SMART|ASX
...
The ASX may not be required since the AUD currency is defined.
In my case I am using Norgate data, so *.AU is required to map the Norgate data used in my strategies to the IB data being streamed and the orders being placed with IB.
4. Cone has code in the help and at the post referenced above that can create the required lines for IBContracts.txt. In my case I used text editor with column editing mode to cut and paste the data out of the bottom left datasets window and create the required additions. Worked for me because there will be no duplicate symbols as the Norgate data is distinct because of the *.AU suffix.
5. The streaming data testbed in the data manager and Log Viewer are useful debugging tools when going through this process.
Hmmm. I have called this "working" prematurely.
Using this IBContracts.txt file in the location C:\Users\[my_name]\AppData\Roaming\WealthLab8\
*only a small part of the file shown above.
Streaming from IB works as per the streaming data testbed below:
However quotes and price triggers is unable to get streaming data and the error appears in the log viewer below:
* I should have used the same A2M above instead of EGG, but same result
I am out of ideas for how to further troubleshoot this since I have it working in the streaming data testbed. Assistance appreciated. Any Aussies out there using IB for the ASX and Norgate data?
Using this IBContracts.txt file in the location C:\Users\[my_name]\AppData\Roaming\WealthLab8\
*only a small part of the file shown above.
Streaming from IB works as per the streaming data testbed below:
However quotes and price triggers is unable to get streaming data and the error appears in the log viewer below:
* I should have used the same A2M above instead of EGG, but same result
I am out of ideas for how to further troubleshoot this since I have it working in the streaming data testbed. Assistance appreciated. Any Aussies out there using IB for the ASX and Norgate data?
I came back from lunch and this data had streamed from IB:
Troubleshooting this is difficult. I am going to take a break for the rest of the day. Tomorrow I will read all the help again looking for clues. Tonight I will watch some more videos on autotrading from WealthLab to make sure I have the procedure correct.
Not sure if related: but errors and exceptions below:
System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App\6.0.4'.
at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
at System.IO.Enumeration.FileSystemEnumerator`1.Init()
at System.IO.Enumeration.FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options)
at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
at System.IO.Directory.GetFiles(String path)
at WealthLab8.cwAssemblyReferences.DisableWriter(String reference)
at WealthLab8.cwAssemblyReferences.Populate()
at WealthLab8.cwAssemblyReferences..ctor()
at WealthLab8.MainWindow.ShowAssemblyReferences()
at WealthLab8.MainWindow.SelectRequest(Object item, RoutedEventArgs attr)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
at System.Windows.Controls.MenuItem.InvokeClickAfterRender(Object arg)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
and..
Troubleshooting this is difficult. I am going to take a break for the rest of the day. Tomorrow I will read all the help again looking for clues. Tonight I will watch some more videos on autotrading from WealthLab to make sure I have the procedure correct.
Not sure if related: but errors and exceptions below:
System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App\6.0.4'.
at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
at System.IO.Enumeration.FileSystemEnumerator`1.Init()
at System.IO.Enumeration.FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options)
at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
at System.IO.Directory.GetFiles(String path)
at WealthLab8.cwAssemblyReferences.DisableWriter(String reference)
at WealthLab8.cwAssemblyReferences.Populate()
at WealthLab8.cwAssemblyReferences..ctor()
at WealthLab8.MainWindow.ShowAssemblyReferences()
at WealthLab8.MainWindow.SelectRequest(Object item, RoutedEventArgs attr)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
at System.Windows.Controls.MenuItem.InvokeClickAfterRender(Object arg)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)
and..
The last error especially leads me to believe that your .NET installation is somehow corrupted.
QUOTE:
I came back from lunch and this data had streamed from IB:
When you turn on Streaming, nothing will be displayed until the symbol trades/ticks.
Personally, I'd like to see the last trade displayed as soon as Stream is turned on, but there's also a case to not change the way it has always worked in WealthLab.
Back at my desktop installation of Wealth-lab which went swimmingly.
I have read the IB extension help file for a third time. I noticed that the help says:
○ IB Provider is compatible with TWS 10.23 and higher.
Alas the TWS version I had installed was 10.19 (Stable). I have now updated my TWS to the latest which is 10.26. Have done this on both my machines.
As Cone indicated, the streaming only works during market hours, so back to testing and troubleshooting this issue tomorrow with the correct version of TWS, a correct IBContracts.txt file and clean Wealth-lab/.Net installation. Fingers crossed.
I have read the IB extension help file for a third time. I noticed that the help says:
○ IB Provider is compatible with TWS 10.23 and higher.
Alas the TWS version I had installed was 10.19 (Stable). I have now updated my TWS to the latest which is 10.26. Have done this on both my machines.
As Cone indicated, the streaming only works during market hours, so back to testing and troubleshooting this issue tomorrow with the correct version of TWS, a correct IBContracts.txt file and clean Wealth-lab/.Net installation. Fingers crossed.
Streaming will work any time the symbol is trading.
If outside of market hours, uncheck the "Filter Pre/Post" on the tool you're using, usually found in the lower status bar.
And, if you want to trade outside of market hours, enable that option in Preferences (F12) > Trading > Misc
If outside of market hours, uncheck the "Filter Pre/Post" on the tool you're using, usually found in the lower status bar.
And, if you want to trade outside of market hours, enable that option in Preferences (F12) > Trading > Misc
I have now done the following two tests. I will complete the third test in 9 hours time when the market is open.
Test environment:
- Windows 10 Pro with Wealth-Lab 8 build 68. TWS 10.26. No issues with installation or .Net.
- IB streaming data subscription to ASX (AU).
- Using Norgate data for ASX (AU) which has suffix .au. eg. BLY.AU
- IBContracts.txt file set up correctly as per the IB extension help file and other posts.
Test 1
1. use the Data Manager->Streaming Providers->Streaming data testbed to get a result for BLY.AU. Result is correct and expected behaviour. No errors shown in Log Viewer.
2. Shut down WL8. Remove the line BLY.AU=BLY|AUD|STK|SMART|ASX from IBContracts.txt. Restart WL8.
3. Attempt 1 again and note that error occurs and contract cannot be found.
Result: This shows that the IBContracts.txt file is working as expected for streaming from IB in the streaming data testbed.
Test 2
1. Restore BLY.AU=BLY|AUD|STK|SMART|ASX line to IBContracts.txt while WL8 is shut down.
2. Confirm this line is in the IBContracts.txt file when WL8 is restarted
3. Set up the out of trading hours test in the image below. Enable trade outside market hours and uncheck the filter pre/post option.
4. Use the streaming data testbed to confirm streaming from IB is working. Turn off the streaming (press the button again) just in case this somehow affects streaming in the Quotes and price triggers tool.
5. Turn on IB streaming in the quotes and price triggers tool.
Result: The following error appears in the log viewer:
"10/01/2024 00:11:21:845" "InteractiveBrokers" "Could not identify a contract for BLY.au"
Test environment:
- Windows 10 Pro with Wealth-Lab 8 build 68. TWS 10.26. No issues with installation or .Net.
- IB streaming data subscription to ASX (AU).
- Using Norgate data for ASX (AU) which has suffix .au. eg. BLY.AU
- IBContracts.txt file set up correctly as per the IB extension help file and other posts.
Test 1
1. use the Data Manager->Streaming Providers->Streaming data testbed to get a result for BLY.AU. Result is correct and expected behaviour. No errors shown in Log Viewer.
2. Shut down WL8. Remove the line BLY.AU=BLY|AUD|STK|SMART|ASX from IBContracts.txt. Restart WL8.
3. Attempt 1 again and note that error occurs and contract cannot be found.
Result: This shows that the IBContracts.txt file is working as expected for streaming from IB in the streaming data testbed.
Test 2
1. Restore BLY.AU=BLY|AUD|STK|SMART|ASX line to IBContracts.txt while WL8 is shut down.
2. Confirm this line is in the IBContracts.txt file when WL8 is restarted
3. Set up the out of trading hours test in the image below. Enable trade outside market hours and uncheck the filter pre/post option.
4. Use the streaming data testbed to confirm streaming from IB is working. Turn off the streaming (press the button again) just in case this somehow affects streaming in the Quotes and price triggers tool.
5. Turn on IB streaming in the quotes and price triggers tool.
Result: The following error appears in the log viewer:
"10/01/2024 00:11:21:845" "InteractiveBrokers" "Could not identify a contract for BLY.au"
Symbols can be case sensitive.
If you're going to use BLY.au, you should put BLY.au=BLY etc
You can see that you got data using BLY.AU in the Streaming testbed, so the case is the issue.
Also, I neglected to add this line to IBDefaultMarkets.txt, so you should add it.
AUD=Australian Securities Exchange
Just in case - it might not be obvious that you have to hit the Stream button to start the Quotes tool.
If you're going to use BLY.au, you should put BLY.au=BLY etc
You can see that you got data using BLY.AU in the Streaming testbed, so the case is the issue.
Also, I neglected to add this line to IBDefaultMarkets.txt, so you should add it.
AUD=Australian Securities Exchange
Just in case - it might not be obvious that you have to hit the Stream button to start the Quotes tool.
The screen shot above has the streaming off as you note. However it was on for the tests.
I did think about this being a case issue. The streaming data test bed converts all keyboard input into uppercase. And that works. So you could be right.
The quotes and price windows takes signals from the backtest. These come in as a lower case suffix. User has no control over the case (that I am aware of), as it comes from Norgate data drag and dropping etc.
At it simplest, this is a case issue. At worst, the quotes and price triggers tool is not using the conversions in the IBContracts.txt file like the Streaming data testbed is.
I did think about this being a case issue. The streaming data test bed converts all keyboard input into uppercase. And that works. So you could be right.
The quotes and price windows takes signals from the backtest. These come in as a lower case suffix. User has no control over the case (that I am aware of), as it comes from Norgate data drag and dropping etc.
At it simplest, this is a case issue. At worst, the quotes and price triggers tool is not using the conversions in the IBContracts.txt file like the Streaming data testbed is.
I added the line to the IBDefaultMarkets.txt file and converted all suffixes (.au) in IBContracts.txt to lowercase as you suggested.
The quotes and price triggers page works as expected now, but the streaming data testbed does not as it converts all keyboard input to uppercase. No biggie that does not work, but maybe the uppercase force can be removed from that text box.
So this seems resolved, but 2am here and will I have a good look at it tomorrow.
Thanks for your help Cone.
The quotes and price triggers page works as expected now, but the streaming data testbed does not as it converts all keyboard input to uppercase. No biggie that does not work, but maybe the uppercase force can be removed from that text box.
So this seems resolved, but 2am here and will I have a good look at it tomorrow.
Thanks for your help Cone.
We'll have to give that Streaming Testbed input box the same reverse-shift treatment that other tools have. Thanks for pointing it out.
Working now. Market in Oz is open and Streaming from IB functioning within normal parameters. Topic closed.
Your Response
Post
Edit Post
Login is required