Occasional-difficulties-getting-order-status-from-IB

You said a lot of things there and I'm not sure of the sequence or even what build you were using when you saw this.

Are you seeing these exceptions now, with IB Provider Build 7?

Yes

@mb5413 Did you use streaming bars?

@MIH
I used 1 min streaming bars.

For me it looks like the issue exists longer than yesterday. Unfortunately, I can't prove that.
At least i would look closer to the streaming bar topic. I used a 1 minute strategy too.
The data load maybe an issue. (intuition, no facts)

https://www.wealth-lab.com/Discussion/IB-Placed-Orders-do-not-apper-at-broker-8163

I'm not seeing this problem here. How many symbols are you running?

i am not sure who you asking now. I used 80.

thanks @MIH

@Cone - I am using 1 symbol. in below pic - you can see the current state of 3 buy at market orders. 2 with IB (CancelPending) and 1 with TDA (Filled)



The 2 IB orders have been executed by IB, however appear not to have been confirmed as filled (this could be the issue).

What happens next is the Sell order will remain as "Placed" because of the CancelPending state for the IB transactions. The Sell order will not reach IB, and the 2 IB positions will remain open

Since you're seeing the same problem, it's good for both to answer.

I'll bump my symbol count up to 80. If you'd share the symbols in your DataSet it could help too. I usually ignore illiquid symbols, but those are the ones that tend to cause issues.

here is the pic following the sell order by the strategy. IB has not executed the sale

Use MSFT

here are the preference settings, in case of relevance

While I can see there are issues out there, what makes it difficult is that it works perfectly here -



If the issue persists, the only thing I can suggest is to restart WL8 to refresh the connection with TWS. It's a good idea to start this way each day as well.

ACGL AIMC AMR AOS APA APOG ARMK ATC AVNS AVY AXL BKNG BTU BV CADE CARS CBT CEIX CEVA CNA CURO CWEN DAL DDOG DEN DNOW EEFT ENV EXPE FA FBHS FCF FIS FOCS GPI GSHD GTN HAE HES HWC ICLR IFF JBHT KFRC LESL LPI MLAB MPAA MSA MTN MTOR MTW NABL NCR NFE NOW NTR NXGN OFG OXY PCG PFGC PTSI PTVE RDWR ROCC SGFY SHLS SOI SOVO SWCH TEN THRN TJX UEIC USFD VAC WDAY WNC ZBH

That was the dataset i used for testing in the paper trading account. I am not sure if there are significant differences with PT and Live Trading. Just to let you know.

have just restarted without any difference - same issue again

The issue appears to be IB is not confirming Filled orders (instantly) then order manager makes them CancelPending and the rest is documented above

I have shared my preference settings - anything there that might clash ?

Maybe there's something in the TWS API Settings that's causing it, like having specified a "Master API client ID" that isn't the same as the clientID you're connected with. Not saying that's it, but if you could post a picture of your TWS API Settings page, I'll look for something like that.

Something just occurred to me - what version of TWS are you running? I would definitely expect to see these problems if you were not using at least TWS 10.12, but you should already be on 10.15.

The proper configuration is to use the latest stable version of TWS and it's probably best NOT to install the TWS API. We distribute the API built in a dll.

Hello Cone, i use the following version with default installation.



I installed the TWS API seperately for dev purposes in a freely choosen folder. Are you talking of the CSharpAPI.dll?
Can you explain in more detail how this can become a problem?

@Cone - I am using TWS Build 10.16 of June 13 2022

Could it be 10.16 has some relation with the problem? As said in an earlier response, on Monday (June 13) everything was still working well. I suspect 10.16 may have downloaded (automatically) on June 14

Let me see if I can find a prior version of TWS to install and test today

learning something new: you can downgrade version from within TWS, in "more settings"



so now I moved from LATEST version to STABLE TWS Build 981

No need to downgrade. TWS 10.16 is fine too. I don't see anything in the release notes that affect the API - unless you checked the "Reject messages above maximum allowed... " API setting. Though if you're only working with one symbol (or even dozens), this is unlikely to be a factor.

QUOTE:
so now I moved from LATEST version to STABLE TWS Build 981

Definitely NOT!

10.12 is the minimum version that is compatible with Wealth-Lab.

It's probably better if use the offline TWS. Start up is faster since you don't have to download it every time you restart. Go to this page -
https://www.interactivebrokers.com/en/index.php?f=14099#tws-software

Currently the latest is Version 10.16.1i and the "time tested" version is 10.12.2m. Either of these are compatible with Wealth-Lab 8 and the current IB Provider (Build 7 - https://www.wealth-lab.com/extension/detail/InteractiveBrokers).

Very frustrating; TWS v10.12 makes no difference, IB does not report filled orders and creates issues in order manager with CancelPending orders !!!

I also tried downgrading to IB B6 extension, no difference either

The issue is IB and order manager are just not functioning well anymore and it doesnt appear to be a TWS issue, nor the latest IB extension B7.

Appreciate any further help . Giving up for now (?!?!)

@Cone you are using a paper account in your pics, perhaps you could replicate the error using real account trading(?)

I only started using Paper trading for heavy testing recently, and I'm usually connected to Live accounts.

Previously asked If you could post a picture of your TWS API Settings page, maybe something we're missing is there.

API settings

Hi.

You can enable "API message log file" and "Include market data in API log file". If it doesn't help, it doen't hurt either.

There's a section for "When closing positions:"... you're trading a Financial Advisor account? (Or maybe it's just that I haven't used that option so it's not displayed for me.)

If it's an FA, that's a pretty big difference. Since we haven't made any effort to be compatible with FA accounts I don't know why it would matter, but this could be it.

how comes this FA (family linked account) was working well all the way through Monday? what has IB changed that no longer works in WL8?

QUOTE:
how comes this FA (family linked account) was working well all the way through Monday?

I don't know why it worked, and I don't know why it doesn't work. We have no way to test that configuration but have done all we can to support connections to multiple accounts - which could be the mistake, because only one can be used in a session.

Are you trying to use multiple IB accounts in the same WealthLab session?

I use 2 IB accounts and 1 TDA - are you saying there is a limitation to just 1 account for auto-trading?

QUOTE:
I don't know why it worked, and I don't know why it doesn't work. We have no way to test that configuration but have done all we can to support connections to multiple accounts - which could be the mistake

I am running the strategy with just 1 IB account and 1 TDA, having the same issue with IB CancelPending because the order Fill is not confirmed. To confirm; the TDA interface is working well with the same strategy

Thanks for all you have done @Cone - I don't know what else to check and test.

Can anyone else confirm they are using auto-trade successfully with IB (trading account) intraday- thanks

You can auto-trade with multiple accounts. However, and at least for now, you can't do that with multiple IB accounts connected to one instance of TWS with Wealth-Lab.

The FA/Family configuration is an unknown. Just log in to TWS with a single account and try that.

Unfortunately, I cannot confirm this. At least in small time frames and with many signals there are (for me) regular problems. It is important for me to say, that it doesn't necessarily mean something is wrong with WL, it might be an issue with IB.

At the moment I only use the paper trading. Currently, a use for live trading is out of the question for me. It is not stable enough.

Here is an example from today.

The strategy i am running today has SL and TP as default.
For unknown reasons three symbols did not get the SL and TP. "FIS" as example.
At 16:11 the order was placed. at 16:58 the SL and TP was added !!! (CBT still doesn't has a SL+TP). The strategy is running. Other symbols are traded "normally".







Now, i don't know how the problem will be traced back. The way i would do it.

1. ignore the trading aspects and run a 1 minute strategy that generates many, many signals.
Example: 50-100 symbols, Knife Juggler (maybe with an even lower entry condition 0.25 and SL,TP rules for OCA orders).
2. activate all available logs including the TWS log.
3. it will take no more than 30-60 minutes to create a problem that can then be encapsulated and investigated independently.

However, one impression is created, namely that the signals are not transmitted stably. Today I can say in any case that it is not a connection problem,because the other symbols are communicated normally.

Do me a favor. Open your Strategy > Settings > Advanced Settings.
Is "Retain NSF Position" checked?

Yes, the strategy that is running today has it checked.

With so many symbols (and not knowing anything else) it's likely you'll run the strategy backtest into a NSF situation. If "Retain" is checked - and was checked before adding the Strategy to the S. Monitor - there shouldn't be a problem with signaling.

I ran 60 symbols for the last 3 hours, had about 50 positions at one point and even got the warning below several times since I pushed the paper account over about an 8:1 margin. Yet everything worked *almost* perfectly.

6/17/2022 12:53:27:380
InteractiveBrokers
2148: ALERT: Your account, while currently margin compliant, maintains qualifying equity (i.e., Equity with Loan Value) at a level only 10% above that which is required. Please note that we do not issue margin calls ...

*Almost* - only because I noticed that our orphan tracking mechanism on a few occasions didn't track live, auto-traded positions due to partial fills, which is actually by design, though I now think it should be changed. Nonetheless, this didn't affect anything though and only may have affected something if the account got out of synch with the strategy with "Exit Orphans" checked.

If you want to, write zoom@weatlh-lab.com and we could meet next week to see this in action on your machine.

Thanks for the effort and the suggestion to talk via zoom.

Today I had another three symbols that are open and for which at some point no more signals came. Two of them were there only after I restarted WL on purpose.

If I have understood you correctly, "Retain" should be switched on and "Exit Orphan" should also be switched on. Both were active today. With the mentioned restart I switched off "Retain" to find out differences, but I could not.

In the afternoon I also opened the TWS log. But after it was opened, it was not continued. I also had difficulty interpreting it correctly. Next time i will open a copy in the trading hours. But I have the minutes before 16:58, so if there is something to be found for "FIS" or a reference for "FIS" is missing, maybe someone can find information about it. If you know what to look for in the protocol.

QUOTE:
Today I had another three symbols that are open and for which at some point no more signals came.

I hear you, but this lacks context. This is always a possibility in a limit system and one reason to use the Exit orphan logic.

QUOTE:
If I have understood you correctly, "Retain" should be switched on and "Exit Orphan" should also be switched on.

Retain for sure. Exiting orphans is optional, but is there to keep your strategy in synch with the account.

Everyone knows that if price precisely reaches a limit order price, there's a chance you won't get filled in real trading. If that happens your strategy will try to exit a position you don't have, but at no consequence with Portfolio Sync, because it won't send the order since you don't own the position.

But here's an example most people aren't aware of:
Chart data are driven only by full-lot trades (100 shares or more). You might get filled on an odd lot, say 10 GOOGL at 2100.50, but the low of the bar is 2100.75. This is one of the orphan scenarios that you may want to exit (or not).

QUOTE:

Everyone knows that if price precisely reaches a limit order price, there's a chance you won't get filled in real trading. If that happens your strategy will try to exit a position you don't have, but at no consequence with Portfolio Sync, because it won't send the order since you don't own the position.

This is off topic!

QUOTE:

But here's an example most people aren't aware of:
Chart data are driven only by full-lot trades (100 shares or more). You might get filled on an odd lot, say 10 GOOGL at 2100.50, but the low of the bar is 2100.75. This is one of the orphan scenarios that you may want to exit (or not).

This is interesting information!

Now, from my side, there is nothing lacking here. WL placed three orders, which were filled. After that, they were treated as if they did not exist.
Do you see any condition why the exit conditions should not be placed (for hours)?
The filled positions did not get any exit signals.

CODE:

            else
            {
                Position p = LastPosition;

                double delta = Math.Max(0.005,1.0 - hh / bars.Close[idx]);
                double tp = 1.0 + delta;
                double sl = 1.0 - delta;
                
                // end of day
                if (bars.IsLastBarOfDay(idx))
                {
                    PlaceTrade(bars, TransactionType.Sell, OrderType.Market, 0, 0, "Sell EOD");
                }

                ClosePosition(p, OrderType.Limit, Math.Max( tp * p.EntryPrice,hh), "Sell TP");
                ClosePosition(p, OrderType.Stop, sl * p.EntryPrice, "Sell SL");
            }

I was and am just thankful for the effort and informed that unfortunately I had 3 other events that should not happen like this.

It's not "off topic". Currently, strategies only work with data, not with what's in your account. (I say currently because in a distant future, this could change.)

In the first scenario, if you don't get filled, the strategy will "think" you are and will try to exit a position you don't have. This probably isn't the case we're concerned with here, but if you trade 1-minute bars with limit orders, this scenario will occur frequently.

And now that I see you're exiting with limit, this same scenario can happen there too. The strategy signaled the exit, the limit order was "hit" but you didn't get filled. That's another orphaned position case.

The other scenario too could absolutely be what is occurring - you have a position, but the strategy isn't aware of it because the bar data didn't hit the limit trigger. And if you're trading 100 symbols and getting 5 or 10 trades every few minutes, it's far too difficult for you to be aware that this is actually happening.

Incidentally, you can see these and other scenarios in the help guide > Preferences > Trading > Exit Orphan Positions at Market

Ok, not off topic in the general context. As you said, it is not the case here.

The TP and SL orders should be OCA orders. The positions were filled completely.
You can see in one of my screenshots, that other symbols have attached SL and TP orders.

What i was not able to follow on friday was that in one case the SL + TP signals were placed after more than 40 minutes, while others got the "attached" orders immediately (correctly). The three that i mentioned losely, did not get any (SL,TP) at all the whole day, but all of them were filled. That is the point what i wanted to make.

QUOTE:

And now that I see you're exiting with limit, this same scenario can happen there too. The strategy signaled the exit, the limit order was "hit" but you didn't get filled. That's another orphaned position case.

As these orders are part of OCA orders they should be "attached" and not filled. The main order was filled already.
At least i can say that in my case, all orders i had a problem with that day, were the "bracketing" orders.
OCA oders are enabled in my settings, as is the setting for orphans.

However, the reason may actually be that the already filled position (main order) is not "visible / known" for WL.
This could lead to the positions being treated as you have already described.

EDIT: Looking at the screenshots above after writing this, i was able to see that the SL + TP has been placed as OCO orders between 16:11 and 16:58 (at 16:27). But it looks like they were not accepted by IB, because they aren't attached to to the main order before 16:58.

The status for the FIS Market order wasn't received, so that was a problem from the start. For sure a persistent CancelPending is a sign something is wrong with the order status, and it could be a bug we need to track down.