quantconnect / lean.brokerages.interactivebrokers Goto Github PK
View Code? Open in Web Editor NEWInteractiveBrokers Brokerage Plugin
License: Apache License 2.0
InteractiveBrokers Brokerage Plugin
License: Apache License 2.0
CME (Bitcoin Futures): Enter the underlying symbol BRR in order to bring up the futures
CME (Ether Futures): Enter the underlying symbol ETHUSDRR to bring up the futures
ICE (Bakkt® Bitcoin Futures): Enter the underlying symbol BAKKT to bring up the futures
N/A
N/A
master
branchIf an order placed to IB is not accepted because of "No security definition has been found for the request", it is set to invalid.
If an order placed to IB is not accepted because of "No security definition has been found for the request", it is marked as an exception and the algorithm is terminated.
If the error message comes from IBPlaceOrder
, the order is set to Invalid
.
master
branchNow 2FA is mandatory, allow users to schedule time for weekly 2FA checks.
Known alert time for 2FA
Random middle-of-night alerts.
New brokerage data field; passed from UX
master
branchHello, it says here that rate limit is 50 messages/3 seconds:
however, in IB API documentation it says that the limit is 50 messages/1 second:
Aside from the TWS API's inherent limitation of 50 messages per second implying a maximum of 50 orders per second being sent to the TWS, there are no further API-only limitations. Interactive Brokers however requires its users to monitor their Order Efficiency Ratio (OER) as detailed in the Considerations for Optimizing Order Efficiency IBKB article.
https://interactivebrokers.github.io/tws-api/order_limitations.html
Is it the same rate limit? If so, can we change this limit here, in order to allow more orders sent quicker?
Be able to trade crypto in an IB account.
Not supported.
Implement crypto support to IB:
https://interactivebrokers.github.io/tws-api/cryptocurrency.html
Deploy crypto algorithm to IB.
master
branchBe able to update the order tickets generated by combo limit order.
We cannot update order tickets generated by combo limit orders since we don't have their broker ids.
N/A
Place Combo Limit Order, then try to update the tickets.
var openOrderTickets = Transactions.GetOpenOrderTickets();
foreach (var ticket in openOrderTickets) {
var direction = Math.Sign(ticket.Quantity);
var midpoint = (slice.QuoteBars[ticket.Symbol].Ask.Close + slice.QuoteBars[ticket.Symbol].Bid.Close)/2;
var limitResponse = ticket.UpdateLimitPrice(midpoint * direction);
The tickets will not be updated.
master
branchIf we deploy an algorithm with an IB account ID, LEAN loads only the capital from that account ID.
LEAN loads the capital from all of the account ID's under your IB user name. For example, say we have an IB user with a trading account that contains $10 USD and a TFSA (tax-free savings account) account that is empty. If we deploy an algorithm using the trading account ID or the TFSA account ID, the Holdings tab in the live results page shows $10 USD.
Only read the capital from the account with the provided account ID.
Deploy to IB when you have an account with multiple account IDs.
QC Cloud.
master
branchHistory requests for Futures' include extended market hours if subscribed to.
The logic in InteractiveBrokersBrokerage.cs#L3485:L3487 only accounts for extended market hours for Equity:
var useRegularTradingHours = request.Symbol.SecurityType == SecurityType.Equity
? Convert.ToInt32(!request.IncludeExtendedMarketHours)
: 0;
N/A
master
branchSupports CFD
It doesn't support CFD: InteractiveBrokersBrokerage.cs#L2385:L2411
N/A.
from AlgorithmImports import *
class UploadTest(QCAlgorithm):
def Initialize(self):
self.SetStartDate(2017,12,11) # Set Start Date
self.SetEndDate(2017,12,23) # Set End Date
self.SetCash(100000) # Set Strategy Cash
self.symbol = self.AddCfd("XAUUSD", Resolution.Minute, Market.Oanda).Symbol
def OnData(self, slice):
self.Debug(f'=========={slice[self.symbol]}==========')
self.Liquidate()
if not self.Portfolio[self.symbol].Invested:
self.MarketOrder(self.symbol, 20000)
else:
self.Liquidate(self.symbol)
def OnEndOfAlgorithm(self):
self.Liquidate(self.symbol)
master
branchAdd possibility to live trade stocks and ETFs listed on LSE and IBIS through Interactive Brokers. Support for backtesting is not in scope.
Request similar to https://www.quantconnect.com/forum/discussion/1269/trading-and-back-testing-international-stocks-on-quantconnect/p1
It is currently not possible to live trade neither European ETFs nor stocks on Interactive Brokers through QC, although these venues/assets are available on IB
My understanding so far is that the following is required:
master
branchIf an order is filled, we don't wait for other responses.
If we are trading with tick-resolution data, we might find ourselves in this case:
1 - Place Limit Order:
2023-06-23T18:45:15.2442393Z TRACE:: LiveTradingResultHandler.OrderEvent(): Time: 06/23/2023 18:45:15 OrderID: 106 EventID: 1 Symbol: TSLA Status: Submitted Quantity: -23 LimitPrice: 256.98 Message: Interactive Brokers Order Event BrokerId: 476
2 - Update Limit Order:
2023-06-23T18:45:16.0520910Z TRACE:: InteractiveBrokersBrokerage.UpdateOrder(): Symbol: TSLA Quantity: -23 Status: Submitted Id: 106
3 - Meanwhile the Limit Order was filled:
2023-06-23T18:45:16.0653963Z TRACE:: InteractiveBrokersBrokerage.HandleOrderStatusUpdates(): OrderId: 476, Status: Filled, Filled: 23, Remaining: 0, AverageFillPrice: 256.98, PermId: 1803924168, ParentId: 0, LastFillPrice: 256.98, ClientId: 0, WhyHeld: ,MktCapPrice: 0
4 - So the Update was rejected:
2023-06-23T18:45:16.1461694Z TRACE:: InteractiveBrokersBrokerage.HandleError(): RequestId: 476 ErrorCode: 104 - Cannot modify a filled order.. Origin: [Id=476] IBPlaceOrder: TSLA (STK TSLA USD Smart NASDAQ 0 )
It led to a NoBrokerageResponse
2023-06-23T18:50:16.1445549Z ERROR:: Brokerage.OnMessage(): Error - Code: NoBrokerageResponse - Timeout waiting for brokerage response for brokerage order id 476 lean id 106
Probably from the Update.
If an order is filled, stop waiting on other responses.
N/A
master
branchWe can use IB Data Provider with other brokerages.
If we deploy an algorithm that adds Futures with IB, we get the following exception:
2024-03-21T14:07:01.8531305Z ERROR:: Extensions.SetRuntimeError(): Extensions.SetRuntimeError(): RuntimeError at 03/21/2024 14:06:59 UTC. Context: LiveSynchronizer System.NullReferenceException: Object reference not set to an instance of an object.
at QuantConnect.Brokerages.InteractiveBrokers.InteractiveBrokersBrokerage.LookupSymbols(Symbol symbol, Boolean includeExpired, String securityCurrency)
at QuantConnect.Lean.Engine.DataFeeds.DataQueueHandlerManager.LookupSymbols(Symbol symbol, Boolean includeExpired, String securityCurrency) in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/LeanEnterprise/Engine/DataFeeds/DataQueueHandlerManager.cs:line 176
N/A
Deploy the following code to QC Paper and select IB as data provider.
class RetrospectiveApricotLlama(QCAlgorithm):
def Initialize(self):
self.SetStartDate(2024, 3, 20)
self.AddFuture("ES")
This issue was observed with Terminal Link with IB Data too.
QuantConnect Cloud.
master
branchLEAN handles BRK B option contracts:
Algorithm.Initialize() Error: InteractiveBrokersBrokerage.MapSymbol(): Failed to convert contract for BRK B; Contract description: OPT BRK B USD AMEX 20231020 362.5 P
The space between BRK and B causes the issue, so we can replace by .
.
public void ReturnsCorrectLeanSymbol()
{
var expiry = new DateTime(2023, 10, 20);
symbol = mapper.GetLeanSymbol("BRK B", SecurityType.Option, Market.USA, expiry, 362.5m, OptionRight.Put);
Assert.AreEqual("BRK.B 231020P00362500", symbol.Value);
Assert.AreEqual(SecurityType.Option, symbol.ID.SecurityType);
Assert.AreEqual(Market.USA, symbol.ID.Market);
Assert.AreEqual(362.5m, symbol.ID.StrikePrice);
Assert.AreEqual(expiry, symbol.ID.Date);
Assert.AreEqual(OptionRight.Put, symbol.ID.OptionRight);
}
master
branchIBDownloader
can fetch data from all securities types.
https://interactivebrokers.github.io/tws-api/historical_data.html
Only forex is supported.
Extends IBDownloader
to support all security types.
master
branchWe are seeing this timeout error more frequently now. The Algo reboot 5/5 times and still failed. Could you investigate and advise what could be the reason, is there improvement to be done to handle this error properly.
Runtime Error: Timeout waiting for brokerage response for brokerage order id xxxxx lean id 27 Stack Trace: Timeout waiting for brokerage response for...
When the algorithm places an order, the contract details should have been cached before.
The contract details are fetched before we place the orders.
If we subscribe to Interactive Brokers' data, fetch and cache this information on subscribe. On the other hand, if we use an alternative feed, e.g. QuantConnect's, the contract details are requested to IB before the orders are placed.
Perhaps we could fetch them during the heartbeats.
Deploy live algorithm with QuantConnect data and place one order to open a position and another to close it. The roundtrip time of the first will be longer. See in the syslog that GetContractDetails is called before the first order.
master
branchNew order type for IB - VWAP.
Be able to subscribe and warm up more than 300 options.
It takes 2.2 seconds to add security, and make a historical request to warm it up:
2024-02-28T23:46:14.7609004Z TRACE:: InfluxDbHistoryProvider.GetHistory(): SPX 240315P05025000 IndexOption 2024-02-28T21:10:00.0000000Z to 2024-02-28T23:46:00.8007143Z. QuoteBar. Quote. Minute
2024-02-28T23:46:14.7609481Z TRACE:: InfluxDbHistoryProvider.GetHistory(): SPX 240315P05025000 IndexOption 2024-02-28T21:10:00.0000000Z to 2024-02-28T23:46:00.8007143Z. TradeBar. Trade. Minute
2024-02-28T23:46:14.7611697Z TRACE:: InteractiveBrokersBrokerage.GetContractDetails(): SPX 240315P05025000 (OPT SPX USD Smart)
2024-02-28T23:46:15.3380276Z TRACE:: InteractiveBrokersBrokerage.GetContractDetails(): clientOnContractDetails event: OPT SPX USD SMART 20240314 5025 P SPX
2024-02-28T23:46:15.3390762Z TRACE:: InteractiveBrokersBrokerage.GetContractDetails(): contracts found: 1
2024-02-28T23:46:15.3391535Z TRACE:: InteractiveBrokersBrokerage::GetHistory(): Submitting request: SPX 240315P05025000 (OPT SPX USD Smart 20240314 5025 P): Minute/Quote 2/28/2024 9:10:00 PM UTC -> 2/28/2024 11:46:00 PM UTC
2024-02-28T23:46:16.0407240Z TRACE:: InteractiveBrokersBrokerage.GetContractDetails(): SPX 240315P05025000 (OPT SPX USD Smart)
2024-02-28T23:46:16.2256337Z TRACE:: InteractiveBrokersBrokerage.GetContractDetails(): clientOnContractDetails event: OPT SPX USD SMART 20240314 5025 P SPX
2024-02-28T23:46:16.2680131Z TRACE:: InteractiveBrokersBrokerage.GetContractDetails(): contracts found: 1
2024-02-28T23:46:16.2681073Z TRACE:: InteractiveBrokersBrokerage::GetHistory(): Submitting request: SPX 240315P05025000 (OPT SPX USD Smart 20240314 5025 P): Minute/Trade 2/28/2024 9:10:00 PM UTC -> 2/28/2024 11:46:00 PM UTC
2024-02-28T23:46:16.7629965Z TRACE:: InteractiveBrokersBrokerage::GetHistory(): Download completed: SPX 240315P05025000 (OPT SPX USD Smart 20240314 5025 P)
2024-02-28T23:46:16.7643986Z TRACE:: EventBasedDataQueueHandlerSubscriptionManager.Subscribe(): SPX 240315P05025000
2024-02-28T23:46:16.7644386Z TRACE:: InteractiveBrokersBrokerage.Subscribe(): Subscribe Request: SPX 240315P05025000
2024-02-28T23:46:16.7645160Z TRACE:: InteractiveBrokersBrokerage.GetContractDetails(): SPX 240315P05025000 (OPT SPX USD Smart)
2024-02-28T23:46:16.9488177Z TRACE:: InteractiveBrokersBrokerage.GetContractDetails(): clientOnContractDetails event: OPT SPX USD SMART 20240314 5025 P SPX
2024-02-28T23:46:16.9922404Z TRACE:: InteractiveBrokersBrokerage.GetContractDetails(): contracts found: 1
2024-02-28T23:46:16.9924096Z TRACE:: InteractiveBrokersBrokerage.Subscribe(): Subscribe Processed: SPX 240315P05025000 (OPT SPX USD Smart 20240314 5025 P) # 22. SubscribedSymbols.Count: 5
2024-02-28T23:46:16.9925928Z TRACE:: DataManager.AddSubscription(): Added SPX 240315P05025000,#0,SPX,Minute,QuoteBar,Quote,Raw,OpenInterest. Start: 2/28/2024 11:46:00 PM. End: 12/31/2050 5:00:00 AM
2024-02-28T23:46:16.9928868Z TRACE:: DataManager.AddSubscription(): Added SPX 240315P05025000,#0,SPX,Minute,TradeBar,Trade,Raw,OpenInterest. Start: 2/28/2024 11:46:00 PM. End: 12/31/2050 5:00:00 AM
2024-02-28T23:46:16.9932410Z TRACE:: DataManager.AddSubscription(): Added SPX 240315P05025000,#0,SPX,Minute,OpenInterest,OpenInterest,Raw,OpenInterest,Internal. Start: 2/28/2024 6:00:00 AM. End: 12/31/2050 5:00:00 AM
It leads to a 10-minute time out with less than 300 contracts:
10 min x 60 = 600 s / 2 s = 300 contracts
See #48
class IndexOptionsAlgorithm(QCAlgorithm):
def Initialize(self):
seeder = FuncSecuritySeeder(self.GetLastKnownPrices)
self.SetSecurityInitializer(lambda security: seeder.SeedSecurity(security))
index_symbol = self.AddIndex("SPX").Symbol
option = self.AddIndexOption(index_symbol)
option.SetFilter(lambda u: u.Strikes(-20, 1).Expiration(0, 100).PutsOnly().StandardsOnly())
master
branchStop Limit orders are processed correctly.
Stop Limit orders are placed, but we don't get a status change from new to submitted:
N/A
Deploy the following algorithm to IB Paper:
# region imports
from AlgorithmImports import *
# endregion
class EnergeticOrangeFox(QCAlgorithm):
def Initialize(self):
self.SetStartDate(2022, 9, 14)
self.equity = self.AddEquity("DLO")
self.ticket = None
def OnData(self, data: Slice):
if self.ticket:
return
self.ticket = self.StopLimitOrder(
self.equity.Symbol, 1,
self.equity.Price * 1.10,
self.equity.Price * 0.90,
tag='Entry.StopLimit')
self.StopMarketOrder(
self.equity.Symbol, 1,
self.equity.Price * 1.10,
tag='Entry.StopMarket')
self.LimitOrder(
self.equity.Symbol, 1,
self.equity.Price * 0.90,
tag='Entry.Limit')
master
branchThe algorithm doesn't try to request data if not connected.
It looks like the algorithm is trying to fetch account data before it's connected. It doesn't have an impact on the algorithms because this data is fetched as soon as the connection is restored.
2022-12-01T16:23:24.0746316Z TRACE:: InteractiveBrokersBrokerage.Connect(): IB message processing thread ended: #28
2022-12-01T16:23:24.0751344Z TRACE:: InteractiveBrokersBrokerage.Connect(): Data subscription count 33, restoring data subscriptions is required
2022-12-01T16:23:24.0751526Z TRACE:: InteractiveBrokersBrokerage.Connect(): Attempting to connect (1/5) ...
2022-12-01T16:23:24.0751829Z TRACE:: InteractiveBrokersBrokerage.HandleConnectionClosed(): API client disconnected [Server Version: 0].
2022-12-01T16:23:26.5753319Z TRACE:: InteractiveBrokersBrokerage.Connect(): calling _client.ClientSocket.eConnect()
2022-12-01T16:23:26.6500363Z TRACE:: InteractiveBrokersBrokerage.HandleConnectAck(): API client connected [Server Version: 157].
2022-12-01T16:23:26.6507793Z TRACE:: InteractiveBrokersBrokerage.Connect(): IB next valid id received.
2022-12-01T16:23:26.6508385Z TRACE:: InteractiveBrokersBrokerage.Connect(): IB message processing thread started: #39
2022-12-01T16:23:26.6510070Z TRACE:: InteractiveBrokersBrokerage.DownloadAccount(): Downloading account data for DU5686920
2022-12-01T16:23:32.0605209Z TRACE:: InteractiveBrokersBrokerage.OnIbAutomaterOutputDataReceived(): Window event: [WINDOW_CLOSED] - Window title: [Pending Tasks] - Window name: [Pending Tasks]
2022-12-01T16:23:46.6515145Z TRACE:: InteractiveBrokersBrokerage.DownloadAccount(): Operation took longer than 15 seconds.
2022-12-01T16:23:46.6516005Z TRACE:: InteractiveBrokersBrokerage.Connect(): DownloadAccount failed. Operation took longer than 15 seconds.
2022-12-01T16:23:46.6666874Z TRACE:: InteractiveBrokersBrokerage.HandleError(): RequestId: -1 ErrorCode: 542 - Request Account Data Sending Error -
2022-12-01T16:23:46.6669102Z TRACE:: Brokerage.OnMessage(): Information - Code: 542 - Request Account Data Sending Error -
2022-12-01T16:23:46.6669841Z TRACE:: CloudLiveTradingResultHandler.BrokerageMessage(): Information - Code: 542 - Request Account Data Sending Error - - Brokerage: InteractiveBrokersBrokerage
2022-12-01T16:23:46.6733330Z TRACE:: InteractiveBrokersBrokerage.HandleConnectionClosed(): API client disconnected [Server Version: 0].
2022-12-01T16:23:46.6734678Z TRACE:: InteractiveBrokersBrokerage.Connect(): IB message processing thread ended: #39
2022-12-01T16:23:47.6737263Z TRACE:: InteractiveBrokersBrokerage.Connect(): Attempting to connect (2/5) ...
2022-12-01T16:23:47.6737840Z TRACE:: InteractiveBrokersBrokerage.HandleConnectionClosed(): API client disconnected [Server Version: 0].
2022-12-01T16:23:47.6738355Z TRACE:: InteractiveBrokersBrokerage.Connect(): calling _client.ClientSocket.eConnect()
2022-12-01T16:23:47.6790085Z TRACE:: InteractiveBrokersBrokerage.HandleConnectAck(): API client connected [Server Version: 157].
master
branchIf IB no longer supports a contract because it has expired, LEAN doesn't throw an error, and returns an empty list of historical data.
The algorithm exists with No security definition has been found for the request
error.
When the error 200 occurs for history requests, we check out if the symbol has expired, and return an empty list.
Warm-up VX futures' algorithm.
var symbol = QuantConnect.Symbol.CreateFuture("VX", Market.CFE, new(2023, 10, 18), "VX18V23");
symbol = AddFutureContract(symbol).Symbol; // <- optional
var history = History(symbol, TimeSpan.FromDays(5), Resolution.Minute);
master
branchIB can occasionally think another TWS instance is using the connection and prevent redeploying.
Possible causes include:
Instantly redeploy the gateway.
Generally need to wait 1 minute to redeploy.
Call the Menu->Exit button instead of killing the process.
- Automator -> Java (use a file to request a clean kill to java)
Restart the gateway quickly on a slow computer such as a cloud instance.
master
branchThe weekly restart is triggered every week.
One live deployment didn't trigger the weekly restart on the third week:
The user got 2FA on Sundays Sept 10th and Sep 17th at 22 UTC as expected but not on Sep 24th.
A few milliseconds before he got the following message:
2023-09-24T21:54:59.9974199Z TRACE:: InteractiveBrokersBrokerage.StartGatewayWeeklyRestartTask(): triggering weekly restart manually
The weekly restart was bumped to Oct 1st (next Sunday).
2023-09-24T21:54:59.9975778Z TRACE:: InteractiveBrokersBrokerage.StartGatewayWeeklyRestartTask(): scheduled weekly restart to 10/1/2023 10:00:00 PM (in 7.00:00:00.0024839)
N/A
master
branchGetOpenOrders returns QC Orders with OrderStatus != None.
OrderStatus = None for every order.
IB provides the orderState has separate object in args at OpenOrder. Passing it through during IB.Order -> QC.Order conversion in following PR.
Have an open limit order with IB. Connect live and investigate Transactions.GetOpenOrders() during initialization / wramup. Those returned tickets need orderStatus
Tested with fix IB Gateway version: 10.192b and TWS 10.24.1c
master
branchBe able to place combo orders.
https://interactivebrokers.github.io/tws-api/basic_orders.html
When we place option strategies with multiple legs, they are executed separately.
N/A
Deploy BasicTemplateOptionStrategyAlgorithm.cs, and observe that they are not placed as combo orders, thus filled separately.
master
branchIf the data connection is lost, LEAN recovers it. No need to restart the deployment.
Apparently the algorithm trading SPX options fails because the algorithm stops receiving SPX (index data), so it didn't trigger the option universe filter.
The latest "Market data farm connection" says
2023-08-08T05:07:28.4818118Z TRACE:: InteractiveBrokersBrokerage.HandleError(): RequestId: -1 ErrorCode: 2108 - Market data farm connection is inactive but should be available upon demand.usopt
but it refers to usopt, not index.
When the user restarted the algorithm, it received the data.
N/A
master
branchPortfolio.Values not shows holding quantities from the broker even though there are holdings.
In the example I give it is called from Initialize function but this can happen throughout the strategy.
I know its a hard bug to fix. Please instruct me on how to actually yes check for the current actual holdings through a quantconnect c# strat. maybe I am doing something wrong.
If Portfolio.Values are not reliable at any point during the strat runtime please also say it so I could implement a different mechanism for positions tracking.
In live mode with broker Portfolio.Values should return the actual portfolio values or indicate in some manner that we dont have the current holdings at the moment.
Portfolio.Values returns 0 from holding.Quantity even if holding.Quantity is not 0.
Populate Portfolio.Values before any user code is running.
in live mode, AddEquity to the maximum amount of tickers (100) minute data and have one position at ONE of the tickers.
in Initialize() go through Portfolio.Values in order to find the one position with holding.Quantity not 0.
no such position is found although actually yes exists in the broker.
master
branchA declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.