eg at 1pm ET, ask form the last 15 minutes of minute data of a ticker and you wont get it. You will get the last ingested data. Solution: when asking for minute data during the day, a portion should be queried directly from IB

reqHistoricalData(
tickerId,
contract,
(intdate.strftime('%Y%m%d %H:%M:%S') + ' GMT'),
(duration),
(barsize),
(what),
int(useRTH),
2 or 1 )

Currently the README.rst points to the live branch on master.
I'm not able to fix it by myself as I have no access to the appveyor project.

Hi,

I'd propose to use 'master' branch instead of 'live' for the following reasons:

• it's default git branch and git checks it out when you do fresh repository clone
• we don't need to change branch in README.rst and .travis.yml
• we reducing possibility to conflict with upstream

Regards,
Ed

USD is hardcoded in several places. That needs to be configurable. Whether it needs to be a property of the exchange or asset or algo I'm not sure

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System: Linux ip-10-0-3-162 4.9.62-21.56.amzn1.x86_64 #1 SMP Thu Nov 16 05:37:08 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
• Python Version: Python 2.7.12
• Python Bitness: 64
• How did you install Zipline: pip install zipline-live[ib]
• Python packages: pip freeze
  alembic==0.9.6
  bcolz==0.12.1
  Bottleneck==1.2.1
  certifi==2017.11.5
  chardet==3.0.4
  click==6.7
  contextlib2==0.5.5
  cyordereddict==1.0.0
  Cython==0.27.3
  decorator==4.1.2
  empyrical==0.3.3
  IbPy2==0.8.0
  idna==2.6
  intervaltree==2.1.0
  Logbook==1.1.0
  lru-dict==1.1.6
  Mako==1.0.7
  MarkupSafe==1.0
  multipledispatch==0.4.9
  networkx==2.0
  numexpr==2.6.4
  numpy==1.13.3
  pandas==0.18.1
  pandas-datareader==0.5.0
  patsy==0.4.1
  python-dateutil==2.6.1
  python-editor==1.0.3
  pytz==2017.3
  requests==2.18.4
  requests-file==1.4.2
  requests-ftp==0.3.1
  scipy==1.0.0
  six==1.11.0
  sortedcontainers==1.5.7
  SQLAlchemy==1.1.15
  statsmodels==0.8.0
  tables==3.4.2
  toolz==0.8.2
  urllib3==1.22
  zipline-live==1.1.0.4

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

• What did you expect to happen?
  I followed the example from http://www.zipline-live.io/tutorial and I think it zipline should run successfully

• What happened instead?
  I am receiving the following error
  File "/home/ec2-user/workspace/zEnv/local/lib64/python2.7/site-packages/zipline/data/benchmarks.py", line 30, in get_benchmark_returns
  df.index = df.index.tz_localize('UTC')
  AttributeError: 'Index' object has no attribute 'tz_localize'

Here is how you can reproduce this issue on your machine:
I use a standard AWS AMI

Reproduction Steps

1. I following the same steps as the zipline-live.io example

...

What steps have you taken to resolve this already?

I attempted to modify the line in the file based on some suggestion in StackOverflow, but no luck.
"/home/ec2-user/workspace/zEnv/local/lib64/python2.7/site-packages/zipline/data/benchmarks.py"

Original:
df.index = df.index.tz_localize('UTC')

Attempt-1:
df.index = df.index.to_series().tz_localize('UTC')

Attempt-2:
df.index = pd.DatetimeIndex(df.index).tz_localize('UTC')

Anything else?

Sincerely,
w2fong

Contrary to what zipline-live's feature page states
order/trading guards are supported, but not tested:
LiveTradingAlgorithm.order() calls TradingAlgorithm._calculate_order(), which calls to TradingAlgorithm.validate_order_params() which finally enforces that the order meets the guards set by the user.

This effort is to prove that Trading Guards are working:
Create a testcase which tests that sets trading guards are called if they are set and used through LiveTradingAlgorithm.

As of today one needs to know the supported broker and it's respective name.
It would be helpful to add a command line argument which would list the available brokers (e.g. --broker-list).

I know, that you already planned to support fundamentals data.

I just wanted to point your attention to a small project I started sometime ago to provide fundamentals data support to zipline: https://github.com/alphaville76/Fundamentals

I'd also like to help in implementing this feature, in zipline-live: let me know, how I could help.

Last but not least: An important question is how to feed the data. Of course it would be great to continue to support Morningstar as source, but I don't know how expensive their data are and how they could be licensed to individual investors.

Hi, I am trying to run the demo after installing zipline-live using python2.7 in a virtualenv but cannot get past this issue.

$ zipline run -f ~/zipline-algos/demo.py --state-file ~/zipline-algos/demo.state --realtime-bar-target ~/zipline-algos/realtime-bars/ --broker ib --broker-uri localhost:4002:1232 --bundle quantopian-quandl --data-frequency minute

[2019-02-01 22:56:29.830541] INFO: IB Broker: Connecting: localhost:4002:1232
Server Version: 76
TWS Time at connection:20190201 17:56:29 EST
[2019-02-01 22:56:29.918050] INFO: IB Broker: [2104] Market data farm connection is OK:usfarm.nj (-1)
[2019-02-01 22:56:29.918326] INFO: IB Broker: [2104] Market data farm connection is OK:usfuture (-1)
[2019-02-01 22:56:29.918490] INFO: IB Broker: [2104] Market data farm connection is OK:usfarm (-1)
[2019-02-01 22:56:29.918640] INFO: IB Broker: [2106] HMDS data farm connection is OK:euhmds (-1)
[2019-02-01 22:56:29.918791] INFO: IB Broker: [2106] HMDS data farm connection is OK:fundfarm (-1)
[2019-02-01 22:56:29.918935] INFO: IB Broker: [2106] HMDS data farm connection is OK:ushmds (-1)
[2019-02-01 22:56:29.919022] DEBUG: IB Broker: Execution details completed for request 0
[2019-02-01 22:56:30.033363] INFO: IB Broker: Managed accounts: ['DUxxxxxx']
[2019-02-01 22:56:30.134061] INFO: IB Broker: Local-Broker Time Skew: 0 days 00:00:00
Traceback (most recent call last):
File "/home/m/zipline/bin/zipline", line 10, in
sys.exit(main())
File "/home/m/zipline/local/lib/python2.7/site-packages/click/core.py", line 764, in call
return self.main(*args, **kwargs)
File "/home/m/zipline/local/lib/python2.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/m/zipline/local/lib/python2.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/m/zipline/local/lib/python2.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/m/zipline/local/lib/python2.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/m/zipline/local/lib/python2.7/site-packages/zipline/main.py", line 102, in _
return f(*args, **kwargs)
File "/home/m/zipline/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/m/zipline/local/lib/python2.7/site-packages/zipline/main.py", line 316, in run
realtime_bar_target=realtime_bar_target
File "/home/m/zipline/local/lib/python2.7/site-packages/zipline/utils/run_algo.py", line 129, in _run
bundle_timestamp,
File "/home/m/zipline/local/lib/python2.7/site-packages/zipline/data/bundles/core.py", line 527, in load
minute_equity_path(name, timestr, environ=environ),
File "/home/m/zipline/local/lib/python2.7/site-packages/zipline/data/minute_bars.py", line 905, in init
metadata = self._get_metadata()
File "/home/m/zipline/local/lib/python2.7/site-packages/zipline/data/minute_bars.py", line 950, in _get_metadata
return BcolzMinuteBarMetadata.read(self._rootdir)
File "/home/m/zipline/local/lib/python2.7/site-packages/zipline/data/minute_bars.py", line 229, in read
calendar = get_calendar(raw_data['calendar_name'])
File "/home/m/zipline/local/lib/python2.7/site-packages/zipline/utils/calendars/calendar_utils.py", line 87, in get_calendar
raise InvalidCalendarName(calendar_name=name)
zipline.errors.InvalidCalendarName: The requested TradingCalendar, XNYS, does not exist.

We need to be able to instruct zipline to do live trading using command line arguments.

On the command line the user should be able to define TWS Connection URI (host:port:client_id).
The start, end date shall not be defined by user (both must be set by zipline to today), data-frequency shall not be defined by user (must be set to 'minute' by zipline).
Optionally a live trading state filename could be accepted by the user (links to: #6).

If live-trading is enabled then the respective live trading classes shall be executed during the execution chain.

the before_trading_starts needs to listen to the calendar which is used. The only problem is what to do with exchanges that are open 24/7, the before_trading_starts gets only ran once at startup

In zipline data.history() call's return value include today's value.
We currently return historical data from the bundle, which does not contain value for today.

Forward filling data using broker will cause execution delays for the first timestamp of the day:
as the first trade might not happen immediately at the beginning of the day.

Per @pbharrin 's design the TWSConnection should be split up into two parts:

• Broker which is broker agnostic interface needed to enable live trading
• IBBroker which implements the Broker's interfaces to bridge between TWS and Zipline

Interactive Brokers provides 'Financial Advisor' (FA) accounts, which groups multiple accounts together. These accounts currently are not supported by zipline-live.

The support could be easily achieved by:

1. extending broker-uri with an optional account code. e.g.: --broker-uri localhost:7492:111:DU32145
2. store the specified account code in IBBroker's constructor (if specified)
3. set the m_account when placing an order:

   zipline/zipline/gens/brokers/ib_broker.py

   Line 664 in 7c68fb4

   order = Order()

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System: (Mac OS 10.13.6)

• Python Version: 3.6.3

• How did you install Zipline: (pip)

• Python packages: alembic==0.9.9 alpha-vantage==2.0.0 bcolz==0.12.1 Bottleneck==1.2.1 certifi==2018.4.16 chardet==3.0.4 click==6.7 contextlib2==0.5.5 cycler==0.10.0 cyordereddict==1.0.0 Cython==0.28.3 DateTime==4.2 decorator==4.3.0 empyrical==0.4.3 ib==0.8.0 IbPy2==0.8.0 idna==2.7 intervaltree==2.1.0 kiwisolver==1.0.1 Logbook==1.4.0 lru-dict==1.1.6 lxml==4.2.1 Mako==1.0.7 MarkupSafe==1.0 matplotlib==2.2.2 multipledispatch==0.5.0 networkx==1.11 numexpr==2.6.5 numpy==1.14.4 pandas==0.18.1 pandas-datareader==0.6.0 patsy==0.5.0 pyparsing==2.2.0 PyQt5==5.11.2 PyQt5-sip==4.19.12 python-dateutil==2.7.3 python-editor==1.0.3 pytz==2018.4 requests==2.19.0 requests-file==1.4.3 requests-ftp==0.3.1 scipy==1.1.0 simplejson==3.16.0 six==1.11.0 sortedcontainers==2.0.4 SQLAlchemy==1.2.8 statsmodels==0.9.0 tables==3.4.4 toolz==0.9.0 urllib3==1.23 wrapt==1.10.11 zipline-live==1.1.0.5 zope.interface==4.5.0

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

• I attempt running buyapple.py demo file, it will buy the contract, but then spits me an error regarding my time
• I have ran buyapple.py again and again, buying shares every time. I can now run it once per day, and then IB throws me an error in the way of datetime.

Here is how you can reproduce this issue on your machine:

Reproduction Steps

1. Updated Mac OS
2. Updated Interactive Brokers TWS
3. Attempt to run BuyApple.py
   ...

What steps have you taken to resolve this already?

I have attempted to reinstall zipline and its dependencies, uninstalled and reinstalled IB TWS, I have created a new account on TWS, which will work fine, until submitting and order, at which point the order will throw an error due to its datetime string, by my understanding.
...

Anything else?

Here is a description of the error:

(venv-zipline-live) HYPERiON:live Derek$ zipline run -f alpha-live.py --state-file alpha-live.state --realtime-bar-target realtime-bars/ --broker ib --broker-uri localhost:4001:1232 --bundle quantopian-quandl --data-frequency minute
[2018-07-31 16:49:09.683764] INFO: IB Broker: Connecting: localhost:4001:1232
Server Version: 76
TWS Time at connection:20180731 09:49:08 PST
[2018-07-31 16:49:09.804428] ERROR: IB Broker: <class 'str'> returned a result with an error set
ValueError: Error parsing datetime string "20180731 08:40:00" at position 9

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/Derek/venv-zipline-live/lib/python3.6/site-packages/ib/ext/EReader.py", line 107, in run
while not self.isInterrupted() and self.processMsg(self.readInt()):
File "/Users/Derek/venv-zipline-live/lib/python3.6/site-packages/ib/ext/EReader.py", line 744, in processMsg
self.eWrapper().execDetails(reqId, contract, exec_)
File "/Users/Derek/venv-zipline-live/lib/python3.6/site-packages/zipline/gens/brokers/ib_broker.py", line 353, in execDetails
exec_time=pd.to_datetime(exec_detail.m_time),
File "/Users/Derek/venv-zipline-live/lib/python3.6/site-packages/pandas/util/decorators.py", line 91, in wrapper
return func(*args, **kwargs)
File "/Users/Derek/venv-zipline-live/lib/python3.6/site-packages/pandas/tseries/tools.py", line 291, in to_datetime
unit=unit, infer_datetime_format=infer_datetime_format)
File "/Users/Derek/venv-zipline-live/lib/python3.6/site-packages/pandas/tseries/tools.py", line 429, in _to_datetime
return _convert_listlike(np.array([arg]), box, format)[0]
File "/Users/Derek/venv-zipline-live/lib/python3.6/site-packages/pandas/tseries/tools.py", line 398, in _convert_listlike
require_iso8601=require_iso8601
File "pandas/tslib.pyx", line 2134, in pandas.tslib.array_to_datetime (pandas/tslib.c:41972)
File "pandas/tslib.pyx", line 2269, in pandas.tslib.array_to_datetime (pandas/tslib.c:40684)
File "pandas/tslib.pyx", line 2229, in pandas.tslib.array_to_datetime (pandas/tslib.c:39919)
File "pandas/src/datetime.pxd", line 141, in datetime._string_to_dts (pandas/tslib.c:85631)
SystemError: <class 'str'> returned a result with an error set
[2018-07-31 16:49:09.808153] ERROR: IB Broker: [None] None (<class 'str'> returned a result with an error set)
[2018-07-31 16:49:09.808440] ERROR: IB Broker: IB Connection closed
[2018-07-31 16:49:09.836936] ERROR: IB Broker: [504] Not connected (-1)

...

Sincerely,
$ HypeSound

Per Eddie Hebert's (zipline core dev) recommendation we should move the realtime & historical data combining logic (implemented in #62) int MinuteBarReader.

The reasoning and the new desing approach could be found here:
quantopian#1963

It would be great to have a basic front-end to monitor the algorithms like Q has. Big project but worth it I think

Hello,

I was just curious as to what a state-file is and how do we create one if we do not have it? I am trying to live paper trade from the command line to test code for our production systems. I cannot find any documentation on the state file, however, and this is the only command that is causing an error thus far. Thank you!

https://github.com/zipline-live/zipline/blob/master/zipline/gens/realtimeclock.py#L87

This module is using the local time but technically it can miss some events that happen once a day like market close, if one iteration takes so long and the next chance to check the local clock is beyond this boundary.

When live trading with pipeline, the attach_pipeline function is forced to reside in Initialise, however the pipeline object is not serialised. If we allow attach_pipeline in before_trading_starts it would work

In case of live execution the orders shall be routed to a real broker (e.g. IB) through the BrokerClient class.

It needs to be evaluated where is the best to put this logic into:

• TradingAlgorithmLive class or (based on TradingAlgorithm)
• BlotterLive class (based on Blotter)

Once the place is identified the wiring between BrokerClient and Zipline should be implemented.

Depends on: #4

Dear Zipline Maintainers,
I've found that the function data.current() doesn't work now for so called fuzzy symbols (symbols that consist of two parts), such as BRK B (Berkshire Hathaway B).

I think the problem is that the way fuzzy symbols are presented now in securities database is incompatible with interactive brokers' smart search. So when we try to get market data for a fuzzy symbol, smart search can't find the security for the symbol which results in the following error:

'No security definition is found for this request, error code 200'

I tried to find BRK B stock in Tws manually and the result was that Tws couldn't find the security when querying 'BRK.B' or 'BRK_B''. But It managed to find it when I typed in the search box 'BRK B' (with a space)

We need to reflect this in how zipline-live handles compounded tickers to query interactive brokers in a way it can understand them.

To reproduce the issue try to retrieve some data_current() information for the stock BRK_B

Sincerely,
Vitaly Ermilov

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System: Ubuntu 16.04

• Python Version: 2.7

• How did you install Zipline: pip

• Python packages:
  alembic==0.9.6
  bcolz==0.12.1
  Bottleneck==1.2.1
  certifi==2017.7.27.1
  chardet==3.0.4
  click==6.7
  contextlib2==0.5.5
  cyordereddict==1.0.0
  Cython==0.27.1
  decorator==4.1.2
  empyrical==0.3.2
  IbPy2==0.8.0
  idna==2.6
  intervaltree==2.1.0
  Logbook==1.1.0
  lru-dict==1.1.6
  Mako==1.0.7
  MarkupSafe==1.0
  multipledispatch==0.4.9
  networkx==1.9.1
  numexpr==2.6.4
  numpy==1.13.3
  pandas==0.18.1
  pandas-datareader==0.5.0
  patsy==0.4.1
  pkg-resources==0.0.0
  python-dateutil==2.6.1
  python-editor==1.0.3
  pytz==2017.2
  requests==2.18.4
  requests-file==1.4.2
  requests-ftp==0.3.1
  scipy==0.19.1
  six==1.11.0
  sortedcontainers==1.5.7
  SQLAlchemy==1.1.14
  statsmodels==0.8.0
  tables==3.4.2
  toolz==0.8.2
  urllib3==1.22
  zipline-live==1.1.0.4

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

zipline fails to ingest data from quandl WIKI

(venv-zipline-live) ubuntu@ip-172-31-34-154:~$ zipline ingest -b quandl
Downloading WIKI metadata: [####################################]
Traceback (most recent call last):
File "/home/ubuntu/venv-zipline-live/bin/zipline", line 11, in
sys.exit(main())
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/zipline/main.py", line 388, in ingest
show_progress,
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/zipline/data/bundles/core.py", line 451, in ingest
pth.data_path([name, timestr], environ=environ),
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/zipline/data/bundles/quandl.py", line 295, in quandl_bundle
show_progress=show_progress,
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/zipline/data/bundles/quandl.py", line 129, in fetch_symbol_metadata_frame
data['auto_close_date'] = data['end_date'] + pd.Timedelta(days=1)
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/pandas/core/ops.py", line 609, in wrapper
na_op)
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/pandas/core/ops.py", line 569, in maybe_convert_for_time_op
return cls(left, right, name, na_op)
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/pandas/core/ops.py", line 282, in init
lvalues = self._convert_to_array(left, name=name)
File "/home/ubuntu/venv-zipline-live/local/lib/python2.7/site-packages/pandas/core/ops.py", line 398, in _convert_to_array
if (inferred_type in ('datetime64', 'datetime', 'date', 'time') or
TypeError: data type "datetime" not understood

What steps have you taken to resolve this already?

This may be because of pandas and numpy versions incompatibility . In some cases the similar issue has been resolved by upgrading pandas.

Sincerely,
Vitaly Ermilov

I imagine that etrade support is a long-shot. It would probably require a library to interact with their api. The closest I've found is pyetrade, but it seems to lack sufficient support/contributions and is a long way from being close to comprehensive. I plan to put some of my free time into supporting said library, but I was wondering if it was worth it; I was wondering if support for etrade as a broker in zipline-live is feasible or not.

Hello Friends,

I've had a very hard time getting zipline-live to function based off of the tutorial on the site.
I've created a project in Spyder to get it going in Py 2.7.
I've properly set up my Trader Workstation settings and logged on, and set the ports correctly.

I've set the PYTHON PATH MANAGER to see the Zipline folder/files, the zipline-live folder, and in the python console the tutorial code seems to be functioning:

from zipline.api import order, symbol

import logbook
log = logbook.Logger('algo')

def initialize(context):
pass

def handle_data(context, data):
sym = symbol("AAPL")
log.info("handle_data: last_traded={} price={} OHLC={}/{}/{}/{} Volume={}".format(
data.current(sym, 'last_traded'),
data.current(sym, 'price'),
data.current(sym, 'open'),
data.current(sym, 'high'),
data.current(sym, 'low'),
data.current(sym, 'close'),
data.current(sym, 'volume')))

seems to process fine (no error messages).

However, when I go back to terminal and put in:

zipline run -f ~/zipline-algos/demo.py --broker ib --broker-uri localhost:7496:1232 --bundle quantopian-quandl --data-frequency minute

as suggested in the tutorial, I get an error message in terminal (on a mac).

Error: no such option: --broker

I'm new to Python (been trading for 12 year) I've had a very difficult time with just setting up a Python Project and properly linking files just to get things going, the programming aspect of it makes complete sense, but the setting up of proper directories and folders is very obtuse and there is very little basic introduction to how to get running from the ground up. Any help is greatly appreciated!

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System: (Ùbuntu 19.10 and ubuntu 18.04 LTS)
• Python Version: $ python 3.7
• Python Bitness: $ python -c 'import math, sys;print(int(math.log(sys.maxsize + 1, 2) + 1))'
• How did you install Zipline: (pip virtualenv)
• Python packages: $ pip

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

• What did you expect to happen?
  Have no error to install pip install zipline-live[ib]

• What happened instead?

I am trying to install zipline-live in order to trade with Interactive Brokers.

I followed this website (http://www.zipline-live.io/tutorial) for installing zipline-live for my python algorithm, but I ran into some issues. I have this error in my LinuxOS (Ubuntu 18.04 LTS or Ubuntu 19.10).
I cannot install the installation always fails.

this my error :

Failed building wheel for bcolz
Failed building wheel for cyordereddict
statsmodels 0.10.1 has requirement pandas>=0.19, but you'll have pandas 0.18.1 which is incompatible.
pandas-datareader 0.8.1 has requirement pandas>=0.21, but you'll have pandas 0.18.1 which is incompatible.

Command "/usr/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-install-m5zc6jna/bcolz/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-0fps0y5y/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-install-m5zc6jna/bcolz/

Thank you!
Here is how you can reproduce this issue on your machine:

Reproduction Steps

1.virtualenv ~/venv-zipline-live
2.source ~/venv-zipline-live/bin/activate
3.pip install zipline-live[ib]
...

What steps have you taken to resolve this already?

I upgraded python librairies

This my pip list

Package Version

alembic 1.2.1
apturl 0.5.2
asn1crypto 0.24.0
attrs 18.2.0
Automat 0.6.0
bcolz 1.2.1
blinker 1.4
catfish 1.4.7
certifi 2018.8.24
chardet 3.0.4
Click 7.0
cloud-init 19.2
colorama 0.3.7
command-not-found 0.3
configobj 5.0.6
constantly 15.1.0
contextlib2 0.6.0.post1
cryptography 2.3
cupshelpers 1.0
Cython 0.29.13
defer 1.0.6
distro 1.3.0
distro-info 0.21ubuntu2
entrypoints 0.3
httplib2 0.11.3
hyperlink 17.3.1
idna 2.6
incremental 16.10.1
Jinja2 2.10
jsonpatch 1.21
jsonpointer 1.10
jsonschema 2.6.0
keyring 17.1.1
keyrings.alt 3.1.1
language-selector 0.1
launchpadlib 1.10.6
lazr.restfulclient 0.14.2
lazr.uri 1.0.3
lightdm-gtk-greeter-settings 1.2.2
Logbook 1.5.3
macaroonbakery 1.2.1
Mako 1.1.0
MarkupSafe 1.1.0
menulibre 2.2.0
mugshot 0.4.1
netifaces 0.10.4
numexpr 2.7.0
numpy 1.17.3
oauthlib 2.1.0
olefile 0.46
onboard 1.4.1
pandas 0.25.2
pexpect 4.6.0
Pillow 5.4.1
pip 18.1
protobuf 3.6.1
psutil 5.5.1
pyasn1 0.4.2
pyasn1-modules 0.2.1
pycairo 1.16.2
pycrypto 2.6.1
pycups 1.9.73
pycurl 7.43.0.2
PyGObject 3.32.0
PyJWT 1.7.0
pymacaroons 0.13.0
PyNaCl 1.3.0
pyOpenSSL 19.0.0
pyRFC3339 1.1
pyserial 3.4
PySimpleSOAP 1.16.2
python-apt 1.8.4
python-dateutil 2.7.3
python-debian 0.1.34
python-debianbts 2.8.2
python-editor 1.0.4
python-magic 0.4.16
pytz 2018.9
pyxdg 0.25
PyYAML 3.13
reportlab 3.5.18
requests 2.21.0
requests-unixsocket 0.1.5
SecretStorage 2.3.1
service-identity 16.0.0
setuptools 40.8.0
sgt-launcher 0.2.4
simplejson 3.16.0
six 1.12.0
SQLAlchemy 1.3.10
ssh-import-id 5.7
system-service 0.3
systemd-python 234
toolz 0.10.0
Twisted 18.9.0
ubuntu-advantage-tools 19.2
ubuntu-drivers-common 0.0.0
ufw 0.36
unattended-upgrades 0.1
urllib3 1.24.1
wadllib 1.3.3
wheel 0.32.3
xcffib 0.5.1
xkit 0.0.0
zope.interface 4.3.2

Anything else?

#1 Do you have more informations for zipline-live? (i.e. A list of the pre-requisites.)
#2 Why install python 2.7 and not python 3.7 ?
#3 How can I connect it to Interactive Brokers after the installation? (Is the connection to IB automatic? Because zipline-live is installed with the following command:
pip install zipline-live[ib] )
Sincerely,

With python 2.7 I have this error :

  File "/tmp/easy_install-rqICVR/numpy-1.17.3/setup.py", line 31, in <module>
    )
RuntimeError: Python version >= 3.5 required.

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-x9i2pV/zipline-live/

Thank you!

Felix [email protected]
Charles [email protected]

Hi!

Currently, zipline-live only supports Interactive Brokers. With CCXT it would add support for all of crypto-land.

gr,

Tom

In order to have test coverage for the project an admin needs to:

Sign up for coveralls using zipline-live (done)
Enable builds for branch 'live' in coveralls

Hi Guys,

I want to live-trade through IB but it seems that the way zipline is designed it does not allow me to enter orders for symbols for which I do not have a live-data feed coming in. If I try to do this then it complains about the symbol being "de-listed" and no "last price" being available.

If I don't have a data subscription I still see no reason why I couldn't enter market orders for whatever symbol I want but zipline is very stubborn in that it only wants to allow me to trade symbols for which it knows the last-price and has a receiving real-time data-feed.

Is there any way to overcome this?

Thanks in advance for your reply,

Best Regards,

Daniel

Currently there are a couple of fields are set to None at account, portfolio & position variables:

Some of the values are dependent on state persistence, while others could be deducted from IB's reporting.

This issue is to implement others, which are not dependent on state.

Zipline's one key feature is the ability to store the backtest result.
Later, the result could be feed into pyfolio for analysis.

Currently zipline-live does not provide such capability due to it's execution model:
start at BOD, exit at EOD. Due to the daily start the perf tracker object always tracks one
day.

To address this shortcoming the perf_tracker object needs to be persisted and updated
at every start.

Hey zipline-live developers,

I am currently working on a change to how we compute risk metrics which may have an impact on the format of the perf packets and risk packet. I wrote up a small post about the proposed change which I posted to the mailing list here. I am looking for feedback from the community.

Some things that I think might apply specifically to the zipline-live project:

1. Are you streaming results from TradingAlgorithms with TradingAlgorithm.get_generator()?
2. Do you use the risk report at all?
3. I saw an issue which referred to the PerfTracker (#84), this object is being replaced with a more modular system: https://github.com/quantopian/zipline/pull/2081/files#diff-b4d729da14c8129685e521fc0953f8ddR26.

Note: The new metric tracker is probably much easier to serialize than the old perf tracker because it basically acts as the algorithm's interface to the Ledger object which is what maintains the portfolio and account.

Dear Zipline Maintainers,

Is there any plan to support live trading on Robinhood? Any ideas on the difficulty of implementing this on top of the current code base?

Sincerely,
Peichao

Hi,

PR 15 fails tests with below error. I suspect that this is caused by changes in 'live'. How come they've got merged if they're failing tests?

======================================================================
ERROR: test_panel_data_0_daily (tests.test_algorithm.TestPanelData)

Traceback (most recent call last):
File "/home/travis/miniconda/envs/testenv/lib/python2.7/site-packages/nose_parameterized/parameterized.py", line 365, in standalone_func
return func(*(a + p.args), **p.kwargs)
File "/home/travis/build/zipline-live/zipline/tests/test_algorithm.py", line 4750, in test_panel_data
data=panel
File "/home/travis/build/zipline-live/zipline/zipline/utils/run_algo.py", line 384, in run_algorithm
environ=environ,
TypeError: _run() takes exactly 20 arguments (18 given)

Regards,
Ed

Description of Issue

Pandas-datareader wants pandas>19.2, but pandas is specfied >=18.1, <0.19. This prevents pipenv from generating a pipfile.lock.

Probably the easiest fix is to remove the <0.19 specification on pandas, since pandas-datareader requires >19.2.

$ pipenv install zipline-live[ib] Could not find a version that matches pandas<0.19,>=0.16.1,>=0.18.1,>=0.19.2

`$ pipenv graph
zipline-live==1.1.0.5

• alembic [required: >=0.7.7, installed: 0.9.9]
  • Mako [required: Any, installed: 1.0.7]
    • MarkupSafe [required: >=0.9.2, installed: 1.0]
  • python-dateutil [required: Any, installed: 2.7.3]
    • six [required: >=1.5, installed: 1.11.0]
  • python-editor [required: >=0.3, installed: 1.0.3]
  • SQLAlchemy [required: >=0.7.6, installed: 1.2.8]
• bcolz [required: <1,>=0.12.1, installed: 0.12.1]
  • numpy [required: >=1.7, installed: 1.14.5]
• bottleneck [required: >=1.0.0, installed: 1.2.1]
  • numpy [required: Any, installed: 1.14.5]
• click [required: >=4.0.0, installed: 6.7]
• contextlib2 [required: >=0.4.0, installed: 0.5.5]
• cyordereddict [required: >=0.2.2, installed: 1.0.0]
• Cython [required: >=0.25.2, installed: 0.28.3]
• decorator [required: >=4.0.0, installed: 4.3.0]
• empyrical [required: >=0.2.2, installed: 0.5.0]
  • bottleneck [required: >=1.0.0, installed: 1.2.1]
    • numpy [required: Any, installed: 1.14.5]
  • numpy [required: >=1.9.2, installed: 1.14.5]
  • pandas [required: >=0.16.1, installed: 0.18.1]
    • numpy [required: >=1.7.0, installed: 1.14.5]
    • python-dateutil [required: >=2, installed: 2.7.3]
      • six [required: >=1.5, installed: 1.11.0]
    • pytz [required: >=2011k, installed: 2018.4]
  • pandas-datareader [required: >=0.2, installed: 0.6.0]
    • lxml [required: Any, installed: 4.2.1]
    • pandas [required: >=0.19.2, installed: 0.18.1]
      • numpy [required: >=1.7.0, installed: 1.14.5]
      • python-dateutil [required: >=2, installed: 2.7.3]
        • six [required: >=1.5, installed: 1.11.0]
      • pytz [required: >=2011k, installed: 2018.4]
    • requests [required: >=2.3.0, installed: 2.19.1]
      • certifi [required: >=2017.4.17, installed: 2018.4.16]
      • chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4]
      • idna [required: >=2.5,<2.8, installed: 2.7]
      • urllib3 [required: <1.24,>=1.21.1, installed: 1.23]
    • requests-file [required: Any, installed: 1.4.3]
      • requests [required: >=1.0.0, installed: 2.19.1]
        • certifi [required: >=2017.4.17, installed: 2018.4.16]
        • chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4]
        • idna [required: >=2.5,<2.8, installed: 2.7]
        • urllib3 [required: <1.24,>=1.21.1, installed: 1.23]
      • six [required: Any, installed: 1.11.0]
    • requests-ftp [required: Any, installed: 0.3.1]
      • requests [required: Any, installed: 2.19.1]
        • certifi [required: >=2017.4.17, installed: 2018.4.16]
        • chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4]
        • idna [required: >=2.5,<2.8, installed: 2.7]
        • urllib3 [required: <1.24,>=1.21.1, installed: 1.23]
    • wrapt [required: Any, installed: 1.10.11]
  • scipy [required: >=0.15.1, installed: 1.1.0]
    • numpy [required: >=1.8.2, installed: 1.14.5]
• intervaltree [required: >=2.1.0, installed: 2.1.0]
  • sortedcontainers [required: Any, installed: 2.0.4]
• Logbook [required: >=0.12.5, installed: 1.4.0]
• lru-dict [required: >=1.1.4, installed: 1.1.6]
• Mako [required: >=1.0.1, installed: 1.0.7]
  • MarkupSafe [required: >=0.9.2, installed: 1.0]
• MarkupSafe [required: >=0.23, installed: 1.0]
• multipledispatch [required: >=0.4.8, installed: 0.5.0]
• networkx [required: >=1.9.1,<2.0, installed: 1.11]
  • decorator [required: >=3.4.0, installed: 4.3.0]
• numexpr [required: >=2.6.1, installed: 2.6.5]
  • numpy [required: >=1.7, installed: 1.14.5]
• numpy [required: >=1.11.1, installed: 1.14.5]
• pandas [required: >=0.18.1,<0.19, installed: 0.18.1]
  • numpy [required: >=1.7.0, installed: 1.14.5]
  • python-dateutil [required: >=2, installed: 2.7.3]
    • six [required: >=1.5, installed: 1.11.0]
  • pytz [required: >=2011k, installed: 2018.4]
• pandas-datareader [required: >=0.2.1, installed: 0.6.0]
  • lxml [required: Any, installed: 4.2.1]
  • pandas [required: >=0.19.2, installed: 0.18.1]
    • numpy [required: >=1.7.0, installed: 1.14.5]
    • python-dateutil [required: >=2, installed: 2.7.3]
      • six [required: >=1.5, installed: 1.11.0]
    • pytz [required: >=2011k, installed: 2018.4]
  • requests [required: >=2.3.0, installed: 2.19.1]
    • certifi [required: >=2017.4.17, installed: 2018.4.16]
    • chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4]
    • idna [required: >=2.5,<2.8, installed: 2.7]
    • urllib3 [required: <1.24,>=1.21.1, installed: 1.23]
  • requests-file [required: Any, installed: 1.4.3]
    • requests [required: >=1.0.0, installed: 2.19.1]
      • certifi [required: >=2017.4.17, installed: 2018.4.16]
      • chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4]
      • idna [required: >=2.5,<2.8, installed: 2.7]
      • urllib3 [required: <1.24,>=1.21.1, installed: 1.23]
    • six [required: Any, installed: 1.11.0]
  • requests-ftp [required: Any, installed: 0.3.1]
    • requests [required: Any, installed: 2.19.1]
      • certifi [required: >=2017.4.17, installed: 2018.4.16]
      • chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4]
      • idna [required: >=2.5,<2.8, installed: 2.7]
      • urllib3 [required: <1.24,>=1.21.1, installed: 1.23]
  • wrapt [required: Any, installed: 1.10.11]
• patsy [required: >=0.4.0, installed: 0.5.0]
  • numpy [required: >=1.4, installed: 1.14.5]
  • six [required: Any, installed: 1.11.0]
• pip [required: >=7.1.0, installed: 10.0.1]
• python-dateutil [required: >=2.4.2, installed: 2.7.3]
  • six [required: >=1.5, installed: 1.11.0]
• pytz [required: >=2016.4, installed: 2018.4]
• requests [required: >=2.9.1, installed: 2.19.1]
  • certifi [required: >=2017.4.17, installed: 2018.4.16]
  • chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4]
  • idna [required: >=2.5,<2.8, installed: 2.7]
  • urllib3 [required: <1.24,>=1.21.1, installed: 1.23]
• requests-file [required: >=1.4.1, installed: 1.4.3]
  • requests [required: >=1.0.0, installed: 2.19.1]
    • certifi [required: >=2017.4.17, installed: 2018.4.16]
    • chardet [required: <3.1.0,>=3.0.2, installed: 3.0.4]
    • idna [required: >=2.5,<2.8, installed: 2.7]
    • urllib3 [required: <1.24,>=1.21.1, installed: 1.23]
  • six [required: Any, installed: 1.11.0]
• scipy [required: >=0.17.1, installed: 1.1.0]
  • numpy [required: >=1.8.2, installed: 1.14.5]
• setuptools [required: >18.0, installed: 39.2.0]
• six [required: >=1.10.0, installed: 1.11.0]
• sortedcontainers [required: >=1.4.4, installed: 2.0.4]
• sqlalchemy [required: >=1.0.8, installed: 1.2.8]
• statsmodels [required: >=0.6.1, installed: 0.9.0]
  • pandas [required: Any, installed: 0.18.1]
    • numpy [required: >=1.7.0, installed: 1.14.5]
    • python-dateutil [required: >=2, installed: 2.7.3]
      • six [required: >=1.5, installed: 1.11.0]
    • pytz [required: >=2011k, installed: 2018.4]
  • patsy [required: Any, installed: 0.5.0]
    • numpy [required: >=1.4, installed: 1.14.5]
    • six [required: Any, installed: 1.11.0]
• tables [required: >=3.3.0, installed: 3.4.4]
  • numexpr [required: >=2.5.2, installed: 2.6.5]
    • numpy [required: >=1.7, installed: 1.14.5]
  • numpy [required: >=1.8.0, installed: 1.14.5]
  • six [required: >=1.9.0, installed: 1.11.0]
• toolz [required: >=0.8.2, installed: 0.9.0]
  `

A Zipline algo accesses historical and realtime data through DataPortal (via data.current property in handle_data()). This property calls DataPortal.get_spot_value() DataPortal.get_adjusted_value()
to get the latest prices.

In live trading we'd like to get the latest data from the Broker, therefore a DataPortalLive class shall be created (based on DataPortal), which reaches out the BrokerClient to obtain the latest pricing data.

Depends on: #4

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System: (Windows Version 10)
• Python Version: $ python --version 2.7
• Python Bitness: $ python -c 'import math, sys;print(int(math.log(sys.maxsize + 1, 2) + 1))'
• How did you install Zipline: (pip)
• Python packages: $ pip freeze or $ conda list
  alabaster==0.7.12
  alembic==1.0.7
  anaconda-client==1.7.2
  anaconda-navigator==1.8.4
  anaconda-project==0.8.2
  asn1crypto==0.24.0
  astroid==1.6.5
  astropy==0.4.1
  atomicwrites==1.2.1
  attrs==18.2.0
  Babel==2.6.0
  backports-abc==0.5
  backports.functools-lru-cache==1.5
  backports.os==0.1.1
  backports.shutil-get-terminal-size==1.0.0
  backports.shutil-which==3.5.2
  backports.ssl-match-hostname==3.5.0.1
  bcolz==0.12.1
  beautifulsoup4==4.6.3
  bitarray==0.8.3
  bkcharts==0.2
  blaze==0.10.1
  bleach==3.0.2
  bokeh==0.13.0
  boto==2.49.0
  Bottleneck==1.2.1
  cdecimal==2.3
  certifi==2018.11.29
  cffi==1.11.5
  chardet==3.0.4
  chest==0.2.3
  Click==7.0
  cloudpickle==0.6.1
  clyent==1.2.2
  colorama==0.4.1
  comtypes==1.1.7
  conda==4.6.2
  conda-build==3.17.6
  conda-verify==3.1.1
  configparser==3.5.0
  contextlib2==0.5.5
  cryptography==2.4.2
  cycler==0.10.0
  cyordereddict==1.0.0
  Cython==0.29.2
  cytoolz==0.9.0.1
  dask==1.0.0
  datashape==0.5.4
  decorator==4.3.0
  defusedxml==0.5.0
  distributed==1.25.1
  docutils==0.14
  empyrical==0.5.0
  entrypoints==0.2.3
  enum34==1.1.6
  et-xmlfile==1.0.1
  fastcache==1.0.2
  filelock==3.0.10
  Flask==1.0.2
  Flask-Cors==3.0.7
  funcsigs==1.0.2
  functools32==3.2.3.post2
  future==0.17.1
  futures==3.2.0
  gevent==1.3.7
  glob2==0.6
  greenlet==0.4.15
  grin==1.2.1
  h5py==2.8.0
  heapdict==1.0.0
  html5lib==1.0.1
  IbPy2==0.8.0
  idna==2.8
  imageio==2.4.1
  imagesize==1.1.0
  importlib-metadata==0.6
  intervaltree==3.0.2
  ipaddress==1.0.22
  ipykernel==4.10.0
  ipython==5.8.0
  ipython-genutils==0.2.0
  ipywidgets==7.4.2
  isort==4.3.4
  itsdangerous==1.1.0
  jdcal==1.4
  jedi==0.13.2
  Jinja2==2.10
  jsonschema==2.6.0
  jupyter==1.0.0
  jupyter-client==5.2.4
  jupyter-console==5.2.0
  jupyter-core==4.4.0
  jupyterlab==0.33.11
  jupyterlab-launcher==0.11.2
  keyring==17.0.0
  kiwisolver==1.0.1
  lazy-object-proxy==1.3.1
  libarchive-c==2.8
  linecache2==1.0.0
  llvmlite==0.26.0
  llvmpy==0.12.7
  locket==0.2.0
  Logbook==1.4.3
  lru-dict==1.1.6
  lxml==4.2.5
  Mako==1.0.7
  MarkupSafe==1.1.0
  matplotlib==1.4.0
  mccabe==0.6.1
  menuinst==1.4.14
  mistune==0.8.4
  mkl-fft==1.0.6
  more-itertools==4.3.0
  mpmath==1.1.0
  msgpack==0.5.6
  multipledispatch==0.6.0
  navigator-updater==0.2.0
  nbconvert==5.4.0
  nbformat==4.4.0
  networkx==1.11
  nltk==3.4
  nose==1.3.7
  notebook==5.7.4
  numba==0.13.4
  numexpr==2.6.9
  numpy==1.16.1
  numpydoc==0.8.0
  odo==0.4.2
  olefile==0.46
  openpyxl==2.5.12
  packaging==18.0
  pandas==0.18.1
  pandas-datareader==0.7.0
  pandocfilters==1.4.2
  parso==0.3.1
  partd==0.3.9
  path.py==11.5.0
  pathlib2==2.3.3
  patsy==0.5.1
  pep8==1.7.1
  pickleshare==0.7.5
  Pillow==3.0.0
  pkginfo==1.4.2
  pluggy==0.8.0
  ply==3.11
  prometheus-client==0.5.0
  prompt-toolkit==1.0.15
  psutil==5.4.8
  py==1.7.0
  pycodestyle==2.4.0
  pycosat==0.6.3
  pycparser==2.19
  pycrypto==2.6.1
  pycurl==7.43.0.2
  pyflakes==2.0.0
  Pygments==2.3.1
  pylint==1.9.2
  pyodbc==4.0.25
  pyOpenSSL==18.0.0
  pyparsing==2.0.1
  PySocks==1.6.8
  pytest==4.0.2
  python-dateutil==2.7.5
  python-editor==1.0.4
  pytz==2018.7
  pywin32==223
  pywinpty==0.5.5
  PyYAML==3.13
  pyzmq==17.1.2
  QtAwesome==0.5.3
  qtconsole==4.4.3
  QtPy==1.5.2
  requests==2.21.0
  requests-file==1.4.3
  rope==0.11.0
  ruamel-yaml==0.15.46
  scandir==1.9.0
  scikit-image==0.10.1
  scikit-learn==0.15.2
  scipy==1.2.0
  seaborn==0.8.1
  Send2Trash==1.5.0
  simplegeneric==0.8.1
  singledispatch==3.4.0.3
  six==1.12.0
  snowballstemmer==1.2.1
  sortedcollections==1.0.1
  sortedcontainers==2.1.0
  Sphinx==1.8.2
  sphinxcontrib-websupport==1.1.0
  spyder==3.2.8
  spyder-kernels==0.3.0
  SQLAlchemy==1.2.15
  statsmodels==0.9.0
  subprocess32==3.5.3
  sympy==1.3
  TA-Lib==0.4.8
  tables==3.4.4
  tblib==1.3.2
  terminado==0.8.1
  testpath==0.4.2
  toolz==0.9.0
  tornado==5.1.1
  tqdm==4.28.1
  traceback2==1.4.0
  traitlets==4.3.2
  typing==3.6.6
  unicodecsv==0.14.1
  unittest2==1.1.0
  urllib3==1.24.1
  virtualenv==16.3.0
  wcwidth==0.1.7
  webencodings==0.5.1
  Werkzeug==0.14.1
  widgetsnbextension==3.4.2
  win-inet-pton==1.0.1
  win-unicode-console==0.5
  wincertstore==0.2
  wrapt==1.10.11
  xlrd==1.2.0
  XlsxWriter==1.1.2
  xlwings==0.15.1
  xlwt==1.3.0
  zict==0.1.3
  zipline==0.7.0
  zipline-live==1.1.0.5

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

• What did you expect to happen?
  I expected that it would run the standard --help command and show me all the internal commands inside this zipline-live, but it didn't and it shows this error. I tried to change it in the code, but then nothing shows up at all.
• What happened instead?
  Zipline-live crashed instead of showing the zipline --help after successful installation.
  Here is how you can reproduce this issue on your machine:

Reproduction Steps

1. - Follow these steps with Anaconda Python.
2. install all dependencies of zipline-live.
3. run zipline --help inside your anaconda terminal
   ...

What steps have you taken to resolve this already?

I've tried to debug the code, but my debugging was unsuccessful.

Anything else?

Sincerely,
Shamil Karimli

Pipeline is one of the key components to Zipline, allowing calculations to be done over thousands of equities. (The current universe of North American equities available on Quantopian is over 8000.) However there is not a yet a working version of Pipeline in zipline-live.

The initial testing of zipline-live is done with Interactive Brokers as the broker. Receiving live updates of 8k+ tickers from the IB would be prohibitively expensive, the default max market data lines from IB is 100.

There is a straight-forward path to enabling Pipeline use with Interactive Brokers for daily data and that is through Zipline bundles. Each day of trading would require the following steps:

1. historical data is loaded from IB's historical data into a Zipline bundle
2. trading decisions are made based off of the data in the bundle and orders are routed to IB

Users are given the freedom and flexibility to control what exchanges and tickers they want included in the universe. This brings up the question: does the user need to know all the tickers for each exchange a priori? Wouldn't hard-coding the tickers introduce survivorship bias? To properly address these questions the data loader can first download a list of all the available tickers in an exchange, then get historical data for that exchange/ticker from IB. In this scheme the user only needs to supply a list of exchanges they wish to trade.

The above scheme naturally separates the following concerns:

• List_Tickers_For_Exchange
  --Input: a list of exchanges to find available tickers from
  --Output: a list of exchange, ticker tuples

• Populate_Bundle_With_Historical_Data
  --Input: a list of exchange, ticker tuples
  --Output: none
  --Side Effects: updates or creates Zipline bundle with historical OHLCV data.

If a user wanted a handful of known tickers that could be hardcoded or read from a file. If a user wanted to use another broker, Populate_Bundle_With_Historical_Data would simply need to be replaced. Populate_Bundle_With_Historical_Data probably should be an abstract class with the IB implementation in: Populate_Bundle_With_Historical_Data_From_IB or something along those lines.

Zipline-Live will support live data feed from Brokers. To match with Quantopian's
original design the algorithms will access the live feed through data.current().

As it is important to have reproducible runs for the algorithms it would be beneficial
to store (or continuously ingest) the live feed. Data is precious.

Live feed storage should be transparent to the algo and broker agnostic.

In order to have CI for the project an admin needs to:

• Sign up for Travis using zipline-live (done)
• Enable builds for branch 'live' in Travis

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

• Operating System: Arch Linux 4.15.6-1-ARCH
• Python Version: Python 3.6.4
• Python Bitness: 64
• How did you install Zipline: pip
• Python packages:
  aiodns==1.1.1
  aiohttp==3.0.1
  alembic==0.9.8
  appdirs==1.4.3
  async-timeout==2.0.0
  attrdict==2.0.0
  attrs==17.4.0
  bcolz==0.12.1
  beautifulsoup4==4.6.0
  bleach==2.1.2
  boto3==1.5.31
  botocore==1.8.45
  Bottleneck==1.2.1
  bs4==0.0.1
  cchardet==2.1.1
  ccxt==1.10.1138
  certifi==2018.1.18
  chardet==3.0.4
  chrome-gnome-shell==0.0.0
  click==6.7
  click-plugins==1.0.3
  cligj==0.4.0
  contextlib2==0.5.5
  cycler==0.10.0
  cyordereddict==1.0.0
  Cython==0.27.3
  cytoolz==0.9.0
  decorator==4.2.1
  descartes==1.1.0
  docutils==0.14
  empyrical==0.4.0
  enigma-catalyst==0.5.3
  entrypoints==0.2.3
  enum34==1.1.6
  eth-abi==1.0.0b0
  eth-account==0.1.0a2
  eth-keyfile==0.5.1
  eth-keys==0.2.0b1
  eth-rlp==0.1.0a2
  eth-utils==1.0.0b1
  Fiona==1.7.11
  geopandas==0.3.0
  googlemaps==2.5.1
  hexbytes==0.1.0b0
  html5lib==1.0.1
  IbPy2==0.8.0
  idna==2.6
  idna-ssl==1.0.0
  intervaltree==2.1.0
  ipykernel==4.6.1
  ipython==6.2.1
  ipython-autotime==0.1
  ipython-genutils==0.1.0
  ipywidgets==6.0.0
  jedi==0.11.1
  Jinja2==2.10
  jmespath==0.9.3
  joblib==0.11
  jsonschema==2.6.0
  jupyter-client==5.2.2
  jupyter-console==5.2.0
  jupyter-core==4.4.0
  Keras==2.1.2
  Logbook==1.2.1
  lru-dict==1.1.6
  lxml==4.1.1
  Mako==1.0.7
  Markdown==2.6.10
  MarkupSafe==1.0
  matplotlib==2.1.1
  mistune==0.8.1
  multidict==4.1.0
  multipledispatch==0.4.9
  munch==2.2.0
  nbconvert==5.3.1
  nbformat==4.4.0
  networkx==2.1
  notebook==5.3.0
  numexpr==2.6.4
  numpy==1.13.3
  packaging==16.8
  pandas==0.18.1
  pandas-datareader==0.6.0
  pandocfilters==1.4.2
  parso==0.1.1
  patsy==0.5.0
  pexpect==4.3.1
  pickleshare==0.7.4
  prompt-toolkit==1.0.15
  protobuf==3.5.1
  ptyprocess==0.5.2
  pycairo==1.16.2
  pycares==2.3.0
  pycryptodome==3.4.11
  pygame==1.9.3
  Pygments==2.2.0
  pygobject==3.26.1
  pyparsing==2.2.0
  pyproj==1.9.5.1
  pysha3==1.0.2
  python-dateutil==2.6.1
  python-editor==1.0.3
  pytz==2017.3
  PyYAML==3.12
  pyzmq==16.0.3
  redo==1.6
  requests==2.18.4
  requests-file==1.4.3
  requests-ftp==0.3.1
  requests-toolbelt==0.8.0
  rlp==0.6.0
  s3transfer==0.1.13
  sas7bdat==2.0.7
  scikit-learn==0.19.1
  scipy==1.0.0
  seaborn==0.8.1
  Send2Trash==1.5.0
  Shapely==1.6.3
  simplegeneric==0.8.1
  six==1.11.0
  sklearn==0.0
  sortedcontainers==1.5.9
  SQLAlchemy==1.2.3
  statsmodels==0.8.0
  TA-Lib==0.4.16
  tables==3.4.2
  team==1.0
  tensorflow==1.4.1
  tensorflow-tensorboard==0.4.0rc3
  terminado==0.8.1
  testpath==0.3.1
  toolz==0.9.0
  tornado==4.5.3
  traitlets==4.3.2
  urllib3==1.22
  virtualenv==15.1.0
  wcwidth==0.1.7
  web3==4.0.0b9
  webencodings==0.5.1
  Werkzeug==0.13
  widgetsnbextension==2.0.0
  wrapt==1.10.11
  xgboost==0.7.post3
  yarl==1.1.0
  zipline==1.1.1
  zipline-live==1.1.0.4

Now that you know a little about me, let me tell you about the issue I am
having:

I fellow the tutorial http://www.zipline-live.io/tutorial with a IB demo account, but when I run the command:
zipline run -f ~/zipline-algos/demo.py --state-file ~/zipline-algos/demo.state --realtime-bar-target ~/zipline-algos/realtime-bars/ --broker ib --broker-uri localhost:7496:1232 --bundle quantopian-quandl --data-frequency minute
I receive the message:

"Requested market data is not subscribed. Delayed market data is not enabled"

So I am not able to see the AAPL prices.
If it can help, I am able to sell and buy stocks.

Finally I also tried to replicate the procedure on a Ubuntu virtual machine, but I get the same error.

Thanks

Zipline uses blotter class to interface with backtesting or normal brokers.

Currently in zipline-live blotter is unused: TradingAlgorithm's respective calls are
overridden to reach out the broker directly. This approach prevents zipline to keep
track of positions in the performance report.
Additionally, if we want to run backtest & live trading at the same time (e.g.: to enable dynamic activation of the strategy based on it's previous performance) the current approach won't work.

This ticket tracks the efforts of migrating to blotter_live.

The brokers are dynamically imported in zipline-live based on the value of --broker command line parameter. If a nonexistent broker is specified an error is presented to the user:

The following error is also presented if the broker is supported, but its dependencies are not present. We should improve the error message in that case and notify the user about the dependency problem.

Steps to reproduce:

• Install based on the tutorial
• Uninstall IbPy2 from virtualenv
• Start zipline-live with ib broker
• Observe the error

In backtest and Quantopian real trading the algo's initialize method called only once: at the beginning of the execution.
It is required that live trading does not diverge from backtest with regards of execution (i.e. initialize is called only once).
It is also important to implement a robust live trading base which survives network outages & restarts.

Therefore state must be introduced to live-trading code path which is able to store the actual state ({{context}} variable, visible in initialize() & handle_data()) in disk. The logic to implement is simple:

• During startup check if state file (pickle?) exists for the algo
  • If not: call algo's initialize() and store context to state file
  • Else: load context from state file
• Save state file after every call to handle_data()

When one calls now data.current the ib_broker will subscribe to the data. This subscription process can take longer then a minute when you have several tickers and therefor the handle_data will not function properly. Can be solved by pre-subscribing to the data in before_trading_starts

Dear Zipline Maintainers,
From the zipline-live.io tutorial:

Installation of zipline-live works fine. Downloading the quandl bundle works fine as well. However, when I try to run the zipline-live demo program, with the following command:

zipline run -f ~/zipline-algos/demo.py --broker ib --broker-uri localhost:7496:1232 --bundle quantopian-quandl  --data-frequency minute

I get an error.

Environment

• Operating System: Darwin MacBook-Pro.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64 i386 MacBookPro13,3 Darwin
• Python Version: Python 2.7.10
• Python Bitness: 64
• How did you install Zipline: pip
• Python packages:

alembic==0.9.5
bcolz==0.12.1
Bottleneck==1.2.1
certifi==2017.7.27.1
chardet==3.0.4
click==6.7
contextlib2==0.5.5
cyordereddict==1.0.0
Cython==0.26
decorator==4.1.2
empyrical==0.3.2
idna==2.6
intervaltree==2.1.0
Logbook==1.1.0
lru-dict==1.1.6
Mako==1.0.7
MarkupSafe==1.0
multipledispatch==0.4.9
networkx==1.11
numexpr==2.6.2
numpy==1.13.1
pandas==0.18.1
pandas-datareader==0.5.0
patsy==0.4.1
python-dateutil==2.6.1
python-editor==1.0.3
pytz==2017.2
requests==2.18.4
requests-file==1.4.2
requests-ftp==0.3.1
scipy==0.19.1
six==1.10.0
sortedcontainers==1.5.7
SQLAlchemy==1.1.13
statsmodels==0.8.0
tables==3.4.2
TBB==0.1
toolz==0.8.2
urllib3==1.22
zipline-live==0.1.0.0

Now that you know a little about me, let me tell you about the issue I am
having:

Description of Issue

[box@MacBook-Pro] ~
❯ zipline run -f ~/zipline-algos/demo.py --broker ib --broker-uri localhost:7496:1232 --bundle quantopian-quandl  --data-frequency minute

Usage: zipline run [OPTIONS]

Error: unsupported broker: can't import module zipline.gens.brokers.ib_broker

• What did you expect to happen?
  I expected to run zipline-live correctly.
• What happened instead?
  The error above occured

Here is how you can reproduce this issue on your machine:

Reproduction Steps

virtualenv ~
source ~/venv-zipline-live/bin/activate
pip install zipline-live[ib]

2. zipline ingest -b quantopian-quandl

zipline run -f ~/zipline-algos/demo.py --broker ib --broker-uri localhost:7496:1232 --bundle quantopian-quandl  --data-frequency minute

...

What steps have you taken to resolve this already?

N/A
...

Anything else?

N/A

Dear Zipline Maintainers,

There is an open problem with WIKI prices dataset. The thing is that WIKI prices is not being updated for several days, the last pricing data there is for 28th of February, 2018.

I emailed the quandl's support and here is their answer:
"Thanks for bringing this to our attention.

Your email has been forwarded to our data team so that this can be added to their development queue.

Please kindly note that as a free database, there is no guaranteed timeline by when our team will be able to investigate this issue. And, there is no guaranteed timeline by when they will be able to apply updates or corrections to the database.

If you need more reliable data, we recommend the following premium databases:
EOD: https://www.quandl.com/data/EOD
SEP: https://www.quandl.com/databases/SEP"

The presence of recent pricing data is not that important when backtesting strategies,
but very essential when trading live. I think most of zipline-live users would consider to use payed data for live trading, so I propose to include a loader for one of the quandl's premium databases as an option, to make zipline-live to support reliable data out of the box

Sincerely,
Vitaly