Giter Club home page Giter Club logo

qdownload's Introduction

qdownload

qdownload is a command line IQFeed CSV market data download tool that can download EOD bars, minute bars or tick data.

Use it to download a list of symbols, or a large number of symbols stored in a text file.

Features

  • Daily bars
  • Minute bars
  • Interval bars (volume, ticks or seconds)
  • Tick data
  • Parallel downloads (8 by default)
  • CSV (default) or TSV format
  • Uncompressed (default) or GZipped files
  • Start and end date filter (all data by default)
  • Bars timestamps at start of bar (default), or end of bar
  • Optional time zone conversion of timestamps

Requirements

Installation

Use go get and install to download and compile qdownload:

go get -u github.com/nhedlund/qdownload
go install github.com/nhedlund/qdownload

Usage

Run qdownload without any arguments for usage:

$ qdownload
USAGE:
   qdownload [global options] command [command options] <symbols or symbols file>

COMMANDS:
     eod       Download EOD bars
     minute    Download minute bars
     tick      Download tick data
     interval  Download interval bars: <length> <seconds|volume|ticks>
     help, h   Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --start value, -s value        start date filter: yyyymmdd
   --end value, -e value          end date filter: yyyymmdd
   --out value, -o value          output directory (default: "data")
   --timezone value, -z value     timestamps time zone (default: "ET")
   --parallelism value, -p value  number of parallel downloads (default: 8)
   --tsv, -t                      use tab separator instead of comma
   --detailed-logging, -d         detailed log output
   --gzip, -g                     compress files with gzip
   --end-timestamp, -m            use end of bar timestamps instead of start
   --help, -h                     show help

Download Apple and Microsoft stock price minute bars from 2007 to today:

$ qdownload minute aapl,msft
   • Read symbols              symbols=2
   • Downloading               symbol=AAPL
   • Downloading               symbol=MSFT
   • Completed                 duration=19177ms rows=1428612 symbol=MSFT
   • Completed                 duration=43670ms rows=1853585 symbol=AAPL

Download daily bars for 10 different symbols from a list in symbols.txt:

$ qdownload eod symbols.txt
   • Read symbols              symbols=10
   • Downloading               symbol=FB
   • Downloading               symbol=SPY
   • Downloading               symbol=NFLX
   • Downloading               symbol=AMZN
   • Downloading               symbol=GOOG
   • Downloading               symbol=AAPL
   • Downloading               symbol=GLD
   • Downloading               symbol=SLV
   • Completed                 duration=468ms rows=1238 symbol=GOOG
   • Downloading               symbol=USO
   • Completed                 duration=473ms rows=1703 symbol=FB
   • Downloading               symbol=FARM
   • Completed                 duration=612ms rows=4219 symbol=NFLX
   • Completed                 duration=611ms rows=3229 symbol=SLV
   • Completed                 duration=624ms rows=3591 symbol=GLD
   • Completed                 duration=745ms rows=5828 symbol=SPY
   • Completed                 duration=769ms rows=7342 symbol=AAPL
   • Completed                 duration=876ms rows=5478 symbol=AMZN
   • Completed                 duration=604ms rows=3242 symbol=USO
   • Completed                 duration=608ms rows=5429 symbol=FARM

Download volume 1000 interval bars for SPY starting from 2019-04-18:

$ qdownload -s 20190418 interval 1000 volume spy
   • Using newer protocol required for bar start timestamps, requiring at least IQFeed 6.0
   • Read symbols              symbols=1
   • Downloading               symbol=SPY
   • Completed                 duration=1126ms rows=50104 symbol=SPY

Download volume 1000 interval bars for SPY with timestamp at end of bar, starting from 2019-04-18:

$ qdownload -m -s 20190418 interval 1000 volume spy
   • Read symbols              symbols=1
   • Downloading               symbol=SPY
   • Completed                 duration=1120ms rows=50359 symbol=SPY

Timezone support

By default intraday timestamps use the default IQFeed time zone US Eastern Time.

However it is possible to output timestamps in a different timezone.

Use the -z option to set the output timezone to either a shortcut alias or an IANA time zone from: Time Zones

Examples:

  • ET or EST for US Eastern Time (New York)
  • CT or CST for US Central Time (Chicago)
  • PT or PST for US Pacific Time (Los Angeles)
  • UTC
  • America/New_York
  • Europe/Stockholm

qdownload's People

Contributors

dependabot[bot] avatar nhedlund avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

qdownload's Issues

one hour shift when downloading with different timezones

problem

When downloading the same data from iqfeed with different timezones, e.g. -z UTC vs without option, there is a one hour time difference.

architecture

Ubuntu 20.04 LTS minimal, x86_64

steps to reproduce

qdownload -o EST --start 20210515 interval 300 seconds AAL
qdownload -z UTC -o UTC --start 20210515 interval 300 seconds AAL

and run attached python script.

sample output

Difference
TimedeltaIndex(['0 days 01:00:00', '0 days 01:00:00', '0 days 01:00:00',
                '0 days 01:00:00', '0 days 01:00:00', '0 days 01:00:00',
                '0 days 01:00:00', '0 days 01:00:00', '0 days 01:00:00',
                '0 days 01:00:00'],


               dtype='timedelta64[ns]', name='datetime', freq=None)

tz_investigation.zip

Request feature: Ability to specify interval bars other than time

Would really appreciate it if you could include the feature to specify the interval type, to download interval bars (eg. new bar every 100 shares traded).

eg. qdownload -v 100 aapl
would download a csv with a new OHLC row after every 100 shares traded

[IntervalType] - Optional - 's' (default) for time intervals in seconds, 'v' for volume intervals, 't' for tick intervals

It's already implemented in the IQFeed developer API for historical data.

Compile issues due to changes in urfave/cli

Currently building will fail with >v1 of urfave/cli. Quick fix is changing the import to "gopkg.in/urfave/cli.v1" without any other code changes.

But didn't want to send that in a pull in case you'd prefer to update the script instead. (I'm not a Go programmer for the most part...and I failed at a stab in updating it the v2/3.)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.