bullsignals / binance-downloader Goto Github PK

Python tool to download Binance Candlestick (k-line) data from REST API

License: MIT License

Python 100.00%

binance binance-api binance-exchange binance-crytpocurrency-exchange trading trading-bot trading-data candlesticks cryptocurrency cryptocurrencies binance-utils cryptocurrency-exchanges backtest backtesting-trading-strategies backtesting kline

binance-downloader's Introduction

Binance Downloader

Python tool to download Binance Candlestick (k-line) data from REST API

Instalation

LINUX

clone repository

$ git clone https://github.com/anson-vandoren/binance-downloader.git
$ cd binance-downloader

activate your virtual enviroment and install dependencies (using python >= 3.5).

$ source /path-to-virtual-env/bin/activate
$ pip3 install -r requirements.txt
$ flit install --symlink

WINDOWS

clone repository

$ git clone https://github.com/anson-vandoren/binance-downloader.git
$ cd binance-downloader

activate your virtual enviroment and install dependencies (using python >= 3.5).

$ source /path-to-virtual-env/bin/activate
$ pip install -r requirements.txt

To execute 'flit install --symlink' you need run command prompt as admin. You can do that by:

Go to start -> All Programs -> Acessories -> Right click on Command Prompt and select "Run as administrator"

Then go to project directory, activate you virtual environment e execute the command.

$ flit install --symlink

Now you can run the kline-binance command line. You don't need execute the command prompt as admin before do that.

Using the Command Line Interface

Help

$  kline-binance --help
usage: kline-binance [-h] [--start START] [--end END] [--output OUTPUT]
                     [--dtfmt DATE_FORMAT]
                     symbol interval

CLI for downloading Binance Candlestick (k-line) data in bulk

positional arguments:
  symbol               (Required) Binance symbol pair, e.g. ETHBTC
  interval             (Required) Frequency interval in minutes(m); hours(h);
                       days(d); weeks(w); months(M); All possibles values: 1m
                       3m 5m 15m 30m 1h 2h 4h 6h 8h 12h 1d 3d 1w 1M

optional arguments:
  -h, --help           show this help message and exit
  --start START        Start date to get data (inclusive). Format: yyyy/mm/dd
  --end END            End date to get data (exclusive). Format: yyyy/mm/dd
  --output OUTPUT      File name to write data. Default:
                       ./downloaded/timestamp_symbol_interval
  --dtfmt DATE_FORMAT  Format to use for dates (DMY, MDY, YMD, etc). Default:
                       YMD

Downloading data

$ kline-binance -i 1m -l 1500 -st 01/01/2016 -e 05/04/2018

License

This code is made available under the MIT License. See LICENSE file for detail.

binance-downloader's People

Contributors

Stargazers

Watchers

Forkers

lvcz devmmaia jsrikantappa juliopr hhefesto svince123 konqueror1 hs41-18 wangwang-io digi-fetch skanda-huayan citracahyati zhaoxiugao sos6548 00mjk xiekunyan-flow

binance-downloader's Issues

KeyError: 'limit'

$ kline-binance -i 1m -st 01/01/2016 -e 31/12/2016 -o ETHBTC-2016.csv
Traceback (most recent call last):
  File "/usr/local/bin/kline-binance", line 4, in <module>
    main()
  File "bullsignals/binance-downloader/binance/cli.py", line 38, in main
    if int(kwargs['limit']) > 500 and not (args.start and args.end):
KeyError: 'limit'

Can you make the --limit set by default?

Optimize processing for large requests

When downloading a large request (10k rows or more) the download is very time-consuming.

The README.md is not in english

The README.md is in portuguese but the default language for this project is english.

IndexError: list index out of range

I wanted to get all of the data, so I set a very high limit, but when I do I get the following error:

0%| | 219500/99999999999999 [03:05<21402766:26:33, 1297.86it/s]
Traceback (most recent call last):
File "\kline-binance", line 4, in
main()
File "\binance\cli.py", line 44, in main
binance.consult(output)
File "site-packages\binance\api.py", line 52, in consult
self.kwargs['startTime'] = update_start_time(self.klines[-1].open_time)
IndexError: list index out of range

Command Ran:
kline-binance -i 1m -s XRPBTC -l 99999999999999 -st 01/01/2017 -e 05/04/2018

error trying to run the requirements

Message from Bradley on Kaggle:

This didn't seem to work. I got an error trying to run the requirements:

Could not find a version that satisfies the requirement flit==0.13 (from -r requirements.txt (line 2)) (from versions: ) No matching distribution found for flit==0.13 (from -r requirements.txt (line 2))

db.py the csv writerows() puts EXTRA newline after each row

This may be a windows only issue, but to resolve this change line 19 in db.py:

FROM:
with open(output, 'a') as f:

TO:
with open(output, 'a', newline='') as f:

Create a data validation from downloaded data

The data downloaded from Binance will be used to test a trading strategy, it is important that:

There is no duplicated data.
There is no missing data.

Add progress bar

When downloading large quantity is not easy to understand if is finished or not or if have an error.

Fetch klines of multiple symbols

Hi,
That is really a nice project. Well done.

Is it possible to add functionality of downloading klines for multiple symbols at once?
Either by passing an array of symbols or perhaps some criteria such as top-x symbols by marketcap.

How much do you think Binance servers won't like such requests?

0%: download finished succesfully

The following command doesn't download anything:

$ kline-binance -i 1m -l 1000 -st 01/01/2016 -e 30/12/2016 -o ETHBTC-2016.csv
  0%|                                                                                                                                           | 0/1000 [00:00<?, ?it/s]All records downloaded for the provided date range.

download finished succesfully.