failed catalyst run leaves empty file in /var/tmp/catalyst/data/poloniex/crypto_prices-USDT_BTC.csv about catalyst HOT 5 CLOSED

Adding more information about this issue:
the file that holds the benchmark (~/.catalyst/data/USDT_BTC_benchmark.csv) is empty

from catalyst.

Exact same issue here. I believe Zipline (which this is based on) had a similar issue when I tried to use it - due to either Google Finance or Yahoo Finance shutting down an API?

from catalyst.

This issue appears to be caused by failure of previous data handling ops. It can be worked around with the following procedure.

1. Must use Python version 2.7.x due to some compatibility issues in file opens in curate/poloniex.py it appears.
2. rm -rf /var/tmp/catalyst
3. rm -rf ~/.catalyst
4. catalyst ingest
5. catalyst run ...

After chasing this around a bit, it seems that we are only checking for existence of files rather than whether the file has valid content before moving to next step of data curation.

from catalyst.

Changed the title of this issue since I have now recreated it on Linux.

If a catalyst run fails, it may leave an empty file in /var/tmp/catalyst/data/poloniex/crypto_prices-USDT_BTC.csv.

This empty file will prevent any future runs from succeeding. This appears to be because we are not handling an existing empty file error condition.

from catalyst.

Addressed in commit a01bcd5, which is part of 0.1.dev7 release: 20a98a3 in the master branch.

from catalyst.