anfederico / clairvoyant Goto Github PK
View Code? Open in Web Editor NEWSoftware designed to identify and monitor social/historical cues for short term stock movement
License: MIT License
Software designed to identify and monitor social/historical cues for short term stock movement
License: MIT License
Thank you for the great work on the project! I'm really excited to test it out.
I downloaded data from five stocks through Yahoo finance and am attempting to use the model on them, but it only returns 0 buys and 0 sells no matter how I adjust the parameters.
I've attached my code, the output I see, and a sample of my data (abridged). I've been experimenting with different buy and sell thresholds trying to force it to sell, but I haven't been able to.
Code:
1 from clairvoyant import Backtest
2 from pandas import read_csv
3
4 # Testing performance on a single stock
5
6 variables = ["Close", "Volume"]
7 trainStart = '2012-05-21' # Start of training period
8 trainEnd = '2015-06-15' # End of training period
9 testStart = '2015-06-16' # Start of testing period
10 testEnd = '2017-02-9' # End of testing period
11 buyThreshold = 0.1 # Confidence threshold for predicting buy (default = 0.65)
12 sellThreshold = 0.1 # Confidence threshold for predicting sell (default = 0.65)
13 C = 1 # Penalty parameter (default = 1)
14 gamma = 10 # Kernel coefficient (default = 10)
15 continuedTraining = False # Continue training during testing period? (default = false)
16
17 backtest = Backtest(variables, trainStart, trainEnd, testStart, testEnd)
18
19 # Testing performance across multiple stocks
20
21 stocks = ["msft2", "fb2", "appl2", "amzn2", "goog2"]
22
23 for stock in stocks:
24 data = read_csv('Stocks/%s.csv' % stock)
25 data = data.round(3)
26 backtest.stocks.append(stock)
27 for i in range(0,10):
28 backtest.runModel(data)
29
30 backtest.displayConditions()
31 backtest.displayStats()
Output:
Conditions
X1: Close
X2: Volume
Buy Threshold: 65.0%
Sell Threshold: 65.0%
C: 1
gamma: 10
Continued Training: False
Stats
Stock(s):
msft2 | Training: 05/21/2012-06/15/2015 Testing: 06/16/2015-02/09/2017
fb2 | Training: 05/21/2012-06/15/2015 Testing: 06/16/2015-02/09/2017
appl2 | Training: 05/21/2012-06/15/2015 Testing: 06/16/2015-02/09/2017
amzn2 | Training: 05/21/2012-06/15/2015 Testing: 06/16/2015-02/09/2017
goog2 | Training: 05/21/2012-06/15/2015 Testing: 06/16/2015-02/09/2017
Total Buys: 0
Buy Accuracy: 0.0%
Total Sells: 0
Sell Accuracy: 0.0%
Sample data:
Date,Open,High,Low,Close,Volume,Adj Close
02/10/2012,183.419998,187.630005,182.520004,185.539993,5797100,185.539993
02/13/2012,187.169998,192.50,185.679993,191.589996,6071400,191.589996
02/14/2012,191.080002,193.570007,186.100006,191.300003,9527100,191.300003
02/15/2012,191.289993,191.539993,183.259995,184.470001,7794400,184.470001
02/16/2012,177.789993,181.679993,175.139999,179.929993,12563800,179.929993
02/17/2012,180.089996,183.410004,179.360001,182.50,7418100,182.50
Hi,
Some errors appeared unusually.
I used the example code to test environment. But, when I use the library, SyntaxError: invalid syntax
note appeared.
charmve@charmve-virtual-machine:~/Clairvoyant$ python test.py
Traceback (most recent call last):
File "test.py", line 1, in <module>
from clairvoyant.engine import Backtest
File "/home/charmve/.local/lib/python2.7/site-packages/clairvoyant/engine.py", line 16
self.svc = SVC(**kwargs, probability=True)
^
SyntaxError: invalid syntax
test.py
from clairvoyant.engine import Backtest
import pandas as pd
features = ["EMA", "SSO"] # Financial indicators of choice
trainStart = 0 # Start of training period
trainEnd = 700 # End of training period
testStart = 701 # Start of testing period
testEnd = 1000 # End of testing period
buyThreshold = 0.65 # Confidence threshold for predicting buy (default = 0.65)
sellThreshold = 0.65 # Confidence threshold for predicting sell (default = 0.65)
continuedTraining = False # Continue training during testing period? (default = false)
# Initialize backtester
backtest = Backtest(features, trainStart, trainEnd, testStart, testEnd, buyThreshold, sellThreshold, continuedTraining)
# A little bit of pre-processing
data = pd.read_csv("SBUX.csv", date_parser=['date'])
data = data.round(3)
# Start backtesting and optionally modify SVC parameters
# Available paramaters can be found at: http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html
backtest.start(data, kernel='rbf', C=1, gamma=10)
backtest.conditions()
backtest.statistics()
backtest.visualize('SBUX')
It would be great to have files with sample data included in repo.
It doesn't have to be real data, but it would be helpful to making Clairvoyant up and running with my own data (e.g. I don't know if it should be minute data, or daily OHLCV).
When trying to run your code I got the above error in this line:
backtest.start(data, kernel='rbf', C=1, gamma=10)
any idea why?
Hi Federico,
I am new to Python, and I am trying the second example , Simulate a Trading Strategy , and I am getting the following Error:
Traceback (most recent call last):
File "stock6.py", line 60, in
simulation = backtester.Simulation(features, trainStart, trainEnd, testStart, testEnd, buyThreshold, sellThreshold, continuedTraining)
NameError: name 'backtester' is not defined
If I try to edit it and use backtest.simulation I get the same error
NameError: name 'backtest' is not defined
Is there a workaround for this ?
When i run sample code, following error occured:
ImportError Traceback (most recent call last)
in ()
----> 1 from clairvoyant import Backtest
.\venv\Lib\site-packages\clairvoyant_init_.py in ()
----> 1 from Backtest import Backtest
.\venv\Lib\site-packages\clairvoyant\Backtest.py in ()
10 from numpy import vstack, hstack
11 from csv import DictWriter
---> 12 from pandas.tslib import relativedelta
13 from dateutil.parser import parse
My Pandas version : 0.14.1
Hi, where can I find your data 'SBUX.csv' in the example?
I got new info and entered it in the SBUX.csv file, but I keep getting this every time I run it:
File "test.py", line 17, in <module>
backtest = Backtest(variables, trainStart, trainEnd, testStart, testEnd)
File "/root/clair/clairvoyant/Backtest.py", line 47, in __init__
self.trainEnd = to_datetime(trainEnd)
File "/usr/local/lib/python2.7/dist-packages/pandas/util/decorators.py", line 91, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pandas/tseries/tools.py", line 430, in to_datetime
return _convert_listlike(np.array([arg]), box, format)[0]
File "/usr/local/lib/python2.7/dist-packages/pandas/tseries/tools.py", line 413, in _convert_listlike
raise e
ValueError: month must be in 1..12
I changed the date format and everything. Here is what my data looks like:
Date,Open,High,Low,Close,Volume
2-1-2015,12.33,12.84,11.87,12.67,221293
5-1-2015,12.57,13.34,12.19,13.02,335037
6-1-2015,13,13.45,12.29,12.8,369810
7-1-2015,12.95,13.23,12.05,13.1,561895
8-1-2015,13.19,14.07,12.91,14.05,539694
9-1-2015,13.98,14.2,13.39,13.61,626020
12-1-2015,13.7,14.25,13.39,13.52,176644
13-1-2015,13.7,14.17,12.94,13.21,457432
14-1-2015,13.06,14.18,12.1,14.17,310948
15-1-2015,14.19,14.5,13.41,13.44,486042
16-1-2015,13.38,13.87,13.21,13.54,283274
My name is Luis, I'm a big-data machine-learning developer, I'm a fan of your work, and I usually check your updates.
I was afraid that my savings would be eaten by inflation. I have created a powerful tool that based on past technical patterns (volatility, moving averages, statistics, trends, candlesticks, support and resistance, stock index indicators).
All the ones you know (RSI, MACD, STOCH, Bolinger Bands, SMA, DEMARK, Japanese candlesticks, ichimoku, fibonacci, williansR, balance of power, murrey math, etc) and more than 200 others.
The tool creates prediction models of correct trading points (buy signal and sell signal, every stock is good traded in time and direction). https://github.com/Leci37/LecTrade/tree/develop
For this I have used big data tools like pandas python, stock technical patterns market libraries like: tablib, TAcharts ,pandas_ta... For data collection and calculation.
And powerful machine-learning libraries such as: Sklearn.RandomForest , Sklearn.GradientBoosting, XGBoost, Google TensorFlow and Google TensorFlow LSTM.
With the models trained with the selection of the best technical indicators, the tool is able to predict trading points (where to buy, where to sell) and send real-time alerts to Telegram or Mail. The points are calculated based on the learning of the correct trading points of the last 2 years (including the change to bear market after the rate hike).
I think it could be useful to you, to improve, I would like to share it with you, and if you are interested in improving and collaborating we could, who knows how to make beautiful things.
Thank you for your time
I'm sorry to contact you here ,by issues, I don't know how I would be able to do it.
mail : [email protected] or https://github.com/Leci37/stocks-Machine-learning-RealTime-telegram/discussions
I m trying to run the example from your instructions. I am getting the error:
ImportError: No module named 'backtester'
Kindly, let me know what is the cause of this?
Is there any specific reason that the required modules are so outdated and strict (==
vs >=
in requirements.txt
)? This makes it tough to use in existing projects.
Anthony,
I suspect there may be an upstream bug in SVC.predict_proba
. I believe it outputs the classifier probabilities in reverse order than what is stated in their documentation. I discovered this when setting my training period equal to my testing period. I would expect that you should get highly accurate buy and sells if you do this. However,
Note: I do not have a lot of data so I'm breaking up the modeling frequency into 30 minute intervals.
Traceback (most recent call last):
File "C:/Users/14753/OneDrive/Desktop/stock.py", line 17, in
data = pd.read_csv("SBUX.csv", date_parser=['date'])
File "C:\Users\14753\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 702, in parser_f
return _read(filepath_or_buffer, kwds)
File "C:\Users\14753\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 429, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "C:\Users\14753\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 895, in init
self._make_engine(self.engine)
File "C:\Users\14753\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 1122, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "C:\Users\14753\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pandas\io\parsers.py", line 1853, in init
self._reader = parsers.TextReader(src, **kwds)
File "pandas_libs\parsers.pyx", line 387, in pandas._libs.parsers.TextReader.cinit
File "pandas_libs\parsers.pyx", line 705, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: [Errno 2] File b'SBUX.csv' does not exist: b'SBUX.csv'
Having this error, whats the problem
I have been trying to run through your Demo and I am stuck on the Portfolio Simulation section, first time I've used Python so probably doing something wrong somewhere. I got the first Demo working and it was returning.
RELIANCE | Training: 02/01/2013-07/15/2015 Testing: 07/16/2015-07/15/2016
Total Buys: 3690
Buy Accuracy: 50.81%
Total Sells: 0
Sell Accuracy: 0.0%
With the Portfolio one I only get back
Run #1 Buying Power: $1000000.0 Shares: 0 Total Value: $1000000.0 Run #2 Buying Power: $1000000.0 Shares: 0 Total Value: $1000000.0 Run #3 Buying Power: $1000000.0 Shares: 0 Total Value: $1000000.0 Run #4 Buying Power: $1000000.0 Shares: 0 Total Value: $1000000.0 Run #5 Buying Power: $1000000.0 Shares: 0 Total Value: $1000000.0 Performance across all runs Runs: 5 Average Performance: 0.0%
Here is my Code.
from clairvoyant import Portfolio
from pandas import read_csv
from nsepy import get_history
from datetime import date
variables = ["Open", "Close", "High", "Low", "Volume"] # Financial indicators of choice # "SSO", "SSC"
trainStart = '2013-03-01' # Start of training period
trainEnd = '2015-07-15' # End of training period
testStart = '2015-07-16' # Start of testing period
testEnd = '2016-07-16' # End of testing period
buyThreshold = 0.65 # Confidence threshold for predicting buy (default = 0.65)
sellThreshold = 0.65 # Confidence threshold for predicting sell (default = 0.65)
C = 1 # Penalty parameter (default = 1)
gamma = 10 # Kernel coefficient (default = 10)
continuedTraining = False # Continue training during testing period? (default = false)
startingBalance = 1000000 # Starting balance of portfolio
#User defined trading logic (see below)
def buyLogic(self, confidence, data, testDay):
quote = data["Close"][testDay] # Leave as is
if confidence >= 0.2: # If model signals buy
shareOrder = int((self.buyingPower*0.3)/quote) # and is 75-100% confident
self.buyShares(shareOrder, quote) # invest 30% of buying power
elif confidence >= 0.1: # If model is 70-75% confident
shareOrder = int((self.buyingPower*0.2)/quote) # invest 20% of buying power
self.buyShares(shareOrder, quote)
elif confidence >= 0.05: # If model is 65-70% confident
shareOrder = int((self.buyingPower*0.1)/quote) # invest 10% of buying power
self.buyShares(shareOrder, quote)
def sellLogic(self, confidence, data, testDay):
quote = data["Close"][testDay]
if confidence >= 0.65: # If model signals sell
self.sellShares(self.shares, quote) # and is 65-100% confident
# sell all shares
def nextDayLogic(self, prediction, nextDayPerformance, data, testDay):
quote = data["Close"][testDay]
# Case 1: Prediction is buy, price increases
if prediction == 1 and nextDayPerformance > 0:
if nextDayPerformance >= 0.025: # If I bought shares
self.sellShares(self.shares, quote) # and price increases >= 2.5%
# sell all shares
# Case 2: Prediction is buy, price decreases
elif prediction == 1 and nextDayPerformance <= 0: pass
# If I bought shares
# and price decreases
# hold position
# Case 3: Prediction is sell, price decreases
elif prediction == -1 and nextDayPerformance <= 0:
if nextDayPerformance <= -0.025: # If I sold shares
shareOrder = int((self.buyingPower*0.2)/quote) # and price decreases >= 2.5%
self.buyShares(shareOrder, quote) # reinvest 20% of buying power
# Case 4: Prediction is sell, price increases
elif prediction == -1 and nextDayPerformance > 0: pass
# If I sold shares
# and price increases
# hold position
# Case 5: No confident prediction was made
portfolio = Portfolio(variables, trainStart, trainEnd, testStart, testEnd)
history = get_history(symbol="RELIANCE",
start=date(2013, 1, 1),
end=date(2017, 1, 1),
index=False)
history[['Open','High','Low','Close','Volume','Close']].to_csv("RELIANCE.csv")
data = read_csv("RELIANCE.csv")
data = data.round(3)
for i in range(0,5):
portfolio.runModel(data, startingBalance, buyLogic, sellLogic, nextDayLogic)
portfolio.displayLastRun()
portfolio.displayAllRuns()
Hi,
I use the data.csv
and following indicators in the test code, but get above error.
features = ["X1", "X2"]
here is the data.head():
date open high low close volume X1 X2
0 4/15/16 60.24 60.625 60.010 60.51 5965310 59.959 0.268
1 4/18/16 60.69 61.070 60.355 60.89 7228573 60.048 0.009
2 4/19/16 61.16 61.250 60.480 60.90 7283570 60.129 0.505
3 4/20/16 61.04 61.430 60.850 60.90 5558770 60.203 1.190
4 4/21/16 60.90 61.100 60.480 60.64 12799083 60.244 2.586
Hello,
Great package. Can you provide sample data with the acceptable data layout.
Thanks,
Amir
I'm trying to run the example, though it has been modified as per the guidance in other issues. Here is the code:
from clairvoyant.engine import Backtest
import pandas as pd
features = ["X1", "X2"] # Financial indicators of choice
trainStart = 0 # Start of training period
trainEnd = 100 # End of training period
testStart = 200 # Start of testing period
testEnd = 300 # End of testing period
buyThreshold = 0.65 # Confidence threshold for predicting buy (default = 0.65)
sellThreshold = 0.65 # Confidence threshold for predicting sell (default = 0.65)
continuedTraining = False # Continue training during testing period? (default = false)
# Initialize backtester
backtest = Backtest(features, trainStart, trainEnd, testStart, testEnd, buyThreshold, sellThreshold, continuedTraining)
# A little bit of pre-processing
data = pd.read_csv("clair-data.csv", date_parser=['date'])
data = data.round(3)
# Start backtesting and optionally modify SVC parameters
# Available paramaters can be found at: http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html
backtest.start(data, kernel='rbf', C=1, gamma=10)
backtest.conditions()
backtest.statistics()
backtest.visualize('X1','X2')
and this is the output I'm getting:
------------ Data Features ------------
X1: X1
X2: X2
---------------------------------------
----------- Model Arguments -----------
kernel: rbf
C: 1
gamma: 10
---------------------------------------
--------- Engine Conditions ----------
Training: 4/15/16 -- 9/7/16
Testing: 1/31/17 -- 6/23/17
Buy Threshold: 65.0%
Sell Threshold: 65.0%
Continued Training: False
---------------------------------------
------------- Statistics --------------
Total Buys: 0
Buy Accuracy: 0.0%
Total Sells: 11
Sell Accuracy: 27.27%
---------------------------------------
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-5-480e8274c951> in <module>
23 backtest.conditions()
24 backtest.statistics()
---> 25 backtest.visualize('X1','X2')
~/.local/lib/python3.6/site-packages/clairvoyant/engine.py in visualize(self, name, width, height, stepsize)
170 xx, yy = meshgrid(arange(x_min, x_max, stepsize), arange(y_min, y_max, stepsize))
171
--> 172 pyplot.figure(figsize=(width, height))
173 cm = pyplot.cm.RdBu # Red/Blue gradients
174 rb = ListedColormap(['#FF312E', '#6E8894']) # Red = 0 (Negative) / Blue = 1 (Positve)
~/.local/lib/python3.6/site-packages/matplotlib/pyplot.py in figure(num, figsize, dpi, facecolor, edgecolor, frameon, FigureClass, clear, **kwargs)
543 frameon=frameon,
544 FigureClass=FigureClass,
--> 545 **kwargs)
546
547 if figLabel:
~/.local/lib/python3.6/site-packages/matplotlib/backend_bases.py in new_figure_manager(cls, num, *args, **kwargs)
3256 from matplotlib.figure import Figure
3257 fig_cls = kwargs.pop('FigureClass', Figure)
-> 3258 fig = fig_cls(*args, **kwargs)
3259 return cls.new_figure_manager_given_figure(num, fig)
3260
~/.local/lib/python3.6/site-packages/matplotlib/figure.py in __init__(self, figsize, dpi, facecolor, edgecolor, linewidth, frameon, subplotpars, tight_layout, constrained_layout)
346 frameon = rcParams['figure.frameon']
347
--> 348 if not np.isfinite(figsize).all():
349 raise ValueError('figure size must be finite not '
350 '{}'.format(figsize))
TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Do you have any idea what this could be? Is there other things that I need to adjust to run the example?
Hi! Thank you for uploading this project.
I am having an issue running Backtest using the code in your README.md file. Using all of your original code and your example data, I receive the following error (I've edited out the traceback parts):
TypeError: 'str' object cannot be interpreted as an integer
File "pandas\_libs\index.pyx", line 106, in pandas._libs.index.IndexEngine.get_value
File "pandas\_libs\index.pyx", line 114, in pandas._libs.index.IndexEngine.get_value
File "pandas\_libs\index.pyx", line 162, in pandas._libs.index.IndexEngine.get_loc
File "pandas\_libs\hashtable_class_helper.pxi", line 1492, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas\_libs\hashtable_class_helper.pxi", line 1500, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'EMA'
Am I setting up the features incorrectly or do you know why I might be receiving this error? I am running python 3.6.4 and all of the dependencies are installed and up to date, as per the versions listed in the SyntaxError issue posted.
First of all, let me start by thanking you very much for all your hard work and dedication. Your work is truly magnificent.
I am trying to run the example from your instructions. I am getting the error:
File "C:\Python27\lib\site-packages\clairvoyant\model.py", line 22
self.svc = SVC(**kwargs, probability=True)
SyntaxError: invalid syntax
What could be the cause of this error ?
I am running Python 2.7.15
Hi, In clairvoyant, inappropriate dependency versioning constraints can cause risks.
Below are the dependencies and version constraints that the project is using
bokeh>=0.12.4
matplotlib>=2.0.0
numpy>=1.11.3
pandas>=0.19.2
scikit-learn>=0.18.1
The version constraint == will introduce the risk of dependency conflicts because the scope of dependencies is too strict.
The version constraint No Upper Bound and * will introduce the risk of the missing API Error because the latest version of the dependencies may remove some APIs.
After further analysis, in this project,
The version constraint of dependency scikit-learn can be changed to >=0.17b1,<=0.21.3.
The above modification suggestions can reduce the dependency conflicts as much as possible,
and introduce the latest version as much as possible without calling Error in the projects.
The invocation of the current project includes all the following methods.
Axes.scatter Axes.set_ylim pyplot.savefig Axes.contourf Axes.set_xlim Axes.set_title pyplot.figure pyplot.subplot
RobustScaler.fit
setup ClosedTrade self.model.predict pyplot.savefig Axes.contourf TypeError self.account.Equity.append print super.__init__ exchange.Account round RobustScaler.fit xx.min pyplot.figure copy.deepcopy enumerate pyplot.subplot self.account.TotalValue vstack helpers.change X.append yy.max yy.min LongPosition Axes.set_xlim show Engine.start super SVC Axes.set_title output_file format helpers.profit change Model range len self.account.PurgePositions self.Positions.append Engine.__init__ self.OpenedTrades.append figure sys.path.append self.model.fit self.model.svc.fit self.model.svc.decision_function Z.reshape hstack self.svc.fit self.svc.predict_proba xx.max X.min meshgrid ShortPosition p.line p.Show Temporary.ClosePosition self.model.scaler.transform Axes.scatter y.append Position.Close float logic ValueError Axes.set_ylim arange self.buyStats RobustScaler self.ClosedTrades.append X.max OpenedTrade yy.ravel xx.ravel ListedColormap self.scaler.transform self.sellStats
@developer
Could please help me check this issue?
May I pull a request to fix it?
Thank you very much.
I am using standard Yahoo downloads in the format
Date,Open,High,Low,Close,Volume,Close
2015-01-01,312.45,315.0,310.7,314.0,6138488,314.0
When I run sample program (as per README) I get the following error -
Traceback (most recent call last):
File "clairvoyance.py", line 35, in <module>
backtest.runModel(data)
File "/Users/8288/source_code/finance/venv/lib/python2.7/site-packages/clairvoyant/Backtest.py", line 75, in runModel
trainStart = DateIndex(data, stock, self.trainStart, False)
File "/Users/8288/source_code/finance/venv/lib/python2.7/site-packages/clairvoyant/Backtest.py", line 16, in DateIndex
lowbound = data["Date"][0]
File "/Users/8288/source_code/finance/venv/lib/python2.7/site-packages/pandas/core/series.py", line 603, in __getitem__
result = self.index.get_value(self, key)
File "/Users/8288/source_code/finance/venv/lib/python2.7/site-packages/pandas/indexes/base.py", line 2175, in get_value
return tslib.get_value_box(s, key)
File "pandas/tslib.pyx", line 946, in pandas.tslib.get_value_box (pandas/tslib.c:19053)
File "pandas/tslib.pyx", line 962, in pandas.tslib.get_value_box (pandas/tslib.c:18770)
IndexError: index out of bounds
I am having the following issue in getting the Clairvoyant project, that identifies and monitors social/historical cues for short term stock movement, to run successfully:
The file hashtable_class_helper.pxl could not be opened.
Here is the exception generated:
Exception "unhandled KeyError"
'SSO'
File: pandas/src/hashtable_class_helper.pxi, Line: 740
I think it might have something to do with the fact that this is a cython-based project and I'm wanting to implement in Python 2.7x in Ubuntu 16.04
Any help would be appreciated.
from clairvoyant import Backtest
from pandas import read_csv
variables = ["SSO", "SSC"] # Financial indicators of choice
trainStart = '2013-03-01' # Start of training period
trainEnd = '2015-07-15' # End of training period
testStart = '2015-07-16' # Start of testing period
testEnd = '2016-07-16' # End of training period
buyThreshold = 0.65 # Confidence threshold for predicting buy (default = 0.65)
sellThreshold = 0.65 # Confidence threshold for predicting sell (default = 0.65)
C = 1 # Penalty parameter (default = 1)
gamma = 10 # Kernel coefficient (default = 10)
continuedTraining = False # Continue training during testing period? (default = false)
backtest = Backtest(variables, trainStart, trainEnd, testStart, testEnd)
data = read_csv(r"H:/Python/ClairVoyantTest/sbux.csv") # Read in data
data = data.round(3) # Round all values
backtest.stocks.append("SBUX") # Inform the model which stock is being tested
for i in range(0,10): # Run the model 10-15 times
backtest.runModel(data)
backtest.displayConditions()
backtest.displayStats()
I have run the above code: The issue coming was
File "", line 1, in runfile('H:/Python/StockPredictionUsingClairVoyant.py', wdir='H:/Python')
File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 685, in runfile execfile(filename, namespace)
File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 71, in execfile exec(compile(scripttext, filename, 'exec'), glob, loc)
File "H:/Python/StockPredictionUsingClairVoyant.py", line 30, in backtest.runModel(data)
File "C:\Anaconda2\lib\site-packages\clairvoyant\Backtest.py", line 72, in runModel data['Date'] = to_datetime(data['Date'])
File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 1914, in getitem return self._getitem_column(key)
File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 1921, in _getitem_column return self._get_item_cache(key)
File "C:\Anaconda2\lib\site-packages\pandas\core\generic.py", line 1090, in _get_item_cache values = self._data.get(item)
File "C:\Anaconda2\lib\site-packages\pandas\core\internals.py", line 3102, in get loc = self.items.get_loc(item)
File "C:\Anaconda2\lib\site-packages\pandas\core\index.py", line 1692, in get_loc return self._engine.get_loc(_values_from_object(key))
File "pandas\index.pyx", line 137, in pandas.index.IndexEngine.get_loc (pandas\index.c:3979)
File "pandas\index.pyx", line 157, in pandas.index.IndexEngine.get_loc (pandas\index.c:3843)
File "pandas\hashtable.pyx", line 668, in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12265)
File "pandas\hashtable.pyx", line 676, in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12216)
KeyError: 'Date'
runfile('H:/Python/StockPredictionUsingClairVoyant.py', wdir='H:/Python') Traceback (most recent call last):
File "", line 1, in runfile('H:/Python/StockPredictionUsingClairVoyant.py', wdir='H:/Python')
File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 685, in runfile execfile(filename, namespace)
File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 71, in execfile exec(compile(scripttext, filename, 'exec'), glob, loc)
File "H:/Python/StockPredictionUsingClairVoyant.py", line 27, in data = data.round(3) # Round all values
File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 4335, in round new_cols = [np.round(self[col], decimals) for col in self]
File "C:\Anaconda2\lib\site-packages\numpy\core\fromnumeric.py", line 2782, in round_ return round(decimals, out)
File "C:\Anaconda2\lib\site-packages\pandas\core\series.py", line 1234, in round result = _values_from_object(self).round(decimals, out=out)
What would be the cause of the error.
I am trying to run the example from your instructions. data comes from example/data.csv
I am getting the error:
Traceback (most recent call last):
File "test.py", line 22, in
backtest.start(data, kernel='rbf', C=1, gamma=10)
File "/home/machol/virtualenv/local/lib/python2.7/site-packages/clairvoyant/engine.py", line 197, in start
Engine.start(self, data, **kwargs)
File "/home/machol/virtualenv/local/lib/python2.7/site-packages/clairvoyant/engine.py", line 60, in start
self.model.fit(X, y)
File "/home/machol/virtualenv/local/lib/python2.7/site-packages/clairvoyant/engine.py", line 22, in fit
self.svc.fit(self.scaler.transform(self.XX), self.yy)
File "/home/machol/virtualenv/local/lib/python2.7/site-packages/sklearn/svm/base.py", line 187, in fit
fit(X, y, sample_weight, solver_type, kernel, random_seed=seed)
File "/home/machol/virtualenv/local/lib/python2.7/site-packages/sklearn/svm/base.py", line 254, in _dense_fit
max_iter=self.max_iter, random_seed=random_seed)
File "sklearn/svm/libsvm.pyx", line 59, in sklearn.svm.libsvm.fit
TypeError: a float is required
My python version is 2.7.9。
So basicly im downloading yahoo data and then convert the date format because I've seen in previously issues in this repo that you don't support yyyy-mm-dd?
So i convert from yyyy-mm-dd to mm-dd-yyyy.
Then i assign variables in the same date format.
I see from your example in readme that you are using integers here instead?
So is date not supported anymore?
As you can see in the traceback at the bottom dates don't work for me.
variables = ["Open", "Close", "High", "Low", "Volume"]
trainStart = '11-01-2015' # Start of training period
trainEnd = '11-01-2017' # End of training period
testStart = '11-02-2017' # Start of testing period
testEnd = '11-17-2017' # End of testing period
So here is the data from the csv file before i read it.
1 Date,Open,High,Low,Close,Volume
2 2015-01-02,5.05,5.18,4.83,5.14,1542452.0
3 2015-01-05,5.15,5.23,4.8,4.99,1207386.0
4 2015-01-06,4.89,4.97,4.75,4.92,704102.0
5 2015-01-07,4.82,4.93,4.75,4.8,614919.0
data = read_csv("file.csv")
data = data.round(3) # Round all values
data['Date'] = to_datetime(data.Date) #convert Date coloum data to datetime
data['Date'] = data['Date'].dt.strftime('%m-%d-%Y') #convert format of dates
output of print(data)
0 01-02-2015 5.05 5.18 4.83 5.14 1542452.0
1 01-05-2015 5.15 5.23 4.80 4.99 1207386.0
2 01-06-2015 4.89 4.97 4.75 4.92 704102.0
3 01-07-2015 4.82 4.93 4.75 4.80 614919.0
4 01-08-2015 4.85 4.98 4.65 4.72 686451.0
Traceback (most recent call last):
File "clair2.py", line 35, in <module>
backtest.start(data, kernel='rbf', C=1, gamma=10)
File "/home/kodi/.local/lib/python3.5/site-packages/clairvoyant/engine.py", line 197, in start
Engine.start(self, data, **kwargs)
File "/home/kodi/.local/lib/python3.5/site-packages/clairvoyant/engine.py", line 50, in start
for i in range(self.trainStr, self.trainEnd+1):
TypeError: Can't convert 'int' object to str implicitly
backtest.start(data, kernel='rbf', C=1, gamma=10)
backtest.conditions()
backtest.statistics()
backtest.visualize('thin.ol')
I created a Stocks.csv file for the first test code you had on the readme page, but I can't figure out the format required for the script to work. Do you have a template?
I am very interetes in your programe,but there is a little difficult for me to get the file in China. Could you please send the file("SUBX.csv") to my email("[email protected]")?
in Engine.start for below code:
Xs = [ data.iloc[i][var] for var in self.features ]
features are mentioned as ["EMA","SSO"] # i assume we need to map them to X1, X2.
iloc takes indexes as integers, it will not work as i.e; data.iloc[0]["EMA"] .
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.