get error
ValueError: setting an array element with a sequence.

i locate error in /home/xxx/.local/lib/python3.6/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py:

            obs, self.buf_rews[env_idx], self.buf_dones[env_idx], self.buf_infos[env_idx] = self.envs[env_idx].step(
                self.actions[env_idx]
            )

self.buf_rews[env_idx] length not equal self.envs[env_idx].step(self.actions[env_idx])[1]

i get self.envs[env_idx].step(self.actions[env_idx])[1] = [0. 0. 0. 0.]

but self.buf_rews[env_idx] origin value = <class 'tuple'>: (0.0,)

and data difference is date = ’2020-12-19 12:00:00‘, and is no tic , then I fill in BTC or AAPL

this project include DQN, DDPG, PPO, SAC, A2C, TD3, etc.I want to do some reach in DQN.whre is the DQN model scipts.if I want to train a DQN.what parameter should I pass.

hey sir thanks for your helpful article
I've read your article in two form ensemble strategy and this model
with comparison of backtesting result could you please speak about why this model is working better than ensemble strategy and what improvement are you going to implement in the new version of this model

Hello, I really appreciate your work.I is excellent. But when I ran the " Single Stock Trading" notebook directly without any modification, an error has occured as below

==============Get Backtest Results===========

ZeroDivisionError Traceback (most recent call last)
in ()
2 now = datetime.datetime.now().strftime('%Y%m%d-%Hh%M')
3
----> 4 perf_stats_all = BackTestStats(account_value=df_account_value)
5 perf_stats_all = pd.DataFrame(perf_stats_all)
6 perf_stats_all.to_csv("./"+config.RESULTS_DIR+"/perf_stats_all_"+now+'.csv')

e:\CCXI工作\ETF投资\FinRL-Library-master\FinRL-Library-master\FinRL-Library-master\finrl\trade\backtest.py in BackTestStats(account_value)
12 def BackTestStats(account_value):
13 df = account_value.copy()
---> 14 df = get_daily_return(df)
15 DRL_strat = backtest_strat(df)
16 perf_func = timeseries.perf_stats

e:\CCXI工作\ETF投资\FinRL-Library-master\FinRL-Library-master\FinRL-Library-master\finrl\trade\backtest.py in get_daily_return(df)
93 df["daily_return"] = df.account_value.pct_change(1)
94 # df=df.dropna()
---> 95 sharpe = (252 ** 0.5) * df["daily_return"].mean() / df["daily_return"].std()
96
97 annual_return = ((df["daily_return"].mean() + 1) ** 252 - 1) * 100

unable to prepare context: path "docker/" not found

It seems your code doesn't consider time series of observations?
Since there'll be no recurrent policy in stable baselines3 before v1.2, current implementation only takes 1 day's observation into consideration.
Did I miss anything that creates time series implicitly?

Hi, can anyone kindly point out if there is a guide on how to make one's own data into the same format as the example dataset? I tried to simply import csv of my own data but there are a couple of attributes missing when I initialize the environment. For example, if I want to use intraday data or tick data how should I format my 'date' attribute? Thanks a lot

Thank you a lot for this wonderful work. It helps me a lot to start in this area. I am wondering is there anyway to print out the stat of the stock (like the date, the price, and current account balance) every time when the model trades?

First of all, thanks a lot for the great work!!!!
I'm using the EnvSingleStock to evaluate the models on a specific stock. I want to better visualize the model training to evaluate it (as I'll train multiple runs of each model with different hyperparameter configuration and want to compare them). Is there a log implemented that would allow me to check:

• the end_total_asset, total_reward, total_cost, total_trades, and Sharpe during model training (could be just a df of the values printed during training the model)
• the chart related to the rewards during the training period
• the trades that were made by the agent during the training period (when they were made and what were their characteristics (buy or sell and how much)
  Is there a way to do it? (it doesn't need to be online, could be after the training is over). It would be easy to do
  Thanks!!!

Hey! Thanks for great article and library. Ive trained model on historical data. Is it possible to add data in realtime from websockets for example? And make predictions every time new data comes?
Thanks

The new commits using stable-baselines-3 broke the training. When I train the agent in the single stock trading jupyter notebooks, it prints the following while it trains:
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost: 0
total_trades: 0

While running FinRL_single_stock_trading.ipynb, I found that the logs of Model 2 (DDPG) and Model 4 (TD3) are weird. It looks like none trading has been done, which is not same as the demo you give. I would appreciate it if you could help to check if there is anything wrong.
BTW, there is a little typo that SAC model should be marked as Model 5 rather than Model 4.

total_trades:  0
---------------------------------
| time/              |          |
|    episodes        | 4        |
|    fps             | 75       |
|    time_elapsed    | 133      |
|    total timesteps | 10064    |
| train/             |          |
|    actor_loss      | -513     |
|    critic_loss     | 496      |
|    learning_rate   | 0.0001   |
|    n_updates       | 7548     |
---------------------------------
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
---------------------------------
| time/              |          |
|    episodes        | 8        |
|    fps             | 64       |
|    time_elapsed    | 310      |
|    total timesteps | 20128    |
| train/             |          |
|    actor_loss      | -330     |
|    critic_loss     | 432      |
|    learning_rate   | 0.0001   |
|    n_updates       | 17612    |
---------------------------------
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
---------------------------------
| time/              |          |
|    episodes        | 12       |
|    fps             | 60       |
|    time_elapsed    | 495      |
|    total timesteps | 30192    |
| train/             |          |
|    actor_loss      | -172     |
|    critic_loss     | 194      |
|    learning_rate   | 0.0001   |
|    n_updates       | 27676    |
---------------------------------

Logging to tensorboard_log/td3/td3_1
begin_total_asset:100000
end_total_asset:687696.4228235682
total_reward:587696.4228235682
total_cost:  10571.455648996345
total_trades:  2513
Sharpe:  0.944539208399143
=================================
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
---------------------------------
| time/              |          |
|    episodes        | 4        |
|    fps             | 60       |
|    time_elapsed    | 165      |
|    total timesteps | 10064    |
| train/             |          |
|    actor_loss      | 3.17e+03 |
|    critic_loss     | 1.28e+04 |
|    learning_rate   | 0.0003   |
|    n_updates       | 7548     |
---------------------------------
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
---------------------------------
| time/              |          |
|    episodes        | 8        |
|    fps             | 51       |
|    time_elapsed    | 392      |
|    total timesteps | 20128    |
| train/             |          |
|    actor_loss      | 2.96e+03 |
|    critic_loss     | 7.85e+03 |
|    learning_rate   | 0.0003   |
|    n_updates       | 17612    |
---------------------------------
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
begin_total_asset:100000
end_total_asset:100000.0
total_reward:0.0
total_cost:  0
total_trades:  0
---------------------------------
| time/              |          |
|    episodes        | 12       |
|    fps             | 48       |
|    time_elapsed    | 625      |
|    total timesteps | 30192    |
| train/             |          |
|    actor_loss      | 2.4e+03  |
|    critic_loss     | 5.31e+03 |
|    learning_rate   | 0.0003   |
|    n_updates       | 27676    |
---------------------------------

Thanks for your work and for making it user friendly.

Hi,
i can not build full working env :(
(win10 , python 3.6.5)

make please version dump for me.
pip freeze > requirements.txt

my actions:
`
git clone https://github.com/AI4Finance-LLC/FinRL-Library.git

cd FinRL-Library

virtualenv -p python3 venv

venv\Scripts\activate

(venv) pip install -r requirements.txt

(venv) pip install ipykernel

(venv) jupyter notebook

open any notebook, for example "FinRL_single_stock_trading.ipynb"

run

on 2.3 'Import Packages' i get errors:
`

C:\Users\setar\AppData\Roaming\Python\Python36\site-packages\pandas_datareader\compat_init_.py:7: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
from pandas.util.testing import assert_frame_equal
C:\Users\setar\AppData\Roaming\Python\Python36\site-packages\empyrical\utils.py:32: UserWarning: Unable to import pandas_datareader. Suppressing import error and continuing. All data reading functionality will raise errors; but has been deprecated and will be removed in a later version.
warnings.warn(msg)

AttributeError Traceback (most recent call last)
in
12 from finrl.env.env_stocktrading import StockTradingEnv
13 from finrl.model.models import DRLAgent
---> 14 from finrl.trade.backtest import BackTestStats, BaselineStats, BackTestPlot
15
16 import sys

D:\Nextcloud\projects\ai\FinRL\FinRL-Library\finrl\trade\backtest.py in
2 import numpy as np
3
----> 4 from pyfolio import timeseries
5 import pyfolio
6 import matplotlib.pyplot as plt

c:\program files\python36\lib\site-packages\pyfolio_init_.py in
----> 1 from . import utils
2 from . import timeseries
3 from . import pos
4 from . import txn
5 from . import interesting_periods

c:\program files\python36\lib\site-packages\pyfolio\utils.py in
26 import empyrical.utils
27
---> 28 from . import pos
29 from . import txn
30

c:\program files\python36\lib\site-packages\pyfolio\pos.py in
20
21 try:
---> 22 from zipline.assets import Equity, Future
23 ZIPLINE = True
24 except ImportError:

~\AppData\Roaming\Python\Python36\site-packages\zipline_init_.py in
22
23 from . import data
---> 24 from . import finance
25 from . import gens
26 from . import utils

~\AppData\Roaming\Python\Python36\site-packages\zipline\finance_init_.py in
14 # limitations under the License.
15
---> 16 from . import execution, trading
17
18 all = [

~\AppData\Roaming\Python\Python36\site-packages\zipline\finance\trading.py in
18
19 from zipline.utils.memoize import remember_last
---> 20 from zipline.utils.pandas_utils import normalize_date
21
22 log = logbook.Logger('Trading')

~\AppData\Roaming\Python\Python36\site-packages\zipline\utils\pandas_utils.py in
221
222 INDEXER_NAMES = [
--> 223 '' + name for (name, _) in pd.core.indexing.get_indexers_list()
224 ]
225

AttributeError: module 'pandas.core.indexing' has no attribute 'get_indexers_list'`

class DRLAgent:
"""Provides implementations for DRL algorithms

Attributes
----------
    env: gym environment class
        user-defined class

Methods
-------
train_PPO()
    the implementation for PPO algorithm
train_A2C()
    the implementation for A2C algorithm
train_DDPG()
    the implementation for DDPG algorithm
train_TD3()
    the implementation for TD3 algorithm
train_SAC()
    the implementation for SAC algorithm
DRL_prediction()
    make a prediction in a test dataset and get results
"""

There does not exist the method train_A2C() under the class DRLAgent. Can you help to fix it?

How to use this library for intraday trading? It originally uses just daily data, but what if I need to make decisions at, let's say, hourly rate?

in FinRL_single_stock_trading.ipynb

after finishing training Model 4 Sac I am getting this:

Model 4: SAC
agent = DRLAgent(env = env_train)
SAC_PARAMS = {
"batch_size": 128,
"buffer_size": 100000,
"learning_rate": 0.00003,
"learning_starts": 100,
"ent_coef": "auto_0.1",
}
​
model_sac = agent.get_model("sac",model_kwargs = SAC_PARAMS)
{'batch_size': 128, 'buffer_size': 100000, 'learning_rate': 3e-05, 'learning_starts': 100, 'ent_coef': 'auto_0.1'}
Using cpu device
trained_sac = agent.train_model(model=model_sac,
tb_log_name='sac',
total_timesteps=30000)
Logging to tensorboard_log/sac/sac_2

| time/ | |
| episodes | 4 |
| fps | 81 |
| time_elapsed | 123 |
| total timesteps | 10064 |
| train/ | |
| actor_loss | -940 |
| critic_loss | 1.6 |
| ent_coef | 0.135 |
| ent_coef_loss | 19 |
| learning_rate | 3e-05 |
| n_updates | 9963 |

day: 2515, episode: 100
begin_total_asset:100000.00
end_total_asset:100000.00
total_reward:0.00
total_cost: 0.00
total_trades: 0

| time/ | |
| episodes | 8 |
| fps | 80 |
| time_elapsed | 250 |
| total timesteps | 20128 |
| train/ | |
| actor_loss | -510 |
| critic_loss | 147 |
| ent_coef | 0.182 |
| ent_coef_loss | 15.8 |
| learning_rate | 3e-05 |
| n_updates | 20027 |

Is there a way to set the seed to get repeatable results?

I am trying to run the code provided for FinRL_multiple_stock_trading, but I get Value error while running the following line.

df_account_value, df_actions = DRLAgent.DRL_prediction(model=model_sac,
                        test_data = trade,
                        test_env = env_trade,
                        test_obs = obs_trade)

@BruceYanghy Can you please tell how it can be solved? Thank you very much for sharing your work, I would like to experiment with your code, appreciate your help.

Best,
Harshad

is the actions means trading action for next trading day?

if the results has the following values (using APPL as an example)-

• date: 2020-12-01
• actions: [200]
• close_price: 122.72

Does this mean the trading action for Dec. 2nd is to buy 100 APPL? Trying to understand how we can use the results from the models.

thanks

A better way to use turbulence Index is to apply Mahalanobis Dist.. on returns timeseries rather than the levels as you are doing...

Hi,
I just want to know how can I train my DRL model with GPU. (to speed up the training process)

Thank you!

When running FinRL_single_stock_trading.ipynb all cells run fine until:

env_train = env_setup.create_env_training(data = train,
env_class = SingleStockEnv)

I get the following error:

AttributeError Traceback (most recent call last)
in
----> 1 env_train = env_setup.create_env_training(data = train,
2 env_class = SingleStockEnv)

/mnt/c/Users/XXX/Documents/PROJECT/STOCK_TRADING/FinRL-Library/finrl/env/environment.py in create_env_training(self, data, env_class, turbulence_threshold)
54
55 def create_env_training(self, data, env_class, turbulence_threshold=150):
---> 56 env_train = DummyVecEnv([lambda: env_class(df = data,
57 stock_dim = self.stock_dim,
58 hmax = self.hmax,

~/.local/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py in init(self, env_fns)
23
24 def init(self, env_fns: List[Callable[[], gym.Env]]):
---> 25 self.envs = [fn() for fn in env_fns]
26 env = self.envs[0]
27 VecEnv.init(self, len(env_fns), env.observation_space, env.action_space)

~/.local/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py in (.0)
23
24 def init(self, env_fns: List[Callable[[], gym.Env]]):
---> 25 self.envs = [fn() for fn in env_fns]
26 env = self.envs[0]
27 VecEnv.init(self, len(env_fns), env.observation_space, env.action_space)

/mnt/c/Users/XXX/Documents/PROJECT/STOCK_TRADING/FinRL-Library/finrl/env/environment.py in ()
54
55 def create_env_training(self, data, env_class, turbulence_threshold=150):
---> 56 env_train = DummyVecEnv([lambda: env_class(df = data,
57 stock_dim = self.stock_dim,
58 hmax = self.hmax,

in init(self, df, stock_dim, hmax, initial_amount, transaction_cost_pct, reward_scaling, state_space, action_space, tech_indicator_list, turbulence_threshold, day)
100 [self.data.high] +
101 [self.data.low] +
--> 102 [self.data.daily_return]
103 # initialize reward
104 self.reward = 0

~/.local/lib/python3.8/site-packages/pandas/core/generic.py in getattr(self, name)
5137 if self._info_axis._can_hold_identifiers_and_holds_name(name):
5138 return self[name]
-> 5139 return object.getattribute(self, name)
5140
5141 def setattr(self, name: str, value) -> None:

AttributeError: 'Series' object has no attribute 'daily_return'

I am following multi stock notebook. When about to train the first model I got this error message:

Model 1: A2C
agent = DRLAgent(env = env_train)
model_a2c = agent.get_model("a2c")
{'n_steps': 5, 'ent_coef': 0.01, 'learning_rate': 0.0007}
Using cpu device
/opt/conda/lib/python3.6/site-packages/torch/cuda/init.py:52: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /opt/conda/conda-bld/pytorch_1607370116979/work/c10/cuda/CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0

I have a RTX2060 graphic card and the latest driver. What else should I do to be able to use it with the algorithm?

KeyError Traceback (most recent call last)
in ()
6 'verbose':0,
7 'timesteps':50000}
----> 8 model_ddpg = agent.train_DDPG(model_name = "DDPG_{}".format(now), model_params = ddpg_params_tuning)

/usr/local/lib/python3.6/dist-packages/finrl/model/models.py in train_DDPG(self, model_name, policy, model_params, policy_kwargs)
82 batch_size=model_params['batch_size'],
83 buffer_size=model_params['buffer_size'],
---> 84 learning_rate=model_params['learning_rate'],
85 action_noise=action_noise,
86 verbose=model_params['verbose'],

KeyError: 'learning_rate'

Hello:
May I ask a question when I try to change the data environment from Dow 30 to SSE 50, I face the following question:

model = agent.train_SAC(model_name = "SAC_{}".format(now), model_params = sac_params_tuning)
Traceback (most recent call last):

File "", line 1, in
model = agent.train_SAC(model_name = "SAC_{}".format(now), model_params = sac_params_tuning)

File "/home/liuyimo/Downloads/FinRL-Library-master_a/finrl/model/models.py", line 136, in train_SAC
model.learn(total_timesteps=model_params['timesteps'], tb_log_name = "SAC_run")

File "/home/liuyimo/anaconda3/lib/python3.7/site-packages/stable_baselines/sac/sac.py", line 378, in learn
obs = self.env.reset()

File "/home/liuyimo/anaconda3/lib/python3.7/site-packages/stable_baselines/common/base_class.py", line 1104, in reset
return self.unvec_obs(self.venv.reset())

File "/home/liuyimo/anaconda3/lib/python3.7/site-packages/stable_baselines/common/vec_env/dummy_vec_env.py", line 62, in reset
self._save_obs(env_idx, obs)

File "/home/liuyimo/anaconda3/lib/python3.7/site-packages/stable_baselines/common/vec_env/dummy_vec_env.py", line 92, in _save_obs
self.buf_obs[key][env_idx] = obs

ValueError: cannot copy sequence with size 255 to array axis with dimension 355

When I run the Dow 30, everything goes fine. I tray to detect the reason behind this error, but I don't know where this '255' coming from? Is there any help for that? Thanks a lot

Hi,

I got the following error when running python main.py --mode=train

==============Start Feature Engineering===========
Traceback (most recent call last):
File "main.py", line 47, in
main()
File "main.py", line 36, in main
finrl.autotrain.training.train_one()
File "/mnt/c/Users/JOSE/Documents/PROJECT/STOCK_TRADING/FinRL-Library/finrl/autotrain/training.py", line 30, in train_one
df = FeatureEngineer(df,
TypeError: init() got multiple values for argument 'use_technical_indicator'

I also get this at the beginning of the run:
/mnt/c/Users/JOSE/Documents/PROJECT/STOCK_TRADING/FinRL-Library/venv/lib/python3.8/site-packages/pyfolio/pos.py:26: UserWarning: Module "zipline.assets" not found; mutltipliers will not be applied to position notionals.

I installed all dependencies and pre-requisites before running. Can you help me please?

Thanks

medium link: https://towardsdatascience.com/finrl-for-quantitative-finance-tutorial-for-multiple-stock-trading-7b00763b7530 9first colab link)
my colab link: https://colab.research.google.com/github/AI4Finance-LLC/FinRL-Library/blob/master/FinRL_multiple_stock_trading.ipynb
part 5, cell 18 is not working, says TypeError: init() got an unexpected keyword argument 'transaction_cost_pct'
when i comment it it says it needs two other vars.

found out an error in compute turbulence,
show error:
LinAlgError: Singular matrix

The problem arises due to the perfect correlation between the two series in your data

so i fix it use np.linalg.inv -->np.linalg.pinv, but this not best solution.

we can fix it like that, add just a little noise to your data, the error disappears, and still use linalg.inv:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from statsmodels.tsa.stattools import grangercausalitytests

n = 1000
ls = np.linspace(0, 2*np.pi, n)
df1Clean = pd.DataFrame(np.sin(ls))
df2Clean = pd.DataFrame(2*np.sin(ls+1))
dfClean = pd.concat([df1Clean, df2Clean], axis=1)
dfDirty = dfClean+0.00001*np.random.rand(n, 2)

grangercausalitytests(dfClean, maxlag=20, verbose=False)    # Raises LinAlgError
grangercausalitytests(dfDirty, maxlag=20, verbose=False)    # Runs fine

but i have question?
which is turbulence effect？Is it necessary？Can other calculations be used instead?

I'm still quite new to this repo so correct me if i'm wrong but i think the reward for this environment cannot be less than the negated initial money (I saw there's reward scaling by roughly one in ten thousand but that does not make it less than the negated initial money either.) During training A2C using the default 100000 initial money, however, I saw this line:
total_reward | -2.93e+20
Could you double check the environment or provide some explanation on the reward? Thanks.

Why do we use 252 for the sharpe ratio calculation at the end of an episode. Looks like we using portfolio change from the beginning of the file which will be 9 years of trading period. so shouldn't 252 be replaced with self.days.

I used FinRL_single_stock_trading.ipynb to get the overview how to use the library but after training model and predicting the actions on trade data when you see the df_actions list the results are bit weird the agent is either selling all stocks(hmax) on each time stamp and then follow the same pattern throughout the trading data (selling all stocks at each time stamp) or sometimes on running it buys all stocks (hmax) on each time stamp and repeats the same action on every timestamp should i be worried about these results aren't they bit abnormal ?

I tried multiple trading data split variations (trading data : 2015-current ) (trading data: 2019-current) but same results
I am using Apple stocks the one which was used in FinRL_single_stock_trading.ipynb and the model was TD3
I tried other models as well still they were giving almost same result (at some timestamp the stock buy or sell were hmax - {1..5}) but the same results can you please verify these changes i think there is some issue in the repo code

First, thank you for this awesome project.

I try to add some leaky features to the model to see if it is capable of exploiting that kind of features. Normally I would expect to see some incredible result, because basically I'm telling the future to the model. But; model still works poorly. Why would that be you think? Is it maybe because of parameters of the model?

Also, in some models I see that, in some iterations rewards became 0 with these leaky features. So, some models choose not the operate instead of exploit these future information.

Thanks.

I have installed successfully all the packages through FinRL library. However, I met an error when i import finrl.env in my code as follows.

from finrl.env.environment import EnvSetup
from finrl.env.EnvMultipleStock_train import StockEnvTrain
from finrl.env.EnvMultipleStock_trade import StockEnvTrade
Traceback (most recent call last):

File "", line 1, in
from finrl.env.environment import EnvSetup

ModuleNotFoundError: No module named 'finrl.env.environment'

I checked my site-package folder, and I found there are only 'env_stocktrading.py' and 'env_portfolio.py' under the folder of finrl.env.

What should I do to import the 'finrl.env.environment' and 'finrl.env.EnvMultipleStock_train'?

Hi all,

First of all I would like to thank you for the fantastic work you have done.
The library is well done, complete and the documentation including tutorials and scientific papers is a really good job.

I am writing because I find a problem in the reproduction of the Notebook in question (Single stock), it seems that the parameters used are not defined in the function present in the source code.

This is the error:
TypeError: DRL_prediction() got an unexpected keyword argument 'trade_data'

Can you fix it please?

SM

I'm using a virtual environment of Python 3.7.3 in Mac 10.15.2
when I run
python main.py --mode=train
it returns

==============Start Feature Engineering===========
Successfully added technical indicators
** On entry to DLASCLS parameter number 4 had an illegal value
** On entry to DLASCLS parameter number 4 had an illegal value
Traceback (most recent call last):
File "main.py", line 53, in
main()
File "main.py", line 38, in main
finrl.autotrain.training.train_one()
File "/Users/angelachangi/Desktop/FinRL-Library/finrl/autotrain/training.py", line 36, in train_one
processed = fe.preprocess_data(df)
File "/Users/angelachangi/Desktop/FinRL-Library/finrl/preprocessing/preprocessors.py", line 53, in preprocess_data
df = self.add_turbulence(df)
File "/Users/angelachangi/Desktop/FinRL-Library/finrl/preprocessing/preprocessors.py", line 111, in add_turbulence
turbulence_index = self.calculate_turbulence(df)
File "/Users/angelachangi/Desktop/FinRL-Library/finrl/preprocessing/preprocessors.py", line 137, in calculate_turbulence
temp = current_temp.values.dot(np.linalg.pinv(cov_temp)).dot(
File "<array_function internals>", line 6, in pinv
File "/Users/angelachangi/Desktop/FinRL-Library/venv/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 2003, in pinv
u, s, vt = svd(a, full_matrices=False, hermitian=hermitian)
File "<array_function internals>", line 6, in svd
File "/Users/angelachangi/Desktop/FinRL-Library/venv/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 1661, in svd
u, s, vh = gufunc(a, signature=signature, extobj=extobj)
File "/Users/angelachangi/Desktop/FinRL-Library/venv/lib/python3.7/site-packages/numpy/linalg/linalg.py", line 97, in _raise_linalgerror_svd_nonconvergence
raise LinAlgError("SVD did not converge")
numpy.linalg.LinAlgError: SVD did not converge

Implementing the class without the DF parameter will allow this to be re-used in other contexts without being re-initialized.
This could be useful in designing backtesting.

I operated according to your instructions, and all the packages are complete, but I don't know how to report this error! Give advice or comments please!

Thanks for your brilliant works. I'm curious about the data_normaliser code in FinRL_single_stock_trading.ipynb, chapter 5.1.

from sklearn import preprocessing
data_normaliser = preprocessing.StandardScaler()
train[feaures_list] = data_normaliser.fit_transform(train[feaures_list])
trade[feaures_list] = data_normaliser.fit_transform(trade[feaures_list])

The code seem to use future information for normalization, am I right?

Everything is fine when stocks belong to the same index, for instance, MULTIPLE_STOCK_TICKER = ["AAPL", "MSFT", "IBM"].
However, using your own configuration (MULTIPLE_STOCK_TICKER = ["AAPL", "MSFT", "FB"]) and the latest code, crashes are being experienced depending on the value of the use_turbulence parameter (FeatureEngineer ctor):

Case 1: use_turbulence = True

~/qargo/argo/FinRL-Library$ python3 main.py --mode=train
/home/srdjan/.local/lib/python3.8/site-packages/pyfolio/pos.py:26: UserWarning: Module "zipline.assets" not found; multipliers will not be applied to position notionals.
warnings.warn(
==============Start Fetching Data===========
[100%] 1 of 1 completed
[100%] 1 of 1 completed
[100%**] 1 of 1 completed
Shape of DataFrame: (8146, 8)
==============Start Feature Engineering===========
Successfully added technical indicators
Traceback (most recent call last):
File "main.py", line 53, in
main()
File "main.py", line 38, in main
finrl.autotrain.training.train_one()
File "/home/srdjan/qargo/argo/FinRL-Library/finrl/autotrain/training.py", line 36, in train_one
processed = fe.preprocess_data(df)
File "/home/srdjan/qargo/argo/FinRL-Library/finrl/preprocessing/preprocessors.py", line 53, in preprocess_data
df = self.add_turbulence(df)
File "/home/srdjan/qargo/argo/FinRL-Library/finrl/preprocessing/preprocessors.py", line 111, in add_turbulence
turbulence_index = self.calculate_turbulence(df)
File "/home/srdjan/qargo/argo/FinRL-Library/finrl/preprocessing/preprocessors.py", line 137, in calculate_turbulence
temp = current_temp.values.dot(np.linalg.pinv(cov_temp)).dot(
File "<array_function internals>", line 5, in pinv
File "/home/srdjan/.local/lib/python3.8/site-packages/numpy/linalg/linalg.py", line 2003, in pinv
u, s, vt = svd(a, full_matrices=False, hermitian=hermitian)
File "<array_function internals>", line 5, in svd
File "/home/srdjan/.local/lib/python3.8/site-packages/numpy/linalg/linalg.py", line 1661, in svd
u, s, vh = gufunc(a, signature=signature, extobj=extobj)
File "/home/srdjan/.local/lib/python3.8/site-packages/numpy/linalg/linalg.py", line 97, in _raise_linalgerror_svd_nonconvergence
raise LinAlgError("SVD did not converge")
numpy.linalg.LinAlgError: SVD did not converge
~/qargo/argo/FinRL-Library$

Case 2: use_turbulence=False

~/qargo/argo/FinRL-Library$ python3 main.py --mode=train
/home/srdjan/.local/lib/python3.8/site-packages/pyfolio/pos.py:26: UserWarning: Module "zipline.assets" not found; multipliers will not be applied to position notionals.
warnings.warn(
==============Start Fetching Data===========
[100%] 1 of 1 completed
[100%] 1 of 1 completed
[100%**] 1 of 1 completed
Shape of DataFrame: (8146, 8)
==============Start Feature Engineering===========
Successfully added technical indicators
Traceback (most recent call last):
File "main.py", line 53, in
main()
File "main.py", line 38, in main
finrl.autotrain.training.train_one()
File "/home/srdjan/qargo/argo/FinRL-Library/finrl/autotrain/training.py", line 75, in train_one
env_train, _ = e_train_gym.get_sb_env()
File "/home/srdjan/qargo/argo/FinRL-Library/finrl/env/env_stocktrading.py", line 297, in get_sb_env
obs = e.reset()
File "/home/srdjan/.local/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 62, in reset
self._save_obs(env_idx, obs)
File "/home/srdjan/.local/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 92, in _save_obs
self.buf_obs[key][env_idx] = obs
ValueError: cannot copy sequence with size 14 to array axis with dimension 19
~/qargo/argo/FinRL-Library$

this problem is data processing, not alignment，but the state must have an alignment data for a day.

so original file can bring an error if you use ticker_list = config.CSI_300_TICKER

Stock Dimension: 298, State Space: 1789
Traceback (most recent call last):
  File "/home/wac/PycharmProjects/drl_crypto_coin/train/multi_train_drl.py", line 105, in <module>
    env_train, _ = e_train_gym.get_sb_env()
  File "/home/wac/.local/lib/python3.6/site-packages/finrl/env/env_stocktrading.py", line 286, in get_sb_env
    obs = e.reset()
  File "/home/wac/.local/lib/python3.6/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 62, in reset
    self._save_obs(env_idx, obs)
  File "/home/wac/.local/lib/python3.6/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 92, in _save_obs
    self.buf_obs[key][env_idx] = obs
ValueError: cannot copy sequence with size 1734 to array axis with dimension 1789

this request fix this error, please merged

pull request:
#46

Hi I've created 2 fresh environments in Python 3.6 and 3.8. Both are giving the below error.

ModuleNotFoundError Traceback (most recent call last)
in
10 from finrl.preprocessing.preprocessors import FeatureEngineer
11 from finrl.preprocessing.data import data_split
---> 12 from finrl.env.environment import EnvSetup
13 from finrl.env.EnvMultipleStock_train import StockEnvTrain
14 from finrl.env.EnvMultipleStock_trade import StockEnvTrade

ModuleNotFoundError: No module named 'finrl.env.environment'

hi thanks for your article
I used 1 h dataset from 2014 to 2020 of 3 stock with calculated turbulence but the model got confused with terrible result
this is models hyperparameters and the result
print("==============Model Training===========")
a2c_params_tuning = {'n_steps':5,
'ent_coef':0.005,
'learning_rate':0.0002,
'verbose':0,
'timesteps':150000}
model_a2c = agent.train_A2C(model_name = "A2C_{}".format(now), model_params = a2c_params_tuning)

begin_total_asset:1000000
end_total_asset:199131.0821982477
total_reward:-800868.9178017523
total_cost: 4804.364243369928
total_trades: 40468
Sharpe: -0.053885447004911706

Training time (A2C): 5.864342439174652 minutes

what's your opinion about implementing 1 hours data to model

Hi,

Congrat for this great work. I have a questions: How are you thinking to deploy these agents for paper trading or production (that is with real streaming price)?

Thank you

print("==============Get Backtest Results===========")

now = datetime.datetime.now().strftime('%Y%m%d-%Hh%M') <--------------please add this line

perf_stats_all = BackTestStats(account_value=df_account_value)
perf_stats_all = pd.DataFrame(perf_stats_all)
perf_stats_all.to_csv("./"+config.RESULTS_DIR+"/perf_stats_all_"+now+'.csv')

It seems your code does not take into consideration stocks with different shapes due to time on market, etc.

For example, if you use a start date of 2009 and include Facebook ($FB) it will error because Facebook didn't start trading in 2009.

When running from main.py (or any of the notebooks), there seems to be a number of function calls which do not exist within the modules that they are called from. Many of these seem to be coming from backtest.py which appears to have renamed the functions. After just switching the names as a quick test, some additional errors arise during the train_one function with array mismatches.

Hello, i found a bug which i don't know why it can happen, can you check out the screenshot below