swopper050 / foundations-drl Goto Github PK

Hi,
Thank you so much for writing an excellent book and provided code. I am just learning the ropes here.
I tried running simple REINFORCE algorithm as suggested in the github page. There are already bunch of issues with the versions used in requirements.txt. Especially Pytorch version which seem quite old and need python version 3.6.
Visual code no longer supports the version 3.6 for debugging

python train_agent.py --env-name CartPole-v0 --algorithm reinforce --save-name cartpole_v0

Anyway I was able to install all the required versions suggested in requiremets.txt
However I got the following error
Traceback (most recent call last): File "train_agent.py", line 85, in <module> main(args) File "train_agent.py", line 51, in main render=args.render, File "/Users/user1/Documents/rl/foundations-drl/algorithms/reinforce/reinforce_trainer.py", line 56, in train_agent action = agent.act(obs, deterministic=False) File "/Users/user1/Documents/rl/foundations-drl/algorithms/reinforce/discrete_reinforce_agent.py", line 99, in act logits = self.forward(torch.from_numpy(np.asarray(observation[0]))) File "/Users/user1/Documents/rl/foundations-drl/algorithms/reinforce/discrete_reinforce_agent.py", line 84, in forward return self.net(X) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/modules/container.py", line 119, in forward input = module(input) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/modules/linear.py", line 94, in forward return F.linear(input, self.weight, self.bias) File "/Users/user1/anaconda3/envs/rl2/lib/python3.6/site-packages/torch/nn/functional.py", line 1753, in linear return torch._C._nn.linear(input, weight, bias) RuntimeError: both arguments to matmul need to be at least 1D, but they are 0D and 2D

Please let me know if you have updated version of the code
regards,