tambetm / gymexperiments Goto Github PK

with mujoco: when run a long time, programe is crack.

1
For comparison, real replay memory contains 21739 experiences
WARNING: Nan, Inf or huge value in QACC at DOF 0. The simulation is unstable. Time = 0.1200.

Performing imagination rollout for 5 steps
Traceback (most recent call last):
File "naf_ir.py", line 255, in
postobs, rewards, terminals = ir_model.predict(preobs, actions, timesteps + i)
File "/home/ubuntu/work/gymexperiments/irmodel.py", line 105, in predict
obsmeans = obsmodel.predict(X)[0]
File "/usr/local/lib/python2.7/dist-packages/sklearn/linear_model/base.py", line 200, in predict
return self._decision_function(X)
File "/usr/local/lib/python2.7/dist-packages/sklearn/linear_model/base.py", line 183, in _decision_function
X = check_array(X, accept_sparse=['csr', 'csc', 'coo'])
File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 398, in check_array
_assert_all_finite(array)
File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 54, in _assert_all_finite
" or a value too large for %r." % X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

2
Done fitting, 200 timesteps covered.
Performing imagination rollout for 5 steps
Done, fictional replay memory now contains 100000 experiences
For comparison, real replay memory contains 21001 experiences
WARNING: Nan, Inf or huge value in QACC at DOF 0. The simulation is unstable. Time = 0.0900.

Performing imagination rollout for 5 steps
Traceback (most recent call last):
File "naf_ir.py", line 255, in
postobs, rewards, terminals = ir_model.predict(preobs, actions, timesteps + i)
File "/home/ubuntu/work/gymexperiments/irmodel.py", line 105, in predict
obsmeans = obsmodel.predict(X)[0]
File "/usr/local/lib/python2.7/dist-packages/sklearn/linear_model/base.py", line 200, in predict
return self._decision_function(X)
File "/usr/local/lib/python2.7/dist-packages/sklearn/linear_model/base.py", line 183, in _decision_function
X = check_array(X, accept_sparse=['csr', 'csc', 'coo'])
File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 398, in check_array
_assert_all_finite(array)
File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 54, in _assert_all_finite
" or a value too large for %r." % X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

Hi! Is there any chance you would be open to licensing this repo? I'd really like to tinker with your implementation of NAF.

Fitting imagination rollout model...
Done fitting, 200 timesteps covered.
Performing imagination rollout for 5 steps
Traceback (most recent call last):
File "naf_ir.py", line 255, in
postobs, rewards, terminals = ir_model.predict(preobs, actions, timesteps + i)
File "/home/ubuntu/gymexperiments/irmodel.py", line 107, in predict
postob = np.random.multivariate_normal(obsmeans, obscov)
File "mtrand.pyx", line 4689, in mtrand.RandomState.multivariate_normal (numpy/random/mtrand/mtrand.c:32216)
ValueError: mean must be 1 dimensional

do you run this code in gpu?

before i am run it in cpu is ok! gpu server the theano is update to git latest version,maybe some lib conflict

That's the error I get when I try to run python naf_ir.py Pendulum-v0. Is there a reason for it?

Hi, I am learning how to implement RL algorithms and I found your code very intuitive. My only confusion is with the multiprocessing of your A2C implementation. I am not very familiar with Python multiprocessing, but is this synchronous or asynchronous? I can't find the logic in the code that causes the runners to synchronize before updating the trainer.

Also, if you would not mind helping me understand the implementation differences better, can you tell me whether this approach of using multiple processes of copied agent models is better in terms of performance or training time than the alternative approach of parallelizing multiple copies of the environment and executing only one model (as in the OpenAI baseline)?