modestyachts / ars Goto Github PK

anyone know how to solve the problem?

Hello, thank you guys for this great code, I've been using it to my research. I wanted to know if there is some code for the Basic Random Search only.

Hi,
First and foremost, thanks for sharing the code. This is greatly appreciated.

Currently testing ARS in other learning environments and found that for very difficult environments the users of the code might face a divide by zero error, particularly at early stages of the learning process (ie, zero reward in all the initial rollouts).

# normalize rewards by their standard deviation
rollout_rewards /= np.std(rollout_rewards)

Thanks,

(If there is a better forum to ask clarification questions, please let me know!)

In aggregate_rollouts in code/ars.py, I see that the code does

        rollout_ids_one = [worker.do_rollouts.remote(policy_id,
                                                 num_rollouts = num_rollouts,
                                                 shift = self.shift,
                                                 evaluate=evaluate) for worker in self.workers]

        rollout_ids_two = [worker.do_rollouts.remote(policy_id,
                                                 num_rollouts = 1,
                                                 shift = self.shift,
                                                 evaluate=evaluate) for worker in self.workers[:(num_deltas % self.num_workers)]]

What is the purpose of doing the rollouts twice, with one doing num_rollouts rollouts per worker and the other doing 1 rollout per worker for num_deltas % self.num_workers workers?

how to solve this error

File "ars.py", line 409, in <module> local_ip = socket.gethostbyname(socket.gethostname()) socket.gaierror: [Errno 8] nodename nor servname provided, or not known

can you explain to me or maybe just give me some pointer about this. what should i do if i want to make the agent output some discrete action? thank you

In the aforementioned blog post, one step in computing the variance looks like this:

However, in this project, the corresponding step looks like this:

Since this

is the mathematical equation corresponding to this computation, it's looking like the blog post implementation is in accordance to the mathematical formulation while this project's implementation is not. Or am I missing something?

Thanks!

Hi,
I run the expert policy python run_policy.py ../trained_policies/Humanoid-v1/policy_reward_11600/lin_policy_plus.npz Humanoid-v1 --render --num_rollouts 20 and stuck at the first few lines.

I got the error like this:

loading and building expert policy
Traceback (most recent call last):
  File "run_policy.py", line 62, in <module>
    main()
  File "run_policy.py", line 23, in main
    lin_policy = lin_policy.items()[0][1]
TypeError: 'ItemsView' object does not support indexing

Could anyone tell me how to fix this?

Hi and thanks for sharing the code.
I've tried to run the training process on a different environment such as the BipedalWalkerHardcore-v2 but it seems that is not able to learn anything. I even tried with different shift values as noted in the code comments but still in the end I get a negative reward. Should we train for longer or there any hyperparams that we are missing?

Hi,

We want to use trained policies data in Nevergrad for the purpose of benchmarking Mujoco envs.
We thus add ARS license in the concerned folder and link your repository in our code, see PR: facebookresearch/nevergrad#790
To avoid license issue, could you let us know if it is fine for you ?

Thank you !

hello
i have mujoco 150 and when i run ARS.py file i got this error

Please put your binaries into ~/.mujoco/mjpro131 or set MUJOCO_PY_MJPRO_PATH. Follow the instructions on https://github.com/openai/mujoco-py for setup.') mujoco_py.error.MujocoDependencyError: Found your MuJoCo license key but not binaries. Please put your binaries into ~/.mujoco/mjpro131 or set MUJOCO_PY_MJPRO_PATH. Follow the instructions on https://github.com/openai/mujoco-py for setup

Dear authors,
just for halfcheetah-v1, l use multiple seeds to try to get the exp result in table1&table2 in paper:
python code/ars.py
but it seems negative. could you give some guide?
thanks so much!

I have no idea about why we need to subtract a shift from reward, and how to set this value?