Reproducing benchmark scores about evojax HOT 2 CLOSED

Hey @EdoardoPona thanks for your interest in EvoJAX!

Many algorithms are sensitive to hyper-parameters, so hyper-parameter searches have been conducted and recorded in scripts/benchmarks/Readme.md. For example scripts/benchmarks/figures/ARS/cartpole_easy.png shows that ARS on Cartpole (Easy) task would have a good performance with init_stdev = 0.1 and lrate_init=0.1.

Thus it is possible to reproduce such result by setting init_stdev: 0.1 and lrate_init: 0.1 in scripts/benchmarks/configs/ARS/cartpole_easy.yaml and rerunning python train.py -config configs/ARS/cartpole_easy.yaml. In doing so, I got the following result that is consistent with the table:

cartpole_easy: 2022-09-26 02:21:50,723 [INFO] [TEST] #tests=100, max=933.8828, avg=913.0647, min=168.9905, std=76.7758

Please let me know how it works for you and if there is any further questions!

from evojax.

Thank you for the clarification! All works correctly now.

My misunderstanding was due to the fact I thought the hyper-params contained in the .yaml files linked in the table such as scripts/benchmarks/configs/ARS/cartpole_easy.yaml were the final optimised ones, instead it reports init_stdev: 0.03 and lrate_init: 0.01.
I did not read the heatmaps correctly.

from evojax.