Code to replicate A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning, which contains code for training a simulated or real A1 quadrupedal robot to walk. Project page: https://sites.google.com/berkeley.edu/walk-in-the-park
Install dependencies:
pip install -r requirements.txt
To install the robot SDK, first install the dependencies in the README.md
To build, run:
cd real/third_party/unitree_legged_sdk
mkdir build
cd build
cmake ..
make
Finally, copy the built robot_interface.XXX.so
file to this directory.
Example command to run simulated training:
MUJOCO_GL=egl XLA_PYTHON_CLIENT_PREALLOCATE=false python train_online.py --env_name=A1Run-v0 \
--utd_ratio=20 \
--start_training=1000 \
--max_steps=100000 \
--config=configs/droq_config.py
To run training on the real robot, add --real_robot=True
Tested on Ubuntu 22.04.
First, comment out jax, tensorflow and dmcgym from the requirements.txt
so that it looks like follows:
pip
# jax[cuda]
# --find-links https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
flax
tqdm
gym
# tensorflow
tensorflow_probability
ml_collections
wandb
dm_control
# dmcgym @ git+https://github.com/ikostrikov/dmcgym
pybullet
attrs
filterpy
Then, create a conda environment with python 3.9 as follows:
conda create -n witp python==3.9
conda activate witp
Then, install jax with cuda again so that it can recognize the GPU:
pip install -U "jax[cuda12_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
Then, install tensorflow according to the official instructions:
pip install --extra-index-url https://pypi.nvidia.com tensorrt-bindings==8.6.1 tensorrt-libs==8.6.1
pip install -U tensorflow[and-cuda]
At this point, it's worth checking whether both tensorflow and jax can recognize the gpu. Run python
, and then run the following commands:
from jax.lib import xla_bridge
import tensorflow as tf
print(xla_bridge.get_backend().platform) ## This should print "gpu"
print(tf.config.list_physical_devices('GPU')) ## This should print a list of your GPUs
Then, install the walk_in_the_park repo as follows:
pip install -r requirements.txt
Finally, clone the dmcgym package either inside or outside this directory from https://github.com/ikostrikov/dmcgym
. Once it is cloned, go into the requirements.txt
file and remove the mujoco gym dependency so that it looks like follows:
gym >= 0.21.0, < 0.24.1
numpy >= 1.20.2
dm_control >= 1.0.0
Then, install dmcgym by running
cd ~/path/to/dmcgym
pip install -e .
Then, I needed to modify the file so that the saved_checkpoint directory is global (orbax required this)
I also needed to install the following conda package to be able to render mujoco:
conda install -c conda-forge libstdcxx-ng