This project was initiated in the RL course of Fall 2021 at The African Master's in Machine Intelligence (AMMI) as a course-project where we implemented the SAC algorithm (Haarnoja et al.) for continuous control tasks. It is now an open project where we care to design code bases and benchmarks for RL algorithms in order to ease the development of new algorithms. We are designing this repo based on existing repositories as well as original papers to produce better general implementations for a selected set of algorithms.

Algorithms we are re-implementing/plannning to re-implement:

🟢 Done || 🟡 Now || 🟣 Next || 🔴 No plan

We aim to finetune our implementations to work with a generalized set of hyperparametrs across different algorithms. We are working with the following hyperparameters in the mean time:

🟩 Best || 🟨 Good || 🟥 Bad || 🟦 In progress

In thoe following we evaluate our code on the following environments. Download gifs from this Google drive folder at drive. Results are averaged across 3 random seeds, and smoothed with 0.75 Exponential Moving Average.

Hopper-v2	Walker2d-v2
HalfCheetah-v2	Ant-v2

DClaw Valve Turning	ShadowHand Cube Re-orientation

Move into AMMI-RL/ directory, and then run the following:

conda create -n ammi-rl python=3.8

pip install -e .

pip install numpy torch wandb gym

If you want to run MuJoCo Locomotion tasks, and ShadowHand, you should install MuJoCo first (it's open sourced until 31th Oct), and then install mujoco-py:

sudo apt-get install ffmpeg

pip install -U 'mujoco-py<2.1,>=2.0'

If you are using A local GPU of Nvidia and want to record MuJoCo environments issue link, run:

unset LD_PRELOAD

Move into AMMI-RL/ directory, and then run the following:

conda create -n ammi-rl python=3.8

pip install -e .

pip install numpy torch wandb gym

If you want to run MuJoCo Locomotion tasks, and ShadowHand, you should install MuJoCo first (it's open sourced until 31th Oct), and then install mujoco-py:

brew install ffmpeg gcc

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.mujoco/mujoco200/bin

pip install -U 'mujoco-py<2.1,>=2.0'

If you are using A local GPU of Nvidia and want to record MuJoCo environments issue link, run:

unset LD_PRELOAD

Move into AMMI-RL/ directory, and then:

python experiment.py -cfg <cfg_file-.py> -seed <int>

for example:

python experiment.py -cfg sac_hopper -seed 1

To evaluate a saved policy model, run the following command:

python evaluate_agent.py -env <env_name> -alg <alg_name> -seed <int> -EE <int>

for example:

python evaluate_agent.py -env Walker2d-v2 -alg SAC -seed 1 -EE 5

(last name alphabetical order) | contribution

• Rami Ahmed | VPG, PPO, SAC, MB{PPO, SAC}
• Wafaa Mohammed | SAC
• Ruba Mutasim | SAC
• MohammedElfatih Salah | SAC

• Bilal Piot, Corentin Tallec and Florian Strub (During RL Course Fall 2021)
• Vlad Mnih, Eszter Vértes and Theophane Weber (During Rami's AMMI project)

This repo was inspired by many great repos, mostly the following ones (not necessarily in order):

• RLKit
• CleanRL
• SpinningUp
• RL for MuJoCo
• MBPO-PyTorch
• Stabel Baselines
• Youtube-Code-Repository