OpenAI gym is currently one of the most widely used toolkit for developing and comparing reinforcement learning algorithms. Unfortunately, for several challenging continuous control environments it requires the user to install MuJoCo, a commercial physics engine which requires a license to run for longer than 30 days. Such a commercial barrier hinders open research, especially in the perspective that other appropriate physics engines exist. This repository provides alternative implementations of the original MuJoCo environments which can be used for free. The environments have been reimplemented using BulletPhysics' python wrapper pybullet, such that they seamlessly integrate into the OpenAI gym framework. In order to show the usability of the new environments, several RL agents from the Keras-RL are configured to be trained out of the box. To further simplify the training of agents, a Trainer class was implemented which helps to capture commandline arguments in a unified fashion. The trainer provides a set of standard arguments, but additional arguments can be defined by the agent and the environment to enable the researcher to provide special parameters to either one.

Thanks to matpalm for a beautiful example on how to use pybullet with OpenAI Gym.

The following environments are available to be run with the agents:

• CartPole Environment [CartPolev0Env]
• CartPole Environment with Detached Pole (2D control problem) [Detached2DCartPolev0Env]
• Motion Environment with different morphologies [Motionv0Env]
• Inverted Double Pendulum [InvertedDoublePendulum-v1]
• Reacher [Reacher-v1]
• Half Cheetah [HalfCheetah-v1]
• Swimmer [Swimmer-v1]
• Hopper [Hopper-v1]
• Walker 2D [Walker-2d-v1]
• Ant [Ant-v1]
• Humanoid [Humanoid-v1]
• Humanoid Standup [HumanoidStandup-v1]

The following agents are available to be run with the environments:

• Keras DQN Agent [KerasDQNAgent]
• Keras DDQN Agent [KerasDDQNAgent]
• Keras DDPG Agent [KerasDDPGAgent]
• Keras NAF Agent [KerasNAFAgent]
• Keras CEM Agent [KerasCEMAgent]

• BulletPhysics library with pybullet installed on the python path: Pybullet quickstart guide [pdf]
• python with numpy and h5py - pip install numpy h5py
• Keras - pip install keras
• TensorFlow - installation according to page
• Keras-RL - installation according to page
• OpenAI Gym - installation according to page