Implements a OpenAI gym robot snake simulation environment using PyBullet, where the goal is to move a snake robot towards a certain location. You can train Reinforcement Learning agents or define a manual movement function.

This program was written using Python 3.8.8. Install all depencencies using:

pip install -r requirements.txt

Next, install PyTorch (v1.8.1) using the instructions here.

The program can be run using the snake.py script. From here, you can use premade manual movement function using:

python snake.py manual

or train a Proximal Policy Optimization (PPO) agent using:

python snake.py train

To test trained models, use:

python snake.py test --model <path_to_model>

If the --model flag is left empty, a random agent is used. For more flag customization, use the -h flag. Several trained modes are located in the models folder. Make sure the snake is of appropriate length (using the snake_length argument).

For a list of all options (training settings etc.), please use:

python snake.py --h

The snake robot is build from a number of Revolute Joints. For a single joint, the state space is defined as:

[angle, speed]

The state of the full environment is defined as the combination of all N joint states, plus the x, y coordinates of the snake head:

[headX, headY, angle1, ... angleN, speed1, ..., speedN]

The reward returned to the agent is defined as the negated distance towards the goal. When the goal is reached (head is within 1 meter of the goal), a reward of 100 is returned.

The agent can control the position of each of the robot's N joints. The action space is thus defined as follows:

[pos1, ..., posN]

• Pybullet docs
• Pybullet examples
• OpenAI env tutorial
• RL snake paper