GCRL-Collection
This repo relates to the survey paper <Goal-Conditioned Reinforcement Learning: Problems and Solutions>. We collects widely used benchmark environments and conclude a series of research works for goal-conditioned reinforcement learning (GCRL).
Summary of collected algorithms
| Approach | Resolved Goal Type | Sub-Goal | Relabel | Optimization |
|---|---|---|---|---|
| UVFA | Vector/Image | ✅ | ||
| HER | Vector/Image | ✅ | ||
| Laplacian Representation Learning | Vector/Image | ✅ | ||
| SR | Vector/Image | ✅ | ||
| DDL | Vector/Image | ✅ | ✅ | |
| SoRB | Vector/Image | ✅ | ||
| Skew-Fit | Vector | ✅ | ||
| HER with Demonstrations | Vector | ✅ | ||
| GoalGAN | Vector | ✅ | ||
| Sub-goal Discovery | Vector | ✅ | ||
| HGG | Vector | ✅ | ||
| CHER | Vector | ✅ | ||
| G-HER | Vector | ✅ | ||
| Hindsight Planner | Vector | ✅ | ||
| GDP | Vector | ✅ | ||
| VDS | Vector | ✅ | ||
| MEGA | Vector | ✅ | ✅ | |
| PlanGAN | Vector | ✅ | ||
| AIM | Vector | ✅ | ||
| I-HER | Vector | ✅ | ||
| GCSL | Vector | ✅ | ||
| MapGo | Vector | ✅ | ✅ | |
| L3P | Vector | ✅ | ||
| RIG | Image | ✅ | ||
| DISCERN | Image | ✅ | ||
| HVF | Image | ✅ | ||
| AMIGo | Image | ✅ | ||
| GAP | Image | ✅ | ||
| LEXA | Image | ✅ |
Widely used benchmark environments.
Note almost all environments support various kinds of goal structures, i.e., vector, image, language, etc.
| Environment | Benchmark Type | Common Used Goal Structure | Description |
|---|---|---|---|
| FetchReach | Gym Robotics/Mujoco | Vector/Image | A robotic arm reaches a target position |
| FetchPush | Gym Robotics/Mujoco | Vector/Image | A robotic arm pushes a block to a target position |
| FetchPickAndPlace | Gym Robotics/Mujoco | Vector/Image | A robotic arm first picks up the block then places it to a target position |
| FetchSlide | Gym Robotics/Mujoco | Vector/Image | A robotic arm slides a buck to a goal position |
| HandManipulateBlock | Gym Robotics/Mujoco | Vector/Image | Orient a block using a robot hand |
| HandManipulateEgg | Gym Robotics/Mujoco | Vector/Image | Orient an egg using a robot hand |
| HandManipulatePen | Gym Robotics/Mujoco | Vector/Image | Orient a pen using a robot hand |
| HalfCheetah | Mujoco | Vector/Image | Make a 2D cheetah robot run and keep a specific speed |
| AntLocomotion/Ant Maze | Mujoco | Vector | Make a 3D four-legged robot walk to a target position |
| Sawyer | Mujoco | Vector | A sawyer robot reaches a target position |
| Reacher | Mujoco | Vector/Image | A reacher robot reaches a target position |
| N-D Mass | Mujoco | Vector | A ball reaches a target position |
| DoorOpening | Mujoco | Image | A sawyer robot opens a door by latching onto the handle |
| Dy-Reaching | Mujoco | Image | A robotic arm reaches a target position moving along a straight line |
| Dy-Circling | Mujoco | Image | A robotic arm reaches a target position moving along a circle |
| Dy-Pushing | Mujoco | Image | A robotic arm moves the box to a moving target position |
| Dy-Pouring | Mujoco | Image | A robotic arm grips the can and pours the water into a cup |
| WaterMaze | DeepMind Lab | Image | Get to a target position described by an image |
| Seaquest | Atari | Image | Get to a target position described by an image |
| Montezuma’s Revenge | Atari | Image | Get to a target position described by an image |
| Reacher | DeepMind Control Suit | Image | A two-link planar reacher poses an aimed shape |
| Manipulator | DeepMind Control Suit | Image | A 3-DoF finger robot rotates the body into a target orientation |
| Finger | DeepMind Control Suit | Image | A planar manipulator brings an object in a target location |
| Place Object in Tray | PyBullet | Image | A robotic arm puts toys into a tray |
| Opening Drawer | PyBullet | Image | A robotic arm opens the drawer |
| PickandPlace | PyBullet | Image | A robotic arm first picks up the block then places it to a target position |
| Five Object Manipulation | CLEVR-Robot/Mujoco | Image/Language | A point mass agent arranges, orders and sorts 5 fixed objects |
| Diverse Object Manipulation | CLEVR-Robot/Mujoco | Image/Language | A point mass agent orders objects in different shapes and colors |
| Playground | Playground | Image/Language | A 2D hand moves and grasps/releases different objects |
| VizDoom | ViZDoom | Image/Language | AI bots reach specific place in the first person shooter 3D game Doom |
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent