This repository contains code for robot exploration under uncertainty that uses graph neural networks (GNNs) in conjunction with deep reinforcement learning (DRL), enabling decision-making over graphs containing exploration information to predict a robot’s optimal sensing action in belief space. A demonstration video can be found here.

• Python 3
• PyTorch
• PyTorch Geometric
• gtsam (Georgia Tech Smoothing and Mapping library)

  git clone -b emex --single-branch https://bitbucket.com/jinkunw/gtsam
  cd gtsam
  mkdir build && cd build
  cmake ..
  sudo make install
  

• pybind11 (pybind11 — Seamless operability between C++11 and Python)

  git clone https://github.com/pybind/pybind11.git
  cd pybind11
  mkdir build && cd build
  cmake ..
  sudo make install
  

You can use the following commands to download and compile the package.

git clone https://github.com/RobustFieldAutonomyLab/DRL_graph_exploration.git
cd DRL_graph_exploration
mkdir build && cd build
cmake ..
make

There is an unsolved memory leak issue in the C++ code. So we use the python subprocess module to run the simulation training. The data in the process will be saved and reloaded every 10000 iterations.

• To run the saved policy:

  cd DRL_graph_exploration/scripts
  python3 test.py
  

• To show the average reward during the training:

  cd DRL_graph_exploration/data
  tensorboard --logdir=torch_logs
  

• To train your own policy:

  cd DRL_graph_exploration/scripts
  python3 train.py
  

Please cite our paper if you use any of this code:

@inproceedings{chen2020autonomous,
  title={Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs},
  author={Chen, Fanfei and Martin, John D. and Huang, Yewei and Wang, Jinkun and Englot, Brendan},
  booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  pages={6140--6147},
  year={2020},
  organization={IEEE}
}

• em_exploration