Giter Club home page Giter Club logo

spinnet's Introduction

PWC License CC BY-NC-SA 4.0 arXiv

SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration (CVPR 2021)

This is the official repository of SpinNet, a conceptually simple neural architecture to extract local features which are rotationally invariant whilst sufficiently informative to enable accurate registration. For technical details, please refer to:

SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration
Sheng Ao*, Qingyong Hu*, Bo Yang, Andrew Markham, Yulan Guo.
(* indicates equal contribution)

[Paper] [Video] [Project page]

(1) Overview

(2) Setup

This code has been tested with Python 3.6, Pytorch 1.6.0, CUDA 10.2 on Ubuntu 18.04.

  • Clone the repository
git clone https://github.com/QingyongHu/SpinNet && cd SpinNet
  • Setup conda virtual environment
conda create -n spinnet python=3.6
source activate spinnet
conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.2 -c pytorch
conda install -c open3d-admin open3d==0.11.1
pip install "git+git://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"

(3) 3DMatch

Download the processed dataset from Google Drive, Baidu Yun (Verification code:d1vn) and put the folder into data. Then the structure should be as follows:

--data--3DMatch--fragments
              |--intermediate-files-real
              |--patches

Training

Training SpinNet on the 3DMatch dataset:

cd ./ThreeDMatch/Train
python train.py

Testing

Evaluate the performance of the trained models on the 3DMatch dataset:

cd ./ThreeDMatch/Test
python preparation.py

The learned descriptors for each point will be saved in ThreeDMatch/Test/SpinNet_{timestr}/ folder. Then the Feature Matching Recall(FMR) and Inlier Ratio(IR) can be calculated by running:

python evaluate.py [timestr]

The ground truth poses have been put in the ThreeDMatch/Test/gt_result folder. The Registration Recall can be calculated by running the evaluate.m in ThreeDMatch/Test/3dmatch which are provided by 3DMatch. Note that, you need to modify the descriptorName to SpinNet_{timestr} in the ThreeDMatch/Test/3dmatch/evaluate.m file.

(4) KITTI

Download the processed dataset from Google Drive, Baidu Yun (Verification code:d1vn), and put the folder into data. Then the structure is as follows:

--data--KITTI--dataset
            |--icp
            |--patches

Training

Training SpinNet on the KITTI dataset:

cd ./KITTI/Train/
python train.py

Testing

Evaluate the performance of the trained models on the KITTI dataset:

cd ./KITTI/Test/
python test_kitti.py

(5) ETH

The test set can be downloaded from here, and put the folder into data, then the structure is as follows:

--data--ETH--gazebo_summer
          |--gazebo_winter
          |--wood_autmn
          |--wood_summer

(6) Generalization across Unseen Datasets

3DMatch to ETH

Generalization from 3DMatch dataset to ETH dataset:

cd ./generalization/ThreeDMatch-to-ETH
python preparation.py

The descriptors for each point will be generated and saved in the generalization/ThreeDMatch-to-ETH/SpinNet_{timestr}/ folder. Then the Feature Matching Recall and inlier ratio can be caluclated by running

python evaluate.py [timestr]

3DMatch to KITTI

Generalization from 3DMatch dataset to KITTI dataset:

cd ./generalization/ThreeDMatch-to-KITTI
python test.py

KITTI to 3DMatch

Generalization from KITTI dataset to 3DMatch dataset:

cd ./generalization/KITTI-to-ThreeDMatch
python preparation.py

The descriptors for each point will be generated and saved in generalization/KITTI-to-3DMatch/SpinNet_{timestr}/ folder. Then the Feature Matching Recall and inlier ratio can be caluclated by running

python evaluate.py [timestr]

Acknowledgement

In this project, we use (parts of) the implementations of the following works:

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{ao2020SpinNet,
  title={SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration},
  author={Ao, Sheng and Hu, Qingyong and Yang, Bo and Markham, Andrew and Guo, Yulan},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2021}
}

References

[1] 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions, Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas Funkhouser, CVPR 2017.

Updates

  • 03/04/2021: The code is released!
  • 01/03/2021: This paper has been accepted by CVPR 2021!
  • 25/11/2020: Initial release!

Related Repos

  1. RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds GitHub stars
  2. SoTA-Point-Cloud: Deep Learning for 3D Point Clouds: A Survey GitHub stars
  3. 3D-BoNet: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds GitHub stars
  4. SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds GitHub stars
  5. SQN: Weakly-Supervised Semantic Segmentation of Large-Scale 3D Point Clouds with 1000x Fewer Labels GitHub stars

spinnet's People

Contributors

qingyonghu avatar 3dpointcloudlearning avatar aosheng1996 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.