This repository is for 2DPASS introduced in the following paper

Xu Yan*, Jiantao Gao*, Chaoda Zheng*, Chao Zheng, Ruimao Zhang, Shuguang Cui, Zhen Li*, "2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds", ECCV 2022 [arxiv].

If you find our work useful in your research, please consider citing:

@InProceedings{yan20222dpass,
      title={2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds}, 
      author={Xu Yan and Jiantao Gao and Chaoda Zheng and Chao Zheng and Ruimao Zhang and Shuguang Cui and Zhen Li},
      year={2022},
      booktitle={ECCV}
}

• 2022-09-20 We release codes for SemanticKITTI single-scan and NuScenes 🚀!
• 2022-07-03 2DPASS is accepted at ECCV 2022 🔥!
• 2022-03-08 We achieve 1st place in both single and multi-scans of SemanticKITTI and 3rd place on NuScenes-lidarseg 🔥!

• pytorch >= 1.8
• yaml
• easydict
• lightning (tested with pytorch_lightning==1.3.8 and torchmetrics==0.5)
• torch-scatter (pip install torch-scatter -f https://data.pyg.org/whl/torch-1.9.0+${CUDA}.html)
• nuScenes-devkit (optional for nuScenes)
• spconv (tested with spconv==2.1.16 and cuda==11.1, pip install spconv-cu111==2.1.16)

Please download the files from the SemanticKITTI website and additionally the color data from the Kitti Odometry website. Extract everything into the same folder.

./dataset/
├── 
├── ...
└── SemanticKitti/
    ├──sequences
        ├── 00/           
        │   ├── velodyne/	
        |   |	├── 000000.bin
        |   |	├── 000001.bin
        |   |	└── ...
        │   └── labels/ 
        |   |   ├── 000000.label
        |   |   ├── 000001.label
        |   |   └── ...
        |   └── image_2/ 
        |   |   ├── 000000.png
        |   |   ├── 000001.png
        |   |   └── ...
        |   calib.txt
        ├── 08/ # for validation
        ├── 11/ # 11-21 for testing
        └── 21/
	    └── ...

Please download the Full dataset (v1.0) from the NuScenes website with lidarseg and extract it.

./dataset/
├── 
├── ...
└── nuscenes/
    ├──v1.0-trainval
    ├──v1.0-test
    ├──samples
    ├──sweeps
    ├──maps
    ├──lidarseg

You can run the training with

cd <root dir of this repo>
python main.py --log_dir 2DPASS_semkitti --config config/2DPASS-semantickitti.yaml --gpu 0

The output will be written to logs/SemanticKITTI/2DPASS_semkitti by default.

cd <root dir of this repo>
python main.py --log_dir 2DPASS_nusc --config config/2DPASS-nuscenese.yaml --gpu 0 1 2

We take SemanticKITTI as an example.

cd <root dir of this repo>
python main.py --log_dir baseline_semkitti --config config/2DPASS-semantickitti.yaml --gpu 0 --baseline_only

You can run the testing with

cd <root dir of this repo>
python main.py --config config/2DPASS-semantickitti.yaml --gpu 0 --test --num_vote 12 --checkpoint <dir for the pytorch checkpoint>

Here, num_vote is the number of views for the test-time-augmentation (TTA). We set this value to 12 as default (on a Tesla-V100 GPU), and if you use other GPUs with smaller memory, you can choose a smaller value. num_vote=1 denotes there is no TTA used, and will cause about ~2% performance drop.

You can download the models with the scores below from this Google drive folder.

Here, we train the model for SemanticKITTI with more epochs and thus gain the higher mIoU. Note that the results on benchmarks are gained by training with additional validation set and using instance-level augmentation.

Code is built based on SPVNAS, Cylinder3D, xMUDA and SPCONV.

This repository is released under MIT License (see LICENSE file for details).