This repository provides the implementation of NeurIPS2020 Paper:

Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation, and supplementary materials can be downloaded here.

This repository is based on maskrcnn-benchmark and DetNAS.

Check INSTALL.md for installation instructions.

Download and extract COCO 2017 train and val images with annotations from http://cocodataset.org. Following Panoptic-FPN, we predict 53 stuff classes plus a single ‘other’ class for all 80 thing classes for semantic segmentation. Thus we squeeze all thing classes label to id 0 and create the folder PanopticAnnotation. For architecture search, we randomly split coco train set into nas_train and nas_val set (5k images). We provide the download link here.

We expect the directory structure to be the following:

maskrcnn-benchmark/
    - datasets/
        - coco/
            - train2017/
            - val2017/
            - nas/
                - instances_nas_train2017.json
                - instances_nas_val2017.json
            - annotations/  
                - ...
                - PanopticAnnotation/ 

We pretrain our model on ImageNet using the same search space and training schedule as DetNAS. Please follow the instructions here or you can download our pretrain model autopanoptic_imagenet_pretrain.pkl directly.

export CURRENT_DIR={your_root_dir}
cd $CURRENT_DIR
sh scripts/architecture_search.sh

We provide our search log here and searched architecture in maskrcnn-benchmark/test_models/. Note that you should change MODEL.WEIGHT to the correct pretrain model path before architecture search.

export CURRENT_DIR={your_root_dir}
cd $CURRENT_DIR
sh scripts/pretrain_searched_model.sh

We provide our imagenet pretrain model here.

export CURRENT_DIR={your_root_dir}
cd $CURRENT_DIR
sh scripts/train_searched_model.sh

We provide our training log and panoptic model. Note that you should change MODEL.WEIGHT to the correct imagenet pretrain model path before retraining.

bash
export CURRENT_DIR={your_root_dir}
cd $CURRENT_DIR
sh scripts/eval.sh

We provide our evaluation log.

@article{wu2020auto, 
    title={Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation}, 
    author={Wu, Yangxin and Zhang, Gengwei, and Xu, Hang and Liang, Xiaodan and Lin, Liang}, 
    journal={Advances in Neural Information Processing Systems},
    volume={33},
    year={2020}
}