The download links of VIPSeg

(20220621) We refined the annotations and further improved the quality of VIPSeg-Dataset.

Google Drive: https://drive.google.com/file/d/1B13QUiE82xf7N6nVHclb4ErN-Zuai-sZ/view?usp=sharing

Baidu YunPan: 链接：https://pan.baidu.com/s/18l05aTnsQCaiHYndSc8SaA 提取码：qllb 

The dataset is organized as following:

NOTE: For panoptic masks in panomask/, the IDs of categories are from 0 to 124. "0" denotes the VOID class. For "stuff" classes, the value of masks is the same as the category ID. For "thing" classes, the value of masks is "category_id x 100 + instance_id". For instance, the category ID of "person" is 61. Then values of masks of the "person" instances are "6100","6101",... Thus, values of masks larger than 124 are belonging to things, otherwise it is stuff.

NOTE: The files "change2_720p.py,create_panoptic_video_labels.py,splitjson.py" are togethor with the dataset. Please download the dataset and unzip it.

pip install git+https://github.com/cocodataset/panopticapi.git

python change2_720p.py

python create_panoptic_video_labels.py

python splitjson.py

The COCO format dataset is organized as following:

NOTE: The category IDs and colors are shown in panoVIPSeg_categories.json.

This implementation is based on PanopticFCN. Please refer to PanopticFCN for installation instructions.

Download pretrained weight and put it in ClipPanoFCN.

cd ClipPanoFCN

sh run_train.sh

sh run_eval.sh

The model weight for inference can be downloaded here. Put it in ./output

@inproceedings{miao2022large,

  title={Large-scale Video Panoptic Segmentation in the Wild: A Benchmark},

  author={Miao, Jiaxu and Wang, Xiaohan and  Wu, Yu and Li, Wei and Zhang, Xu and Wei, Yunchao and Yang, Yi},

  booktitle={Proceedings of the {IEEE} Conference on Computer Vision and Pattern Recognition},

  year={2022}

}

@inproceedings{miao2021vspw,
  title={Vspw: A large-scale dataset for video scene parsing in the wild},
  author={Miao, Jiaxu and Wei, Yunchao and Wu, Yu and Liang, Chen and Li, Guangrui and Yang, Yi},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={4133--4143},
  year={2021}
}

The data is released for non-commercial research purpose only.