Iterative Loop Method Combining Active and
Semi-Supervised Learning for Domain
Adaptive Semantic Segmentation
by Licong Guan, Xue Yuan
This repository provides the official code for the paper Iterative Loop Method Combining Active and Semi-Supervised Learning for Domain Adaptive Semantic Segmentation.
Abstract Semantic segmentation is an important technique for environment perception in intelligent transportation systems. With the rapid development of convolutional neural networks (CNNs), road scene analysis can usually achieve satisfactory results in the source domain. However, guaranteeing good generalization to different target domain scenarios remains a significant challenge. Recently, semi-supervised learning and active learning have been proposed to alleviate this problem. Semi-supervised learning can improve model accuracy with massive unlabeled data, but some pseudo labels containing noise would be generated with limited or imbalanced training data. And there will be suboptimal models if human guidance is absent. Active learning can select more effective data to intervene,while the model accuracy can not be improved because the massive unlabeled data are not used. And the probability of querying sub-optimal samples will increase when the domain difference is too large, increasing annotation cost. This paper proposes an iterative loop method combining active and semi-supervised learning for domain adaptive semantic segmentation. The method first uses semi-supervised to learn massive unlabeled data to improve model accuracy and provide more accurate selection models for active learning. Secondly, combined with the predictive uncertainty sample selection strategy of active learning, manual intervention is used to correct the pseudo-labels. Finally, flexible iterative loops achieve the best performance with minimal labeling cost. Extensive experiments show that our method establishes state-of-the-art performance on tasks of GTAV→Cityscapes, SYNTHIA→Cityscapes, improving by 4.9% mIoU and 5.2% mIoU, compared to the previous best method, respectively.
For more information on ILM-ASSL, please check our [Paper].
- Python 3.6.9
- Pytorch 1.8.1
- torchvision 0.9.1
Step-by-step installation
git clone https://github.com/licongguan/ILM-ASSL.git && cd ILM-ASSL
conda create -n ILM-ASSL python=3.6.9
conda activate ILM-ASSL
pip install -r requirements.txt
pip install torch==1.8.1+cu102 torchvision==0.9.1+cu102 -f https://download.pytorch.org/whl/torch_stable.html- Download The Cityscapes Dataset, The GTAV Dataset, and The SYNTHIA Dataset
First, the data folder should be structured as follows:
├── datasets/
│ ├── cityscapes/
| | ├── gtFine/
| | ├── leftImg8bit/
│ ├── gtav/
| | ├── images/
| | ├── labels/
│ └── synthia/
| | ├── RGB/
| | ├── LABELS/
Second, generate _labelTrainIds.png for the cityscapes dataset:
pip install cityscpaesscripts
pip install cityscpaesscripts[gui]
export CITYSCAPES_DATASET='/path_to_cityscapes'
csCreateTrainIdLabelImgsFinal, rename the gtav and synthia files for by running:
python ILM-ASSL/datasets/rename.pyBefore training, please download ResNet101 pretrained on ImageNet-1K from one of the following:
- Google Drive
- Baidu Drive Fetch Code: 0305
After that, modify model_urls in ILM-ASSL/models/resnet.py to </path/to/resnet101.pth>
We have put our model checkpoints here [Google Drive] [百度网盘] (提取码0305).
| Method | Net | budget | mIoU | Chekpoint | Where in Our Paper |
|---|---|---|---|---|---|
| ILM-ASSL | V3+ | 1% | 70.0 | Google Drive/BaiDu | Table1 |
| ILM-ASSL | V3+ | 2.2% | 75.0 | Google Drive/BaiDu | Table1 |
| ILM-ASSL | V3+ | 5.0% | 76.1 | Google Drive/BaiDu | Table1 |
| Method | Net | budget | mIoU | Chekpoint | Where in Our Paper |
|---|---|---|---|---|---|
| ILM-ASSL | V3+ | 1% | 73.2 | Google Drive/BaiDu | Table2 |
| ILM-ASSL | V3+ | 2.2% | 76.0 | Google Drive/BaiDu | Table2 |
| ILM-ASSL | V3+ | 5.0% | 76.6 | Google Drive/BaiDu | Table2 |
We provide the training scripts using Multiple GPU.
# training for GTAV to Cityscapes
# use GTAV 2000 labeled images and Cityscpaes 30(1%) labeled images
cd experiments/gtav2cityscapes/1.0%
# use torch.distributed.launch
sh train.sh <num_gpu> <port>
sh eval.shThis project is based on the following open-source projects: U2PL and RIPU. We thank their authors for making the source code publically available.
If you find this project useful in your research, please consider citing:
@article{guan2023iterative,
title={Iterative Loop Method Combining Active and Semi-Supervised Learning for Domain Adaptive Semantic Segmentation},
author={Guan, Licong and Yuan, Xue},
journal={arXiv preprint arXiv:2301.13361},
year={2023}
}If you have any problem about our code, feel free to contact
or describe your problem in Issues.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent