Pytorch implementation of our Label Evolution with Single Point Supervision (LESPS). [Paper]   [Web]

News: We recommend our newly-released repository BasicIRSTD, an open-source and easy-to-use toolbox for infrared small target detction. [link]

Fig. 1. Illustrations of mapping degeneration under point supervision. CNNs always tend to segment a cluster of pixels near the targets with low confidence at the early stage, and then gradually learn to predict groundtruth point labels with high confidence.

Fig. 2. Quantitative and qualitative illustrations of mapping degeneration in CNNs.

Fig. 3. Illustrations of Label Evolution with Single Point Supervision (LESPS). During training, intermediate predictions of CNNs are used to progressively expand point labels to mask labels. Black arrows represent each round of label updates.

• Python 3
• pytorch (1.2.0), torchvision (0.4.0) or higher
• numpy, PIL
  
  

• NUAA-SIRST   [download dir]   [paper]
• NUDT-SIRST   [download dir]   [paper]
• IRSTD-1K   [download dir]   [paper]

SIRST3 is used for training, and is a combination of NUAA-SIRST, NUDT-SIRST, IRSTD-1K datasets.

• Our project has the following structure:

  ├──./datasets/
  │    ├── images
  │    │    ├── XDU0.png
  │    │    ├── Misc_1.png
  │    │    ├── ...
  │    │    ├── 001327.png
  │    ├── masks
  │    │    ├── XDU0.png
  │    │    ├── Misc_1.png
  │    │    ├── ...
  │    │    ├── 001327.png
  │    ├── img_idx
  │    │    ├── train_SIRST3.txt
  │    │    ├── test_SIRST3.txt  
  │    │    ├── test_NUAA-SIRST.txt
  │    │    ├── test_NUDT-SIRST.txt
  │    │    ├── test_IRSTD-1K.txt
  

python train.py --model_names ['DNANet', 'ALCNet', 'ACM'] --dataset_names ['SIRST3'] --label_type 'centroid'

python train.py --model_names ['DNANet', 'ALCNet', 'ACM'] --dataset_names ['SIRST3'] --label_type 'coarse'

python test.py --model_names ['DNANet', 'ALCNet', 'ACM'] --pth_dirs None --dataset_names ['NUAA-SIRST', 'NUDT-SIRST', 'IRSTD-1K']

python test.py --model_names ['DNANet', 'ALCNet', 'ACM'] --pth_dirs ['SIRST3/DNANet_full.pth.tar', 'SIRST3/DNANet_LESPS_centroid.pth.tar', 'SIRST3/DNANet_LESPS_coarse.pth.tar','SIRST3/ALCNet_full.pth.tar', 'SIRST3/ALCNet_LESPS_centroid.pth.tar', 'SIRST3/ALCNet_LESPS_coarse.pth.tar','SIRST3/ACM_full.pth.tar', 'SIRST3/ACM_LESPS_centroid.pth.tar', 'SIRST3/ACM_LESPS_coarse.pth.tar']--dataset_names ['NUAA-SIRST', 'NUDT-SIRST', 'IRSTD-1K']

Fig. 4. IoU and visualize results of mapping degeneration with respect to different characteristics of targets (i.e.,(a) intensity, (b) size, (c) shape, and (d) local background clutter) and point labels (i.e.,(e) numbers and (f) locations). We visualize the zoom-in target regions of input images with GT point labels (i.e., red dots in images) and corresponding CNN predictions (in the epoch reaching maximum IoU).

Table 1. Average results achieved by DNAnet with (w/) and without (w/o) LESPS under centroid, coarse point supervision together with full supervision.

Fig. 5. Quantitative and qualitative results of evolved target masks.

Fig. 6. Visualizations of regressed labels during training and network predictions during inference with centroid and coarse point supervision.

Fig. 7. PA (P) and IoU (I) results of LESPS with respect to (a) initial evolution epoch, (b) Tb and (c) k of evolution threshold, and (d) evolution frequency.

Table 2. Results of different methods. “CNN Full”, “CNN Centroid”, and “CNN Coarse” represent CNN-based methods under full supervision, centroid and coarse point supervision. “+” represents CNN-based methods equipped with LESPS.

Fig. 8. Visual detection results of different methods. Correctly detected targets and false alarms are highlighted by red and orange circles, respectively.

Table 3. Results of DNA-Net trained with pseudo labels generated by input intensity threshold, LCM-based methods and LESPS under centroid and coarse point supervision. Best results are shown in boldface.

@article{LESPS,
  author = {Ying, Xinyi and Liu, Li and Wang, Yingqian and Li, Ruojing and Chen, Nuo and Lin, Zaiping and Sheng, Weidong and Zhou, Shilin},
  title = {Mapping Degeneration Meets Label Evolution: Learning Infrared Small Target Detection with Single Point Supervision},
  journal = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2023},
}

Welcome to raise issues or email to [email protected] for any question.