An empirical study on few-shot counting using segment anything (SAM)

Online Demo

arXiv

Meta AI recently released the Segment Anything model [SAM], which has garnered attention due to its impressive performance in class-agnostic segmenting. In this study, we explore the use of SAM for the challenging task of few-shot object counting, which involves counting objects of an unseen category by providing a few bounding boxes of examples. We compare SAM's performance with other few-shot counting methods and find that it is currently unsatisfactory without further fine-tuning, particularly for small and crowded objects.

Install python dependencies. We use conda and python 3.10.4 and PyTorch 1.13.1

conda env create -f env.yaml

• For FSC-147: Images can be downloaded from here: https://drive.google.com/file/d/1ymDYrGs9DSRicfZbSCDiOu0ikGDh5k6S/view?usp=sharing

• For COCO val2017: Images can be downloaded from here: https://cocodataset.org/

Download the ViT-H SAM model

• For FSC-147:

python test_FSC.py --data_path <FSC-147 dataset path> --model_path <path to ViT-H SAM model>

• For COCO val2017:

python test_coco.py --data_path <coco val2017 dataset path\> --model_path <path to ViT-H SAM model>

You can run vis_FSC.ipynb for FSC-147 or vis_coco.ipynb for coco.

We thank facebookresearch for their segment-anything model [project], cvlab-stonybrook for their Learning To Count Everything [project] and coco [datasets].

If you find the code useful, please cite:

@article{ma2023countanything,
  title={CAN SAM COUNT ANYTHING? AN EMPIRICAL STUDY ON SAM COUNTING},
  author={Ma, Zhiheng and Hong, Xiaopeng and Shangguan Qinnan},
  journal={arXiv preprint arXiv:2304.10817},
  year={2023}
}