This repo contains the official detection and segmentation implementation of paper "DaViT: Dual Attention Vision Transformer", by Mingyu Ding, Bin Xiao, Noel Codella, Ping Luo, Jingdong Wang, and Lu Yuan.

Python3, PyTorch>=1.8.0, torchvision>=0.7.0 are required for the current codebase.

# An example on CUDA 10.2
pip install torch===1.9.0+cu102 torchvision===0.10.0+cu102 torchaudio===0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install thop pyyaml fvcore pillow==8.3.2

• cd mmdet & install mmcv/mmdet

  # An example on CUDA 10.2 and pytorch 1.9
  pip install mmcv-full==1.3.0 -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.9.0/index.html
  pip install -r requirements/build.txt
  pip install -v -e .  # or "python setup.py develop"

• mkdir data & Prepare the dataset in data/coco/ (Format: ROOT/mmdet/data/coco/annotations, train2017, val2017)

• Finetune on COCO

  bash tools/dist_train.sh configs/davit_retinanet_1x_coco.py 8 \
  --cfg-options model.pretrained=PRETRAINED_MODEL_PATH

• cd mmseg & install mmcv/mmseg

  # An example on CUDA 10.2 and pytorch 1.9
  pip install mmcv-full==1.3.0 -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.9.0/index.html
  pip install -e .

• mkdir data & Prepare the dataset in data/ade/ (Format: ROOT/mmseg/data/ADEChallengeData2016)

• Finetune on ADE

  bash tools/dist_train.sh configs/upernet_davit_512x512_160k_ade20k.py 8 \
  --options model.pretrained=PRETRAINED_MODEL_PATH

• Multi-scale Testing

  bash tools/dist_test.sh configs/upernet_davit_512x512_160k_ade20k.py \ 
  TRAINED_MODEL_PATH 8 --aug-test --eval mIoU

If you find this repo useful to your project, please consider citing it with following bib:

@article{ding2022davit,
    title={DaViT: Dual Attention Vision Transformer}, 
    author={Ding, Mingyu and Xiao, Bin and Codella, Noel and Luo, Ping and Wang, Jingdong and Yuan, Lu},
    journal={arXiv preprint arXiv:2204.03645},
    year={2022},
}

Our codebase is built based on timm, MMDetection, MMSegmentation. We thank the authors for the nicely organized code!