selfsup's Introduction

SelfSup

Collections of self-supervised methods (MoCo series, SimCLR, SiMo, BYOL, SimSiam, SwAV, PointContrast, etc.).

Get Started

Install cvpods following the instructions.

Install cvpods from https://github.com/Megvii-BaseDetection/cvpods.git .

Prepare Datasets

cd cvpods
ln -s /path/to/your/ImageNet datasets/imagenet

Train your own models

cd /path/to/your/SelfSup/examples/simclr/simclr.res50.scratch.imagenet.224size.256bs.200e
# pre-train
pods_train --num-gpus 8
# convert to weights
python convert.py simclr.res50.scratch.imagenet.224size.256bs.200e/log/model_final.pth weights.pkl
# downstream evaluation
cd /path/to/your/simclr.res50.scratch.imagenet.224size.256bs.200e.lin_cls
pods_train --num-gpus 8 MODEL.WEIGHTS /path/to/your/weights.pkl

Model Zoo

Supervised Classification

ImageNet

Methods	Training Schedule	Top 1 Acc
Res50	100e	76.4

CIFAR 10

Methods	Training Schedule	Top 1 Acc
Res50	200e	95.4

STL 10

Methods	Training Schedule	Top 1 Acc
Res50	150e	86.1

Self-Supervised Learning - Classification

All results in the below table are trained using resnet-50 and reported on the ILSVRC2012 dataset.

Methods	Training Schedule	Batch Size	Our Acc@1	Official Acc@1
MoCo	200e	256	60.5	60.5
MoCov2	200e	256	67.6	67.5
SimCLR	200e	256	63.2	61.9
SimCLR*	200e	256	67.3	Ours
SiMo	200e	256	68.1	Ours
SimSiam	100e	256	67.6	67.7
SwAV	200e	256	73.0	72.7
BYOL	100e	2048	69.8	66.5 (bs4096 from SimSiam paper)
BarlowTwins	300e	1024	Comming Soon	71.7

Self-Supervised Learning - Detection (2D)

All the results reported below are trained on ILSVRC2012 and evaluated on MS COCO using Faster-RCNN-FPN and resnet-50.

Methods	Training Schedule	Batch Size	Box AP
SCRL	200	4096	39.9 ( official: 40.5 with bs 8192)
DetCon	200	256	Comming Soon.

Self-Supervised Learning - 3D Scene Understanding

Methods	Training Schedule	Downstream task
PointContrast	-	Comming Soon.

Citation

SelfSup is a part of cvpods, so if you find this repo useful in your research, or if you want to refer the implementations in this repo, please consider cite:

@article{zhu2020eqco,
  title={EqCo: Equivalent Rules for Self-supervised Contrastive Learning},
  author={Zhu, Benjin and Huang, Junqiang and Li, Zeming and Zhang, Xiangyu and Sun, Jian},
  journal={arXiv preprint arXiv:2010.01929},
  year={2020}
}

@misc{zhu2020cvpods,
  title={cvpods: All-in-one Toolbox for Computer Vision Research},
  author={Zhu*, Benjin and Wang*, Feng and Wang, Jianfeng and Yang, Siwei and Chen, Jianhu and Li, Zeming},
  year={2020}
}

selfsup's People

Contributors

Stargazers

Watchers

selfsup's Issues

DetCon Implementation

Hi, thanks for open sourcing the great work!
I found DetCon is also on the roadmap. Is there any plan to release the implementation?

training on multi-gpu

Hi,
I am really new to self supervised learning. I would like to know how do you adjust no.of epochs when you train on multi-gpu.

I am intending to train on ImageNet. Unfortunately, I cannot fit mini-batch of 256 in one gpu. As for as I understand, when I train on 4-gpu, even if I train for 100 epochs, the no.of epochs trained will be effectively 100/4=25 as number of gradient updates will also be divided by 4. Please correct me if I am wrong.

Please let me know how have you accounted for that in your implementation.

Thanks

Recommend Projects

v2ai / selfsup Goto Github PK