PyTorch implementation of "Emerging Properties in Self-Supervised Vision Transformers"

Concept ot the DINO [1].

	Student ($x_1$)	Teacher ($x_2$)
Before (mismatch)
After (matched)

Epoch	Student ($x_1$)	Teacher ($x_2$)
0
1
30
150
300

• PyTorch 2.0.1

[1] Mathilde Caron, et al. "Emerging Properties in Self-Supervised Vision Transformers." Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.