Comments (3)
how bad ? can you plot the loss and eval curves ?
from dinov2.
The loss curves seem relatively normal, continuing to decrease throughout training, but here are plots of the results of 16 evaluation tasks. As you can see, very often the models at 50% do better than the models at 100%. These are ViT-S models. For some of the tasks (eg, the bottom middle two), the curves are extremely irregular, and so these might not be great tasks, but the aforementioned behavior seems to be present in many of the others.
from dinov2.
I had the same experience, not too surprised. we have plots showing similar behaviour here: https://arxiv.org/abs/2401.04720
from dinov2.
Related Issues (20)
- Can I use dino model to match images with pixels? Just like with the clip model you can match pixels with text.
- Can overfitting lead to high-norm patches? HOT 10
- "inverted" first principal component of vitb14_reg4 HOT 1
- Issuess of unexpected keyword argument 'antialias' when using the DINOv2 backbone for finetuing HOT 1
- About the inability to successfully train Dinov 2-Distill
- The loss value about training is significant HOT 3
- Requirements are uncompatible HOT 8
- Choice of Segmentation Head HOT 5
- Training mask2former head with ViT adapter for semantic segmentation HOT 1
- Pretrain Dinov2 with custom dataset using Image net 1k weights HOT 5
- Run vanilla DinoV2 training with unlabelled dataset to fit specific field data HOT 1
- why do Self-supervised image retrieval? HOT 1
- Can you release the weight of dino head and ibot head๏ผ HOT 2
- Resume training from intermediate checkpoint? HOT 1
- Why is total_iters unrelated to batch size and number of data? HOT 2
- ResNet50 architecture HOT 1
- Train depth / segmentation head HOT 1
- Implementation using ytorch lightning
- Drop path implementation
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dinov2.