DINOV2 generate different outputs for the same inputs about transformers HOT 6 OPEN

Hi @y22ma, thanks for opening the issue!

Hm, it is strange, but I was not able to reproduce the bug. I'm getting consistent results

import torch
import requests
from PIL import Image
from transformers import AutoImageProcessor, AutoModel

# Initialize the DinoV2Embedding instance
model_name = "facebook/dinov2-large"
device = "cuda" if torch.cuda.is_available() else "cpu"
preprocess = AutoImageProcessor.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name).to(device).eval()

url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)

# Loop to get embeddings multiple times
for i in range(5):
    with torch.no_grad():

        inputs = preprocess(images=image, return_tensors="pt").to(device)
        outputs = model(**inputs)
        print(f"Output {i + 1}: {outputs.last_hidden_state[:, :3, :3]}")

Env:

- `transformers` version: 4.44.2
- Platform: Linux-6.1.85+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.24.6
- Safetensors version: 0.4.4
- Accelerate version: 0.33.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.4.0+cu121 (False)

How different results did you get between iterations? Can you provide an output from my snippet above?

from transformers.

Thanks for getting back to me. Here's my output with your image as input (000000039769.jpg from coco val 2017):

Output 1: tensor([[[-2.2247, -2.0199,  0.5695],
         [ 1.3696,  2.1394,  3.7765],
         [ 1.1757,  2.2480,  3.4782]]], device='cuda:0')
Output 2: tensor([[[ 0.0169, -0.0767, -0.4897],
         [-0.6269, -0.8694,  0.5277],
         [-0.5765, -0.1864,  0.3269]]], device='cuda:0')
Output 3: tensor([[[ 0.0169, -0.0767, -0.4897],
         [-0.6269, -0.8694,  0.5277],
         [-0.5765, -0.1864,  0.3269]]], device='cuda:0')
Output 4: tensor([[[ 0.1425,  0.6349, -1.0352],
         [-1.6960, -0.3588,  0.7011],
         [-0.8738, -0.4230,  0.9493]]], device='cuda:0')
Output 5: tensor([[[ 0.8900, -0.6973,  0.1371],
         [ 1.3878, -0.8789, -1.0418],
         [ 0.8685, -0.7731, -0.8942]]], device='cuda:0')

It's interesting that the first output matches with yours, and it just goes haywire from there? I'm using a NVIDIA GeForce RTX 3080. I did verify that the input has consistent values.

from transformers.

Can you please provide full output of transformers-cli env?

from transformers.

Here it is:
Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

• transformers version: 4.44.2
• Platform: Linux-5.15.0-117-generic-x86_64-with-glibc2.31
• Python version: 3.11.9
• Huggingface_hub version: 0.24.6
• Safetensors version: 0.4.5
• Accelerate version: not installed
• Accelerate config: not found
• PyTorch version (GPU?): 2.2.2+cu121 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using distributed or parallel set-up in script?: no
• Using GPU in script?: yes
• GPU type: NVIDIA GeForce RTX 3080

from transformers.

Thanks @y22ma, unfortunately I don't have an excess to 3080 RTX, and in my environment the issue is not appear. Do you have access to any other machine to reproduce the bug? Have you tried google colab?

from transformers.

I'm not able to reproduce this on a T4 machine on AWS unfortunately.

from transformers.