HuggingFace GroundingDINO inference execution time is slower than the original groundingDINO (~100ms) about transformers HOT 2 OPEN

Hi @lucasMichelis, thanks for opening this issue!

Regarding the processing times, this isn't that surprising. The image processors we currently have are unfortunately pretty slow, as they do everything through list comprehensions and in numpy. Their main purpose is to enable users to quickly go from image -> model in a single step. We're working on improving this and will hopefully be introducing a "fast" version of grounding dino's processor soon c.f. #28847 Until then, I'd recommend using a torchvision pipeline is inference time is important.

For the different in model inference time, it's something we can look into! As the differences aren't huge i.e. not indicative of a bug, it's not something which I can guarantee will be addressed immediately, but we'd like to have comparable numbers. If you or anyone in the community would like to do some investigation and open a PR to make things faster I'd be very happy to review!

cc @EduardoPach

from transformers.

@lucasMichelis do you have ninja installed as well? If I'm not mistaken the original implementation uses the custom kernels for the MultiScaleDeformableAttention, but the transformers implementation will only use if you have ninja installed

from transformers.