output masks consistently showing the same structure about groupvit HOT 5 OPEN

Thank you for this interesting work. I have trained this model from scratch using medical images. When evaluating the model, all the output masks (# of masks used = 8) consistently show the same structure with different intensity. Have you seen this issue using natural images? Any idea what could cause this? Thank you.

from groupvit.

Hi @aryaabdi,
Did you manage to solve the problem you mentioned? I'm getting a similar behavior.

from groupvit.

I get the same issue. @xvjiarui Can you help please?

from groupvit.

Hi all,

Sorry for the late reply. If you are training with specific domain images, I would suggest you start with pre-training on large scale natural images first. And the contrastive loss needs large batch size and large dataset to work.

from groupvit.

I tried training from scratch and also from the pre-trained (on natural images) model. The latter performed better. However, I realized the contrastive loss is not going to be effective if the number of entities within a batch is limited. I believe @xvjiarui can use a very large batch size because the training dataset contains many different entities. For example, gcc3m contains ~16k different entities. This was not the case in my training dataset and I think that is why I was not getting the desired behavior. Hope this helps.

from groupvit.