multipole objects about personalize-sam HOT 4 OPEN

Hi, thanks for your interest in our work. Yes, the current version of PerSAM can handle one-object image or multiple objects with different appearances (like video object segmentation). For the image you provided, PerSAM only segments the most dominant one. Also, it can be simply modified to find two sneakers in four steps:

1. Use image encoder to obtain the image feature only once, and calculate the feature similarity map.
2. Find the location prior and segment the first object by the decoder, obtaining mask M_1.
3. Assign zeros to the pixels within M_1 on the feature similarity map, and then find the second location prior
4. Segment the second object by the decoder

from personalize-sam.

@ZrrSkywalker Hi. Is there any way to modify PerSAM to automatically determine how many items in the image are similar to the prompt and mask them all？

from personalize-sam.

Hi, thanks for the response
that can help but the issue is that even if I have the sneaker as a reference and segment cat image

its segments the cat ( and the sim matrix looks high in the cat area )
so for my case (a scene where there are multiple objects but no guarantee of the same object )
do you think persam can help?

Thanks again for the help!

from personalize-sam.

Thanks for your in-depth analysis. This is because, compared to the background, the sneaker feature is more similar to the cat feature.

If we give the one-shot image containing two objects, sneaker and cat, the reference sneaker feature would correctly match other sneakers with the highest scores in new images, rather than the cat. As shown in Figure 9 for video object segmentation.

from personalize-sam.