Question about retrieval of commercial-consumer clothing items about deepfashion2 HOT 6 OPEN

1. The process of choosing the detected consumer clothing item is to choose a detected clothing item among all the detected clothing items in an image to represent one ground truth consumer clothing item for retrieval.
2. For each detected consumer clothing item, its feature will be fed into matching network to retrieve commerical clothing gallery item. But only the retrieved result of the chosen detected consumer clothing item in step 1 will be evaluated.

from deepfashion2.

@geyuying Thanks for the response!

I got a couple more questions:

1. Specifically what is the evaluation of step 1 based on? Is it based on the quote below? And so if the result of the proposed method in the below quote is positive, does that mean that the evaluation of step 1 will be positive?

If IoU between retrieved item from shop images and one of the ground truth corresponding gallery item is over the thresh(we set thresh as 0.5), the retrieved result is positive.

2. I still don't understand fully how the gallery item is retrieved? The matching network only outputs a similarity score, it doesn't retrieve anything. So do you compare the similarity score of ALL gallery items with the chosen detected consumer clothing item, and choose the gallery item with the highest similarity score?
3. Also where are the gallery items in the training dataset? Do you have to construct them? And do the gallery set contain ALL the commerical clothing items?
4. Since you used the top-k accuracy metric for clothing retrieval, does that mean that you have to choose k gallery items with the highest similarity score for a given chosen detected consumer clothing item?

from deepfashion2.

@Dayan-Zhanchi

I am also re-implementing the Match-Net. From my perspective:

1. Specifically what is the evaluation of step 1 based on? Is it based on the quote below? And so if the result of the proposed method in the below quote is positive, does that mean that the evaluation of step 1 will be positive?

I think you can refer to the retrieval test script.

2. I still don't understand fully how the gallery item is retrieved? The matching network only outputs a similarity score, it doesn't retrieve anything. So do you compare the similarity score of ALL gallery items with the chosen detected consumer clothing item, and choose the gallery item with the highest similarity score?

I think yes. You need to compare the similarity score of ALL (detected) gallery items with the chosen detected consumer clothing item.

3. Also where are the gallery items in the training dataset? Do you have to construct them? And do the gallery set contain ALL the commerical clothing items?

I think gallery items are the images with source = shop. You can construct the image pairs (and instance pairs) between user and shop images (instances). I think the gallery set contains all the commercial clothing items.

4. Since you used the top-k accuracy metric for clothing retrieval, does that mean that you have to choose k gallery items with the highest similarity score for a given chosen detected consumer clothing item?

Yes, I think so.

Still, I hope that the author can help to confirm the correctness of my answers.

from deepfashion2.

@xwjabc

Cool, thanks for the answers!

I got a question if you can answer it:

During evaluation time, when performing the commercial clothing retrieval, the author states in issue #14 the following:

If IoU between retrieved item from shop images and one of the ground truth corresponding gallery item is over the thresh(we set thresh as 0.5), the retrieved result is positive.(If none detected item is assigned the given query item label, this query item is counted as missed. )

My question is do you know how to get this ground truth corresponding gallery item? If I understand correctly, the only thing the model is doing at the clothing retrieval task is:

• Retrieve a query item (detected consumer clothing item)
• Use the retrieved query item to get a gallery item (commercial clothing item with the highest similarity score).
• And then to evaluate the result, you want to compare this retrieved gallery item with the ground truth corresponding gallery item.

But the ground truth corresponding gallery item is not given? You have to somehow find the ground truth corresponding gallery item to compare it with the retrieved gallery item?

from deepfashion2.

Well, in the retrieval test script, the evaluation pipeline is like this:

1. Calculate the IoU between all pred_query(s) and gt_query(s). For each pred_query, the gt_query with the highest IoU is assigned as corresponding gt_query.

2. For each gt_query, find all the pred_query(s) which has the same corresponding gt_query. Then choose the one from the found pred_query(s) which has the same class and the highest score. Thus, we can build a (gt_query, pred_query) pair for all gt_query(s).

3. For each (gt_query, pred_query) pair above, we can know the pred_gallery(s) from the results given the pred_query. Besides, we can also obtain the style and pair_id given the gt_query. Using the style and pair_id, we can find all gt_gallery(s) that have the same style and pair_id.

4. For top-k evaluation, we choose top-k pred_gallery(s). Next, if any one of the top-k pred_gallery(s) has IoU >= 0.5 with any one of the gt_gallery(s) in the previous step, we will consider that for current gt_query, we can find a gt_gallery that matches. It will contribute to a positive match. On the contrary, if all top-k pred_gallery(s) have IoU < 0.5 with all the gt_gallery(s) in the previous step, we will say that for current gt_query, we CANNOT find a gt_gallery that matches. It will contribute to a negative match.

5. top-k accuracy = #positive matches / (#positive matches + #negative matches) = #positive matches / #gt_query(s)

from deepfashion2.

@xwjabc

Okay, I'll look more closely into that, but thank you very much for the elaborated answer! Very much appreciated!

from deepfashion2.