Is that right to evaluate performance on PASCAL VOC via COCO metric? about tide HOT 5 CLOSED

Yeah, I agree that it would be best if we were able to use the VOC version of mAP for PASCAL VOC, but I'm not well versed in the differences so I wouldn't be able to implement that.

Out of curiosity, what's the difference in mAP that you observe? If it's not a huge difference, then I think it's fine. And TIDE is meant as a way to find places you can improve your model, not necessarily to replace the official evaluation numbers. So as long as the numbers correlate, there shouldn't be any issue.

from tide.

Well, I got a ~10% higher mAP with PASCAL VOC metric than COCO metric on VOC2007 test. Some people say that PASCAL VOC greedily finds best match (using IoU criteria) for current pred box and if it is already matched it marks the current pred box as false. While in COCO the search continues if the current best match already matched.

Anyways, I agree with you that the role of TIDE is to find how one can improve the OD model, regardless of ways of evaluating mAP. After all, COCO metric is a reasonable metric.

from tide.

Well, I got a ~10% higher mAP with PASCAL VOC metric than COCO metric on VOC2007 test. Some people say that PASCAL VOC greedily finds best match (using IoU criteria) for current pred box and if it is already matched it marks the current pred box as false. While in COCO the search continues if the current best match already matched.

Anyways, I agree with you that the role of TIDE is to find how one can improve the OD model, regardless of ways of evaluating mAP. After all, COCO metric is a reasonable metric.

Hi，@machengcheng2016， could you share the code to convert pascal detection results to coco json style? My conversion get a much lower mAP in tide than mmdetection.

from tide.

Details of my conversion are in this issue

from tide.

I've solved it.

from tide.