Reproduction of paper results about moc-detector HOT 5 CLOSED

It seems that your reproduction results are lower than ours for K7 RGB. I guess the FLOW results may have the same problem.

Our experimental settings are Python 3.5.2, PyTorch 0.4.1 with cuda9.0 for K7 RGB+FLOW. For pytorch1.6 we only tested the inference results without reproducing it.

Would you mind using pytorch0.4.1 to reproduce it again?

BTW, the performance gap is about 0.5-2.0 for N10 w/o flip for our released model (depends on the evaluation metric).

from moc-detector.

Thanks for fast response. Ok, I'll try with PyTorch 0.4.1

from moc-detector.

Hi!

I've tried with recommended parameters to train rgb model. Results are the same:
(scores are located as follows: [email protected] | [email protected] | @0.5 | @0.75 | 0.5:0.95):
(paper) ucf_dla34_K7_rgb_coco.pth: 73.14 | 78.81 | 51.02 | 27.05 | 26.51

ucf_dla34_K7_rgb_coco_N100_flip 71.56 | 78.40 | 50.24 | 26.18 | 26.16

ucf_dla34_K7_rgb_coco_N10 70.24 | 77.78 | 47.64 | 25.22 | 24.61

Again, I'm more curious about why inference with N100 + flip significantly better than N10 w/o flip which is opposite to your results. And variance of scores is also a bit high. I checked this on different machines. So I don't know where to look.
Hope you give me some advice.
Thank you again

from moc-detector.

For our RGB model, the experimental results are:
ucf_dla34_K7_rgb_coco_N100_flip: 73.14 | 78.81 | 51.02 | 27.05 | 26.51
ucf_dla34_K7_rgb_coco_N10: 72.05 | 78.23 | 50.77 | 26.10 | 26.16
(The performance gap is 0.5-1.1)

It seems that your reproduction results are a little lower than ours.(For example, the [email protected] for RGB K=7 should be at least 72.xx. And I got the similar results on my machine.)

As for N10 w/o flip, I wonder this may be just a coincidence for my own model.
I have not mentioned it in our paper because it was just an accidental discovery when I organized the code after the paper is accepted.

Here I give you more experimental results:
K=5 RGB
ucf_dla34_K5_rgb_coco_N100_flip: 71.63 | 77.74 | 49.55 | 27.04 | 26.09
ucf_dla34_K5_rgb_coco_N10: 70.10 | 76.08 | 49.14 | 26.28 | 25.70

ImageNet pretrain
ucf_dla34_K7_rgb_flow_ImageNet_N100_flip: 76.92 | 81.26 | 54.43 | 29.49 | 28.42
ucf_dla34_K7_rgb_flow_ImageNet_N10: 76.03 | 80.75 | 53.47 | 27.16 | 27.52

I this this performance gap is acceptable because it significantly boosts the inference speed.

Thanks for your correction and I will revise it.

from moc-detector.

Thank you so much. It helps me.

from moc-detector.