Lower COCO AP after pre-training SoCo_FPN_100ep about soco HOT 5 OPEN

I have now tried fine-tuning my pre-trained model using both your hologerry/detectron2 codebase and the standard facebookresearch/detectron2 and in both cases I get around 39.8 bb AP. However, when I use your pre-trained weights for SoCo_FPN_100epand fine-tune on the standard detectron2 code I get 42.1 AP (very similar to the reported number in your paper of 41.9).

This suggests there is something different in the pre-training hyperparameters since I'm unable to replicate the paper's results.
Thanks for any help!

from soco.

Hi, I found that the result of mocov2 in Table 1 of the SOCO paper is particularly high (40.4 bb AP). But the experiment I have run shows that the result of fine-tuning a mask RCNN-R50 FPN with mocov2 backbone should be similar to the supervised pre-training (38.9 bb AP). Do you find this problem? Thanks!!!

from soco.

I've found the source of the difference. De to an error in the apex automatic mixed precision library I ended up using optimization setting O2 in my original run. When I fixed the bug and reran with optimization O1 it achieves 41.8 bb AP.

@yanjk3 Sorry I've not experimented on mocov2 in this setting so can't help. But good luck!

from soco.

Does the author not update this library ？

from soco.

Hello @linusericsson and @xiaoxiong007 The optimization level does affect the performance, for all results of our models are trained with O1 level.
There might be quite small differences among different runs as provided in the supplemental materials.

from soco.