performance drop about insightface-rest HOT 8 CLOSED

Hi! That's interesting, could you provide your testing code and more info about models you have tested?

from insightface-rest.

So i just used the test pairs .txt file provided in their website and then run your onnx arcface model that was produced by running your docker using onnxruntime. and then check the results on various thresholds.

I noticed that you converted the model from mxnet. would there be a difference in the result if we convert the pytorch model into onnx?

from insightface-rest.

Have you tested same code with backend set to mxnet?
The accuracy drop you mentioned is way too large, it might be caused by improper validation, as well as some massive bug in my code, but such accuracy drop should be easily noticable in most use cases.

I haven't tested torch models yet.

from insightface-rest.

Okay let me double check and re-run the code then.

from insightface-rest.

Any updates?

from insightface-rest.

Hi, sorry for the late update, I still find the same result for mobilenet retinaface + arcface resnet100. However. yesterday I just tried the onnx models that was uploaded in insightface (antelope) containing scrfd 10g and arcface resnet100 glint, and I got 0.9985 accuracy on threshold==0.26 on LFW test set. So, I think the newer models are better.

from insightface-rest.

Are you sure that correct preprocessing was used for older models? If I understand you right, you used onnx models in your own code?

from insightface-rest.

It is certainly a possibility that I screwed up somewhere. I tried to translate the python code to c++ version and ran a test on it. One suspicion that I have is that I installed the wrong onnx/mxnet version because I did not use your docker. Anyway, I think the issue is solved now. So I am going to close this issue.

from insightface-rest.