I converted a model to fp32 and fp16 in mlprogram format, fp16 output is wrong but fp32 is right. about coremltools HOT 5 CLOSED

I checked the torch model, find no layer's input/output contains number beyond 65504.
Two possible reasons:

1. The model underflow with fp16.
2. Some intermediate results overflow with fp16. Even each torch layer's input/output is within fp16 range, the converted model could still overflow on some intermediate tensors (because the op in torch and op in CoreML is not strictly 1:1 mapping).

I would recommend to use "binary search" to find which op is the culprit: Get the first half of the torch model, convert it and see if results match; And then keep halfing the part which introduces the numerical issue.

from coremltools.

Now I located the problem caused by https://github.com/TencentARC/GFPGAN/blob/master/gfpgan/archs/stylegan2_clean_arch.py , but binary search is time-consuming, any suggestion for this arch?

from coremltools.

#1671. same problem

from coremltools.

Thank you for locating the culprit part in the model!
Yeah it's the same issue you mentioned, due to fp16 intermediate results overflow.

If you want to try it on GPU, one method to try is changing the compute precision by compute_precision=ct.precision.FLOAT32 in ct.convert. Here is an example: https://apple.github.io/coremltools/docs-guides/source/convert-to-ml-program.html?highlight=compute_precision#set-the-ml-program-precision.
Then profiling the converted model by Xcode to see if it's GPU resident.

from coremltools.

@junpeiz fp32 model on GPU works fine, thanks, issue close.

from coremltools.