CoreML EP inference result is improperly scaled about onnxruntime HOT 2 OPEN

COREML_FLAG_USE_CPU_ONLY results in CoreML executing the same nodes using its reference CPU implementation. We set this as the MLModelConfiguration.computeUnits. The rest of the ORT CoreML EP code runs exactly the same. That would strongly suggest an issue with the internal CoreML handling of a large input when running on GPU/NPU.

COREML_FLAG_ONLY_ALLOW_STATIC_INPUT_SHAPES is applied on a per-node basis. Parts of the model may have fixed shapes leading to CoreML executing them. If you set the session logging severity to VERBOSE it will print out details of which nodes are/aren't assigned to CoreML. That would at least narrow down which CoreML operator could be going wrong.

from onnxruntime.

This appears to be a CoreML NeuralNetwork specific problem. There are only a few Div and Sub nodes assigned to CoreML as the rest have dynamic input shapes. Most of those produce the expected output.

There are 2 Div nodes (Div_185 and Div_143) that end up doing 2 / (2048 - 1) (one for the height and one for the width). For some reason the NeuralNetwork Div is somewhat inaccurate for this floating point operation.

Python as a reference (double precision):
2.0 / 2047.0 = 0.0009770395701025891

That difference must become significant across all the other downstream operations in the model, leading to the output discrepancies. I would guess it comes down to floating point inaccuracies from 2 divided by a large number as to why smaller numbers for the height or width don't trigger the issue.

from onnxruntime.