oil3 / onnxruntime-mac Goto Github PK

Thank you for your effort on the investigation of the onnx runtime os intel mac,
I hv' tried to install on mac but in vain,
here is some outputs,
how it can, is there any particular prerequisites

I'm not yet sure what I did, I messed around with the onnxruntime source to make sure that CPU-only provider flag was changed, casually into require ANE., and eventually got it to build.

Running some face restoration onnx model, I was checking "powermetrics", disapointed to see GPU at 0, with my M2 in low power mode, and then I plugged it to power, and suddenly ANE power usage spiked, maintaining it self around 10%.
I checked with the M1 max, and same, ANE in use topping 20, 25% sometimes.
I had never seen ANE being used before, ceteris paribus besides two things:
either my swapping of every "bicubic", non-core-ml supported operation, into "bilinear", and the runtime I'v been trying to figure out.

In this matter, I came to various conclusions, or hypothesises, one of which I chose the words carefully:
. We should consider that we have only 1 device (eg: M1 pro) that has 3 computing units (CPU, GPU, ANE).
-CoreML is one way to communicate with the device, there are alternatives, like Metal, OpenCL, Vulkan, WebGL, ARM providers)
-In this particular context of ours, onnxruntime, we are at the same level as 3 years ago.
I think there is some sort of confusion since the moment it was decided to limit to CPU. I am not saying it's a feature and not a bug, because it was clearly stated that this decision was for at least offering something that works, because it does work using only the CPU as computing unit, a very well documented CoreML processing unit configuration option.
For converting models using onnx, all computing units are in the code.

So, am I the only one?, Could be an update in something else.
This Neural Engine in use, was it something that happened before that I just realized?
When running "sudo powermetrics" in terminal, is there a uncontestable relation between model inference with correML execution provider and ANE usage?
In Activity Monitor, do you also see for ANE nothing signifcant, besides mucho bytes written?
I used sudo powermetrics --hide-cpu-duty-cycle -i 2000

Also, I compiled CoreML for intel x86_64 macs and runs flawlessly on my mac mini 2018, giving a 10 fold increase in frames per seconds (still slow though)