Build with XNNPACK about tf-lite-unity-sample HOT 11 CLOSED

Could this boost the performance?
I've tried mediapipe hands on gpu on android (s8) and they get about 20-25 fps but current hands implementation are very slow on pc. (i7 16 gb ram gti 960 ti)

from tf-lite-unity-sample.

Hi @TheBricktop, This is because the GPU delegate (a GPU acceleration feature of the TensorFlow Lite) is not supported on Window at the moment. I would like to support it eventually.

from tf-lite-unity-sample.

What ones should do to implement it? Recompile tf lite library?

from tf-lite-unity-sample.

It looks like to be not officially supported.
tensorflow/tensorflow#40325

from tf-lite-unity-sample.

So on android the performance would be higher?

from tf-lite-unity-sample.

Yes, if you are interested in using MediaPipe without the GPU delegate, please refer XNNPACK(this issue) or the integer quantized model

from tf-lite-unity-sample.

Would exporting tflite models to onnx and running in barracuda improve the performance?

from tf-lite-unity-sample.

Yes, if all ops are supported in Barracuda, could improve it. Please refer to the supported ops in Barracuda

from tf-lite-unity-sample.

Now, XNNPACK options are enabled in v2.4 libraries

from tf-lite-unity-sample.

@asus4 I suspect XNNPACK is not correctly enabled based on the following observations:

1. Currently the CPU mode performance of tf-lite-unity-sample is closer to what I get from the official benchmark tool (https://www.tensorflow.org/lite/performance/measurement) with use_xnnpack=false.
2. From Tensorflow source code, I think TfLiteXNNPackDelegateCreate must be called in order for XNNPACK to be actually enabled. I don't see it being called anywhere in tf-lite-unity-sample.
3. When running the official benchmark tool with use_xnnpack=true, there will be a line in log that says Created TensorFlow Lite XNNPACK delegate for CPU. It's missing when tf-lite-unity-sample is run.

Would you like to take a look?

from tf-lite-unity-sample.

Thanks, @tonysung

I mistakenly thought it would be automatically enabled on the CPU mode. I will add XNNPack delegate.

https://github.com/tensorflow/tensorflow/blob/3f878cff5b698b82eea85db2b60d65a2e320850e/tensorflow/lite/delegates/xnnpack/xnnpack_delegate.h#L48

from tf-lite-unity-sample.