Support passing GPU buffer directly to GLDelegate about tf-lite-unity-sample HOT 14 CLOSED

This feature is supported in the TFLite C++ API, but not yet in the C API that is used by Unity.
When I have time, I will try to fix the TFLite C API.

from tf-lite-unity-sample.

@asus4 Wow I jumped the gun on that one, took a quick glance at the description and thought it depended on missing functionality in TFLite itself. Guess I got too excited that my project might be directly relevant 😩. I suppose there's still a need in TFLite for a bindTensorToGLTexture function to avoid the texture to buffer copy, though it's not expensive in the grand scheme of things... I'm still absorbing the backend structure of TFLite GPU delegates, there's undocumented functionality I keep finding like the CLDelegate's BindBufferToTensor method. I'll post here again if I push any GLDelegate changes to the main TF repo

from tf-lite-unity-sample.

@asus4 that would be really great! This issue is the biggest source of lag in my game.

from tf-lite-unity-sample.

Working on this issue today: Modified both TensorFlow and Unity codes. and it worked with the Metal delegate now.

• TensorFlow branch

  • Exposed the GPU buffer-binding APIs to the C framework.

• Unity branch

  • Added a test scene to evaluate the effect of this modification.

Next step: Supports OpenGLES delegate as well.

from tf-lite-unity-sample.

Hi there, I wish I had seen this project a couple weeks ago! I've been working on a fork of tensorflow allowing for easy integration with rendering pipelines https://github.com/WHSnyder/tensorflow/tree/general_opengl, so pretty much this exact issue. So far I've just gotten the delegates to build for MacOS and Ubuntu and decoupled OpenGL from OpenGLES. The GL delegate works on Ubuntu now (given you have created a context beforehand) and the Metal delegate no longer crashes when used on MacOS (CoreGraphics wasn't linked in the /lite/delegates/gpu BUILD file). I also added some code to expose the Metal/GL delegates in the Python API (the pyexposure files).

However, I have not been able to get any of these delegates to work with this style transfer example https://www.tensorflow.org/lite/models/style_transfer/overview or this one https://github.com/tensorflow/examples/tree/master/lite/examples/style_transfer
I get the exact same error as discussed here tensorflow/tensorflow#32101 I've been using the Python API to create & call delegates, since that hasn't worked I've dedicated today to testing with the C++ API. Sorry to bombard you with links, once I get this worked out I'll see if I can merge it cleanly into this repo.

The GPU buffer binding you mention is only for Metal, but once I get everything working with the style transfer example I'll write a couple functions to expose the buffers for the GL delegate.

EDIT: Didn't notice the TfLiteGpuDelegateBindBufferToTensor function. Nobody seems to mention it, I'll see if I can get it working and implement a BindTextrueToTensor function for render target usage.

from tf-lite-unity-sample.

Hi @WHSnyder, Thank you for your interest. This repository does not contain any TensorFlow's code. Please make PR to TensorFlow repository then I'll build the shared lib 😃

from tf-lite-unity-sample.

For those working with ARFoundation you can indeed skip a fair few of the steps as outlined above.

from tf-lite-unity-sample.

@WHSnyder do your have any updates on that issue? atm I'm facing the problem that the readPixels functions blocks my main thread and reduces the performance to measly 15 fps... -> #96

from tf-lite-unity-sample.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

from tf-lite-unity-sample.

@asus4 any new information regaring that issue?

from tf-lite-unity-sample.

really nice!

from tf-lite-unity-sample.

Android's GPU delegate worked as well with the following branch
https://github.com/asus4/tensorflow/tree/bindbuffer-gpudelegate

It was super tricky to figure out the correct setting in Unity

We can use this feature with my branch's TensorFlow. and preparing a PR for the main repository.
tensorflow/tensorflow#49313

from tf-lite-unity-sample.

I suppose that multi-threaded rendering would be possible if the GPU-fence could be shared with TensorFlow. but there is no API to share it at the moment.
https://docs.unity3d.com/ScriptReference/Rendering.GraphicsFence.html

Correct me if I'm wrong, I'm a complete beginner at this.

from tf-lite-unity-sample.

Although the PR on TensorFlow has not merged yet, I merged these changes to proceed with this issue.

The modified TensorFlow branch ↓
https://github.com/asus4/tensorflow/tree/2.5.0-with-bindbuffer

from tf-lite-unity-sample.