[windows] Consider using the flutter TextureRegistrar API about dart_vlc HOT 13 CLOSED

This plugin is really ahead of time, there isn't much documentation available and Linux doesn't support texture either yet.

The idea is to have a cross platform, native & performant media playback, broadcast & recording solution for all platforms (although I'm having few problems in the process right now).
I have already started to work on ffi based binding here which might eventually replace current platform channel interface. This project isn't just a pure Dart and C++ mapping, but a wrapper around libVLC, which is here and shared by both platforms as of now. I'm connecting this existing wrapper with FFI (WIP) instead of platform channel interface (here in case you're interested).
I would've picked ffi for the first place but it doesn't support async callbacks from another isolate, although now that's been circumvented, this project can also do media playback from Dart as well (not just Flutter). Main issue that I'm having is to compile libVLC along with my shared library for various architectures of Android. After this FFI migration, I'd love to see this running on mac too.

Current implementation just sends RAW frames in RV32 format and fortunately there exists a RawImage widget/class in Flutter for me to use. So, current video playback is kinda wacky, but is the best that I could've done.

First of all, I‘d like to thank you for your efforts.

I received fine amount of donation from @DomingoMG to work on this. It isn't my sole interest, but a motivation by supporters.

Thanks.
Glad you are liking the project, looking forward to make more additions.

from dart_vlc.

@alexmercerind
Does LibVLC support rendering to a DirectX texture? Once flutter/engine#26840 gets merged, you should be able to reduce the copy overhead even further then.
But I don't think that the high CPU usage is primarily caused by copying the buffers. Did you profile it?

from dart_vlc.

@jnschulze
Hell yeah 🥳.
You are right, first of all I really really appreciate the contributions that you have made to the Flutter engine, its all done by you. The texture support & the PR above.

The CPU usage is actually low.

I just noticed that running application flutter run --release also takes some performance expenses (maybe observatory) compared to running from explorer. I believe this is the best it can get.

3 - 10% unbelievable.

from dart_vlc.

One more thing.
There was a bug in the Texture registrar API that copied the buffer regardless of calling MarkTextureFrameAvailable.

flutter/engine#27508

This should be fixed on the Flutter beta channel and will likely reduce CPU usage.

from dart_vlc.

Hi, I have the same problem as unorovi had which is very high CPU usage. Is there any update on this? Thanks🙏

from dart_vlc.

@qorn flutter/flutter#80057

from dart_vlc.

@qorn
Please checkout this issue comment flutter/flutter#78517 (comment).
It was an issue in Flutter's Windows Runner itself.

from dart_vlc.

@alexmercerind there's no windows/runner/run_loop.cpp file in my flutter project. I'm using Flutter 2.4.0-1.0.pre.65

from dart_vlc.

@qorn

See my CPU usage is always about 15-20% (in release mode). And my processor is also not powerful (old Zen I, entry level Ryzen 3 laptop variant 2200U) processor.

from dart_vlc.

Closing this issue as this project is no longer a plugin but FFI binding to existing C++ wrapper to libVLC.

I've refactored code to use FFI instead of Flutter's platform channel & now this can be used from Dart aswell (independent of Flutter).
As a result, now whole API is completely synchronous & more performant aswell.

Thanks

from dart_vlc.

So, the current video playback journey has been like:

• Sending video frame buffer through method channel (when this issue was opened).
• Sending video frame buffers through NativePorts of DartVM (current).
• Now sending video frame buffers through flutter::TextureRegistrar API. (windows-texture-migration branch)

So, now I've tested to flutter::TextureRegistrar for Windows.
I didn't require swizzling between RGBA & BGRA since I could just configure the format inside libVLC. (Changed RV32 to RGBA).

There are performance improvements (very little) but not much compared to NativePorts. At the end, CPU usage is still high.

@qorn You can try out Texture based video player (only for Windows) by cloning windows-texture-migration branch. Let's see if it makes a difference for you.

Use like this:

import 'package:dart_vlc/dart_vlc.dart' hide Video;
import 'package:dart_vlc/src/experimental/video.dart';

/* Inside build method. */
Video(
  player: player,
  width: 480,
  height: 360,
),

Having platform views would be best for everyone. Using flutter texture registrar is making the dart_vlc wrapper less cross platform. Currently, one can just compile the same shared library for mac, windows & linux and it would work just fine. But, now along with FFI, I'm using texture registrar for Windows. Now shipped with 0.1.2, noticeable performance differences.

from dart_vlc.

Let me know, if you want to ship this for Windows. (Will keep it inside original Video widget)

from dart_vlc.

Released in 0.1.2. Same Video widget now uses Texture on Windows. No breaking changes.

from dart_vlc.