creativeikep / blazeposebarracuda Goto Github PK

This project is the best working body tracking project I have seen so far. I tried many different artificial intelligence models, from blazepose to tenserflow, mediapipeline, move net, pose net ETC. But this was the smoothest one, but unfortunately, the application cannot even be opened on mobile phones, it lags a lot. and I want to use it on mobile, but I have no idea how to improve the performance problems, can you help me?

Hi @creativeIKEP ,

Hope you are all well !

I was wondering if it would be possible to add a scene with the Unity Chan avatar as a demo ?

Thanks for any insights or inputs on that.

Cheers,
Luc Michalski

see title

Hi, is it possible to get a landmark screen position in C# script? Thanks

Thanks for the project - love keijiro's work and seeing more in its spirit!

I was hoping for confirmation on which in this set of models are running on the CPU vs the GPU and what the path would look like to achieve inference purely on the GPU if not already there. I can see a heavy load on the CPU and so am unable to hit 30fps in testing (even on PC) so am curious if it'll be achievable in the future (by myself or others).

Cheers

Hey, could you give any details for how the landmark data could be mapped to the joint of a rigged avatar? Thanks!

Is it possible?

Hey @creativeIKEP How can I use custom onnx file pose_detetion.onnx and pose_landmark_full.onnx converted by mediapipe model card pose_detetion.tflite and pose_landmark_full.tflite ? It's link is https://google.github.io/mediapipe/solutions/models.html

Thanks !  
Wish your reply!

Is it possible to track several people using your solution?

Hi there!

BlazePose runs just fine in the Unity Project from this repository on my computer. I'm trying to integrate BlazePoseBarracuda into a different Unity Project, but I'm experiencing odd behavior from the PoseVisualizer/PoseVisualizer3D scripts in that project.

I've exported all of the assets from this original Unity Project as a .unitypackage file. After that, I install the packages required for BlazePoseBarracuda using this technique in the README in the new Project, and there seems to be no problems with that process. Then, I import my BlazePoseBarracuda .unitypackage in the new Project. When I run a demo scene in the new project, e.g. the 3DSampleScene, no pose skeleton appears & the Debug.Log() calls in PoseVisuallizer3d.cs output Vector4s of the format (NaN, NaN, NaN, w), where w is the landmark position visibility score. w seems to consistently be a number, but the first three coordinates (the x, y, z of each pose landmark) are NEVER a number. They're only NaN. The 33rd index data is always consistently (0.0, 0.0, 0.0, 0.0) too.

See the below screenshot for an example (I was visible to the webcam for this capture, so, if BlazePose were working, it would have been able to detect a pose.)

Technical details:
I'm running MacOS 12.3/Unity 2020.3.33f1. I'm using Barracuda 2.1.0-preview (because it's required for another NN pose detection implementation. I've tried other ver of Barracuda with BlazePoseBarracuda in this Unity Project and had same result.)
My build target is Android. My computer has an Intel CPU and an AMD GPU.
Graphics APIs: OpenGLES2 & 3 are enabled, but Metal I think since I'm on a Mac?

Hey !!!

Just wanted to clarify the way the data is stored in the world landmark raw buffer output in the Pose Landmarker Script

Is it x, y, z for each frame and each point or in any other shape.

Thanks in Advance

Hello,

Thanks for your great work.
Do you consider to update your solution to latest MediaPipe ? https://github.com/google/mediapipe/tree/v0.8.11
Thanks,
Thomas

Thanks for releasing this project,
are you planning to add support for the Linux standalone build?

WindowsVideoMedia error unhandled Color Standard: 0 falling back to default this may result in rendering issues

Process.compute: Kernel at index (0) is invalid
UnityEngine.StackTraceUtility:ExtractStackTrace ()
Mediapipe.BlazePose.BlazePoseDetecter:ProcessImage (UnityEngine.Texture,Mediapipe.BlazePose.BlazePoseModel,single,single) (at Packages/BlazePoseBarracuda/Script/BlazePoseDetecter.cs:108)
PoseVisuallizer3D:LateUpdate () (at Assets/Script/PoseVisuallizer3D.cs:48)

IndexOutOfRangeException: Invalid kernelIndex (1) passed, must be non-negative less than 6.
Unity.Barracuda.ComputeFunc..ctor (Unity.Barracuda.ComputeShaderContext ctx, System.String kn) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/BarracudaReferenceCompute.cs:2653)
Unity.Barracuda.ComputeKernel.CalculateEntryScore (Unity.Barracuda.ComputeShaderContext ctx, Unity.Barracuda.ComputeKernelLibrary+Entry entry, System.Boolean verbose) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/BarracudaCompute.cs:915)
Unity.Barracuda.ComputeKernel.BestKernel (Unity.Barracuda.ComputeShaderContext ctx, System.Collections.Generic.List1[T] entrees, System.Boolean verbose) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/BarracudaCompute.cs:960) Unity.Barracuda.ComputeOps.BestKernel (System.Collections.Generic.List1[T] entrees) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/BarracudaCompute.cs:1028)
Unity.Barracuda.PrecompiledComputeOps.PrepareModel (Unity.Barracuda.Model model, System.Collections.Generic.IDictionary2[TKey,TValue] inputShapes) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/BarracudaPrecompiledCompute.cs:302) Unity.Barracuda.StatsOps.PrepareModel (Unity.Barracuda.Model model, System.Collections.Generic.IDictionary2[TKey,TValue] inputShapes) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/StatsOps.cs:51)
Unity.Barracuda.GenericWorker+d__33.MoveNext () (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/GenericWorker.cs:214)
Unity.Barracuda.GenericWorker.Execute () (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/GenericWorker.cs:160)
Unity.Barracuda.GenericWorker.Execute (Unity.Barracuda.Tensor input) (at Library/PackageCache/[email protected]/Barracuda/Runtime/Core/Backends/GenericWorker.cs:152)
Mediapipe.PoseDetection.PoseDetecter.ProcessImage (UnityEngine.Texture inputTexture, System.Single poseThreshold, System.Single iouThreshold) (at Library/PackageCache/[email protected]/Script/PoseDetecter.cs:67)
Mediapipe.BlazePose.BlazePoseDetecter.ProcessImage (UnityEngine.Texture inputTexture, Mediapipe.BlazePose.BlazePoseModel blazePoseModel, System.Single poseThreshold, System.Single iouThreshold) (at Packages/Blaz

Is it possible to build this to WebGL? I tried without really changing anything and get the following shader errors:

Shader error in 'Hidden/BlazePose/Visualizer': SV_VertexID semantic is not supported on GLES 2.0 at line 145 (on gles)
Shader error in 'Hidden/BlazePose/Visualizer': SV_VertexID semantic is not supported on GLES 2.0 at line 152 (on gles)
Shader error in 'Hidden/BlazePose/Visualizer': SV_VertexID semantic is not supported on GLES 2.0 at line 159 (on gles)
Shader error in 'Hidden/BlazePose/Visualizer': SV_VertexID semantic is not supported on GLES 2.0 at line 166 (on gles)

Are there some player settings I can adjust or will this not be possible?

Thanks in advance!

Hello,
Thanks for repository .
I have two questions :
1.For multiple person skeltons move one person to another . Can we do to stay on single person ?
2. After detection when user move away from infront of camera ,skelton is still there . I mean creat and destroy . What can be done to not generate if user is not there .

Does the code have functionality to perform 3D avatar animation (like this)? I only see landmark points being animated (stick figure style) in the GitHub page.