kariaro / vrigunity Goto Github PK

Add an experimental option to allow leg movements. This would be useful for both vr and mmd programs.

Instead of making large issues lets make them small.

First part of the bodytracking components is making sure the wrist and arm are rotated together

Make a script that can automatically update the project version.

This could be updated from an editor script in Unity e.g the ApplicationVersion script

These files needs to be automatically updated

• installer.iss
• ProjectSettings/ProjectSettings.asset

Add a baseline for language in the app

• All text fields in the app should be changable during runtime
• Each language should get its own language file

Add a -resetSettings command to make it possible to fix configuration errors

Update the MediaPipeUnityPlugin to v0.10.1 and add a build readme for how to download the packages for local builds.

Update the workflow scripts to download and unpack the libraries for mediapipe.

An issue has been posted on the MediaPipeUnityProject:
homuler/MediaPipeUnityPlugin#779

Hi, how can I free up the movement of the hips in 3D space? What is the part that is blocking the hips?
thanks

Add some workflow code that automatically changes target branches to develop if the branch is not develop

Add a box selection tool that allows a user to select were the hands should be tracked.

Add a way to use this tool for VRChat

Update project readme and make it easier to use.

• Update the gif to display the a newer version of the project UI
• Add a link to the latest release (Make it a big Download button)
• Add some more information about the project

The project is starting to contain very large files for different systems.
To fix all these issues the project will migrate to git lfs

This will break all current forks

OS

Windows

App Version

0.5.0

Description

There's an issue with tracking of fingers being constantly lost or not picked up straight away, while tracking of wrists (and everything else) is working fine. Happens when using this program in conjunction with vSeeFace. Changing settings doesn't seem to fix the problem.

VOD Showing the issue: https://www.youtube.com/live/1pHrDu1hecI?si=I3io7epS-T0PYhCN

Logs

No response

Add the version number to the window title

OS

Mac

App Version

lastest version

Description

After installing the app on my laptop, a MacBook Pro with the cpu M2, and the Mac OS 13.2.1, it didn't work. It shows the only hint "The application 'VRigUnity' cannot be opened."

Logs

No response

How does your motion capture project support importing fbx models? My Fbx model is based on APose, and VRM is based on TPose.

Adding a virtual camera mode for linux systems

Steps:

• Check how easy it would be to add v4l2loopback support
• Give a guide on how to install it
• Create a test that checks if the virtual webcam works

Hey Kariaro, I see that MediaPipeUnityPlugin has been updated MediaPipeUnityPlugin v0.12.0 ， then ARKit BlendeShape support has been added, is it possible to add this support in the next version, thanks.

Thanks for the amazing example on mapping skeleton data to vrm. I'm using the following code to map the legs.

{
        Vector3 rKnee = ConvertPoint(eventArgs.value, 25);
        Vector3 rAnkle = ConvertPoint(eventArgs.value, 27);
        
        Vector3 vRigA = Vector3.right;
        Vector3 vRigB = rKnee - rhip;
        Quaternion rot = Quaternion.FromToRotation(vRigA, vRigB);
        rUpperLeg = rot;
        
        Vector3 vRigC = rAnkle - rKnee;
        rot = Quaternion.FromToRotation(vRigA, vRigC);
        rLowerLeg = rot;
}

But the mapping appears to be wrong. The leg is rotated and translated wrong. Anything wrong with the way that I compute the rotation?

To reduce CPU usage check if other projects than mediapipe gives better performance

• https://github.com/creativeIKEP/HolisticBarracuda
• Custom made python script / camera setup
  This might help with adding virtual cameras to more systems

This is in response to #66

I found several bugs.

1. The accuracy of finger recognition is not enough.
2. There is a problem with fist reversal.
3. 3. The thumb is not recognized.
4. 4. The left and right swing of fingers is not recognized.
5. 5. It would be great if expression recognition could be added.

For large models > 100 MB add a warning message that pops up that it can take a long time to load

I cannot get this to work at all, I downloaded DSSBodyTracker which was a clone of an older version of this project and that clone version actually worked, other than not sending anything through VMC. So I found the original (this project) and whenever I choose start camera the model's eyes close. It doesn't matter which camera I use. This results in this program being completely useless in it's current state.

Add a recording button to export/save mocap data that should be able to be imported into tools like blender

Try update how the tracking is applied to the 3d model

• Find a way to use more face expression blend shapes
• Find a way to make finger tracking more accurate
• Check if the mic could be used to move the mouth
• Make the IK system default and fix body rotation bug

Branch feature/optimize-app

Take inspiration from other UI’s and make a small sketch of how it could look.

• It should be possible to see the model when a user updates the settings
• Text size should be smaller (Find a layout that makes this possible)
• GUI size should be automatic unless explicitly specified
• Add more icons and less text, a camera icon instead of text (But with hover tooltip)

Talk with the community on what should be prioritized.

Check that MediaPipe works during a pull request to develop

Add an issue template to make it easier to fix bugs or issues

Add a button that shows debug information:

• Current Fps
• Current triangle count
• How much cpu used
• How much memory used

This could maybe also contain more information such as:

• A graph of missed frames
• Average fps
• Mediapipe AI update rate

https://issuetracker.unity3d.com/issues/could-not-find-specified-video-device-error-when-using-webcamtexture-with-obs-virtual-camera

Update to version 2021.3.14f1 to fix this issue

Update the project UI and make use of better design patterns

Why do I use the Apose character model to replace the Vrm model in the project, and there are ghosts in the activity?

Find a way to implement v4l2loopback to allow virtual camera devices to be used on Linux.
Try find solutions that require as little maintenance as possible.

Research and find a way to denoise the output data using temporal cohesion

For version v0.3.0 read the camera resolution and update the camera resolution list to get the maximum size for the camera.

Also look if it's possible to add a custom field to edit the camera resolution manually

Add an option to select the output resolution of the virtual camera
This should not be dependant on the size of the window

Are you interested in supporting an Android build?

Find a way to remove .meta fiels from the StreamingAssets folder.

Tasks:

• Check if the folder could be moved outside of Assets
• If not check if the app builds if a .gitignore is introduced removing .meta files from the folder

Will the project support whole-body detection or even motion posture detection?

Currently, I can only track upper body, will you be able to track lower body?

Hello author! Your holistic motion capture is much smoother than what I wrote, thank you for making it open source. But I noticed a problem, I can't move the position of the VRM model in the scene. I have a VR treadmill. In my code, as long as I import the SDK of the VR treadmill, I can move the VRM model while motion capture. But in your code, I can't change the position of VRM. Can you tell me what to change in the code to move the VRM model?

Check if lowering the AI model complexity will lower the CPU usage

Add a low poly default model as an option

Currently even when approved external PRs can't run any unity tests because they do not have access to any secrets.
Find a way to make this work while still being secure

Great repo. Could I run it in mobile device?

OS

Windows

App Version

v0.4.1

Description

Windows 11, CPU i7 7700k , webcam Logitech c925e
The software using the camera and sending VMC is using 65% of my CPU.
Even when webcam and VMC sender is off , the CPU usage this is 18 %.

It does not make sense to me. I have done the handtracking in mediapepi in python and my usage was lower than 10%.

Maybe an option to change de model complexity, mine was 0 and working on the hands.

extras issues:

• cannot get nice performance when using webcam with higher resolution than 848x480 (30hz) (becames choppy)
• cannot set webcam to use 60 fps.
• Bones configuration are lost at restart of program

Logs

No response

Check how easy it would be to add code linting to the project.
This would make sure all code if formatted correctly

After receiving some more views on this project I’m going to see if I could spend some time improving the model rigging :D

Compile this project for OSX/iOS and make sure the basic features work.
Virtual webcams are not required to work for this issue.