Mesekai is a real-time motion tracking virtual avatar application. Use your physical body through webcam video to control the avatar's arms, fingers, head, and facial expressions. It supports personalized ReadyPlayerMe avatars.

The primary use case is Vtubing. Additionally, the motion tracking feature can be applied to a 3rd person online multiplayer game for immersive social interactions. Enter the hub world to try this feature with other players.

Mesekai - Webcam Motion Tracking Avatars

itch.io page

Mesekai repository

Motion Tracking Avatars in Desktop Multiplayer Online Games

This project performs pose estimation using my fork of the MediaPipeUnityPlugin. Refer to the git documentation on setting up submodules. Follow MediaPipeUnityPlugin's instructions to build the plugin. Once successful, move /Packages/com.github.homuler.mediapipe/ and /Assets/StreamingAssets/ into the main project.

Also need git LFS.