danial-kord / digihuman Goto Github PK

Automatic 3D Character animation using Pose Estimation and Landmark Generation techniques

License: GNU General Public License v3.0

ShaderLab 9.39% C# 61.77% Python 27.38% HLSL 1.43% Dockerfile 0.02% Shell 0.01%

mediapipe 3d-landmarks blendshapes pose-estimation animation blazeface blazepose character-animation deep-neural-networks facemesh

digihuman's Introduction

Hi there 👋

About Me

My name is Danial Kordmodanlou. I am currently studying graduate program in Computer Science at York University, Toronto.
I love programming and learning new things! My primary skills and programming languages are C#, Unity3D, Java, Python, and Deep Learning. I am presently concentrating on learning more about Artificial Intelligence(especially Computer Vision). I am also interested in AR and VR applications.
I am passionate about computing science, and I see myself as somebody who loves working on challenging problems and coming up with creative solutions!

My Skills

My Github information:

digihuman's People

Contributors

Stargazers

Watchers

digihuman's Issues

[book-keeping] Local filepaths were left in the code

Searching for your home-directory C:/Danial results in 136 results. In some cases, it might be nice to un-hardcode those and turn them into parameters:

The only ones that might cause an issue are in pose_estimator.py

hand rotation deformation

When the hand trap and movable trap are all turned on, the wrist will be deformed when the hand is turned over. Do you have this problem?

Why don't you use MediaPipeUnityPlugin

Hello,
I'm working an a similar project and your work here is most valuable. I'm wondering if you considered using the MediaPipeUnityPlugin? It seems it would greatly simply the code architecture so it would not be necessary to use the python backend. Is there a reason you chose not to use it?

face animation

Thank you for sharing your great work!
I'm interested in face animation.
I cloned this repository and run on unity, but Unity-chan's face was not animated.
I have questions.

where can I get the charactor using demo?
how to specify each blendshape value to the BlendShapeController component?

The mixamo character doesn't work

After doing the procedure to add a new mixamo character, when I upload only a face video, it doesn't work.

Support the latest 52 blendshapes in your Face Capture

Feature Description

The following commit provides the latest mediapipe 52 blendshapes
google/mediapipe@ba10ae8

The geometry and coefficients are listed here

With this new addition, this provides facial mocap for avatars that use the ARKit 52 blendshapes.

Current Behaviour/State

Currently, Users who are using industry standards e.g. Character Creator 3 limit themselves only facial mocap apps from Apple App Store

Support of Apple Chipset?

Hi there,

I am wondering if this project can support the Apple CPU?
Here are the problems I get:

mediapipe package is not officially support which may not be your problem
warnings package is not supported: I removed it from the code and requirements.txt, which is also fine
python server.py caused errors, this maybe cause by the opencv

objc[1834]: Class CaptureDelegate is implemented in both /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cv2/cv2.abi3.so (0x121fb25a0) and /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mediapipe/.dylibs/libopencv_videoio.3.4.16.dylib (0x122d30860). One of the two will be used. Which one is undefined.
objc[1834]: Class CVWindow is implemented in both /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cv2/cv2.abi3.so (0x121fb25f0) and /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mediapipe/.dylibs/libopencv_highgui.3.4.16.dylib (0x116620a68). One of the two will be used. Which one is undefined.
objc[1834]: Class CVView is implemented in both /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cv2/cv2.abi3.so (0x121fb2618) and /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mediapipe/.dylibs/libopencv_highgui.3.4.16.dylib (0x116620a90). One of the two will be used. Which one is undefined.
objc[1834]: Class CVSlider is implemented in both /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/cv2/cv2.abi3.so (0x121fb2640) and /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mediapipe/.dylibs/libopencv_highgui.3.4.16.dylib (0x116620ab8). One of the two will be used. Which one is undefined.
Traceback (most recent call last):
  File "/Users/nobody/Documents/Projects/playground/DigiHuman/Backend/server.py", line 122, in <module>
    app = Flask(__name__)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py", line 558, in __init__
    self.add_url_rule(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py", line 66, in wrapper_func
    return f(self, *args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/flask/app.py", line 1216, in add_url_rule
    self.url_map.add(rule)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 1562, in add
    rule.bind(self)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 711, in bind
    self.compile()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 767, in compile
    self._build = self._compile_builder(False)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 1128, in _compile_builder
    return self.BuilderCompiler(self).compile(append_unknown)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/werkzeug/routing.py", line 1119, in compile
    co = types.CodeType(*code_args)
TypeError: code() takes at least 14 arguments (13 given)