Comments (1)
Hi @moo1o . Thanks for your good questions! Both are important for the story of Champ.
Question 1:
-
The SMPL used here is to enhance the 3D consistency of the condition maps. There are also many great work to directly predict normal, depth, and so on from a single RGB Image. But these work somehow ignore the temporal and 3D consistency. For example, when we do SMPL fitting to a video, the adjacent frames maintain the geometry consistency for human body: you can find a whole body with a head and two hands🕺, instead of flickered condition maps without 3D constraints on human from those prediction networks.
-
SMPL is way more than a mesh. It has been widely used in both industry and academy. So the use of SMPL extends Champ's downstream applications. Like there are many Text(audio)-2-SMPL, Text-to-Texture map work, which could easily integrated into Champ at inference time. If you are familiar with Blender, you could do lots of creative things with Champ. You can try this CEB Blender adds-on.
Question 2:
- Yeah they are not perfectly aligned. You may refer to works of Human Mesh Recovery to follow the SOTA. But actually the 4D-Humans we use in Champ has enough accuracy and robustness, except the hands and faces expression. If you want more realistic motions, you may try directly predict depth, normal, and use the semantic map from SMPL.
from champ.
Related Issues (20)
- basicModel_neutral_lbs_10_207_0_v1.0.0.pkl HOT 5
- Improving results with anime characters HOT 1
- Hardware and data requirements for model training. HOT 3
- How to avoid distortion of facial and hand features in generated videos HOT 4
- Free guidance: IndexError: The shape of the mask [0] at index 0 does not match the shape of the indexed tensor [1, 9216, 320] at index 0 HOT 3
- Organization of training data per the README causes errors HOT 6
- cuda out of memory HOT 2
- when i test my video, i have error HOT 3
- A problem about scaled_focal_length during motion datas producing HOT 1
- Transfer SMPL HOT 2
- [MISSING FILE] SMPL_NEUTRAL.pkl
- raise ValueError('Invalid device ID ({})'.format(device_id, len(devices))), ValueError: Invalid device ID (0) HOT 1
- High memory usage for inference HOT 1
- reference image HOT 6
- how can I get "--reference_path reference_imgs/smpl_results/your_ref_img_A.npy"? HOT 2
- Have you try to use SDXL for training?If chang from SD1.5 to SDXL will further to improve performance?
- render_condition_map sometimes has errors
- How to install Blender in the environment 'champ' HOT 2
- WARNING HOT 2
- SMPL-X integration
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from champ.