vchoutas / smplx Goto Github PK
View Code? Open in Web Editor NEWSMPL-X
Home Page: https://smpl-x.is.tue.mpg.de/
License: Other
SMPL-X
Home Page: https://smpl-x.is.tue.mpg.de/
License: Other
Is the function converting joint in canonical view to camera view?
'
Line 316 in 2144d5c
which means converting 'joints_3d' to 'transform_chain'?
as in the blog: https://ikvision.github.io/
and how to convert joints_3d to transform_chain step by step?
Hi, @vchoutas!
First of all, thank you for such nice work and clear code!
I want to fit SMPL-X model to images, where some joints are not visible, e.g. upper-body photos or even face photos. I tried several images and found out that the method works much worse when some joints are out of the frame (you can see examples below). The resulting fits don't much the input at all.
I believe this issue can be somehow fixed by nullifying some joints' weights during optimization or by some other optimization tweak. What do you think is the best way to do it?
Input | SMPLify-X |
---|---|
Hi,
Suppose I already have a smplx human body model, given the pose, hand, and face detected by openpose, how to generate corresponding pose of the smplx human body mode?
Thanks.
How should I fix this error?
[jalal@goku smplx]$ python tools/clean_ch.py --input-models /scratch3/3d_pose/humanpose/SMPL_models/models/smplh/*.pkl --output-folder output-folder
Traceback (most recent call last):
File "tools/clean_ch.py", line 68, in <module>
clean_fn(input_model, output_folder=output_folder)
File "tools/clean_ch.py", line 34, in clean_fn
body_data = pickle.load(body_file)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 1036: ordinal not in range(128)
[jalal@goku smplx]$ python tools/clean_ch.py --input-models /scratch3/3d_pose/humanpose/SMPL_models/models/smpl/*.pkl --output-folder output-folder
Traceback (most recent call last):
File "tools/clean_ch.py", line 68, in <module>
clean_fn(input_model, output_folder=output_folder)
File "tools/clean_ch.py", line 34, in clean_fn
body_data = pickle.load(body_file)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x81 in position 1224: ordinal not in range(128)
Also, do you remove the Chumpy objects only from smpl and smplh *.pkl files or also smplx *.pkl files?
When I try to run the following code :
python3 smplifyx/main.py --config cfg_files/fit_smplx.yaml --data_folder test_images --output_folder results --visualize="True" --model_folder models/smplx/SMPLX_NEUTRAL.npz --vposer_ckpt vposer --part_segm_fn smplx_parts_segm.pkl
Output:
Processing: test_images/images/01_img.jpg
Found Trained Model: vposer/snapshots/TR00_E096.pt
Traceback (most recent call last):
File "smplifyx/main.py", line 272, in
main(**args)
File "smplifyx/main.py", line 262, in main
**args)
File "/media/Simpl-x/smplify-x-master/smplifyx/fit_single_frame.py", line 274, in fit_single_frame
focal_length=focal_length, dtype=dtype)
File "/home/username/.local/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 43, in decorate_no_grad
return func(*args, **kwargs)
File "/media/Simpl-x/smplify-x-master/smplifyx/fitting.py", line 89, in guess_init
for edge in edge_idxs:
TypeError: 'NoneType' object is not iterable
I get this error. I have installed all the requirements and I have cuda installed. I tried to run this example with the test_images folder provided.
test_images.zip
Thanks for your code! Is there any way to get the texture map of smplx?
I'm currently studying the model to try to use it for a project and I can't understand the difference between the SMPLX and SMPLXLayer classes.
More concretely, why does SMPLX use angle-axis format and SMPLXLayer rotation matrix format for joint rotations? Are there any other differences like this? Which one should I use?
Thanks in advance.
hello .would you be kind to tell me about the vertices between the SMPL and SMPL-X model?
Thanks for your awesome work!
I would like to get the semantic part info of smplx, but I can't find it. Can you tell me where can I get this? or how to generate by myself?
Hello! Thanks for the great work you've done!
I have a question about the class of SMPLH (as well as SMPLX) model of package smplx from PyPi. There are following lines:
left_hand_pose = torch.einsum(
'bi,ij->bj', [left_hand_pose, self.left_hand_components])
right_hand_pose = torch.einsum(
'bi,ij->bj', [right_hand_pose, self.right_hand_components])
which as I understand provide axis-angle hand pose coefficients from PCA coefficients.
But when parameter use_pca
set to False when initializing the model, it causes an obvious error when calling forward()
:
AttributeError: 'SMPLX' object has no attribute 'left_hand_components'
My question is: shouldn't there be an if-statement before this coefficient expansion?
I'd just like to take a minute to celebrate all the work that went into making this installation work. Thanks to all the contributors! (Choutas, Pavlakos, [manyMoreNamesSorryImLazyRightNow], MJBlack, et al.) And their families, friends, institutions that helped mold them into what they are today. Also to the many contributors who helped make many versions of python, conda, and pip possible!
I did all this on August 12th and 13th of 2020
Practically, here's commands that worked for me:
# conda version == conda 4.8.3 (output of "conda activate")
conda create --name smplx0 python=3.6 # ended up downloading 3.6.10
conda activate smplx0
# install:
pip install smplx
pip install pyrender
conda install -c conda-forge trimesh
# Download SMPL-X model like they say:
... [cmds, Googling, Chrome browsing]
python3 examples/demo.py --model-folder /home/nathan_bendich/Downloads/SMPL-X_Models/models/ --plot-joints=True --gender="neutral"
# NOTE above: model folder doesn't point at smplx, but at "models"
and system information:
uname -a
: Linux nathanBendich-HP-Laptop-14-fq0xxx 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
If anyone has issues installing, hopefully some of this information is helpful. I'm sure it will be helpful to future-me at some point 😃
Hi,
I found a bug on shapedirs
of MANO.
Although the v_template
of the right and left hands, are different, shapedirs
is the same.
This can make the same betas
value make different hand shape.
Setting betas=torch.FloatTensor([4.4785, 0.3873, -0.3134, -4.4374, -1.7321, -2.1621, -0.0627, 5.1609, -1.5386, -0.3230]).view(1,-1)
outputs below results. The left hand right hands have very different hand shape although the betas
is the same.
Setting left_hand_mano.shapedirs[:,0,:] *= -1
and forward the betas
outputs good result.
Lines 360 to 364 in 85f8570
hello, thx your demo. When I want run demo.py, I got the error like this:
`Traceback (most recent call last):
File "examples/demo.py", line 99, in
use_face_contour=use_face_contour)
File "examples/demo.py", line 66, in main
pyrender.Viewer(scene, use_raymond_lighting=True)
File "/home/liwensh/anaconda3/lib/python3.7/site-packages/pyrender/viewer.py", line 347, in init
self._init_and_start_app()
File "/home/liwensh/anaconda3/lib/python3.7/site-packages/pyrender/viewer.py", line 997, in _init_and_start_app
height=self._viewport_size[1])
File "/home/liwensh/anaconda3/lib/python3.7/site-packages/pyglet/window/xlib/init.py", line 170, in init
super(XlibWindow, self).init(*args, **kwargs)
File "/home/liwensh/anaconda3/lib/python3.7/site-packages/pyglet/window/init.py", line 592, in init
config = screen.get_best_config(config)
File "/home/liwensh/anaconda3/lib/python3.7/site-packages/pyglet/canvas/base.py", line 197, in get_best_config
raise window.NoSuchConfigException()
pyglet.window.NoSuchConfigException
`
any suggestion can you give me about how to solve it?
Do you guys provide the correspondences of the MANO vertices to the vertices of the SMPL-X model? Thanks
Hello,
I used your tool to remove chumpy
from older SMPL male model. However, I still get this error.
UnicodeDecodeError: 'ascii' codec can't decode byte 0x81 in position 1224: ordinal not in range(128)
Hi,
I'm not sure I missed something or not.
Because there are a lot of implicite requirements needed (like geting my keypoint from OpenPose).
Anyhow, I'd like to get a 3d model output from a single image.
I've successfully managed:
however, I've got the following error:
File "C:\Users\user1\Desktop\WoT\Video\smplify-x\smplifyx\fit_single_frame.py", line 196, in fit_single_frame
body_mean_pose = body_pose_prior.get_mean().detach().cpu()
File "C:\Users\user1\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 591, in getattr
type(self).name, name))
AttributeError: 'L2Prior' object has no attribute 'get_mean'
(i am running torch 1.2.0, for this test)
From that point I have no clue how to debug that.
A little help would be much appreciated :-)
thx
Hi,
I'm trying to find ways to convert SMPL-X models to SMPL and noticed the updates in this repo on transfer between models in SMPL family. Unfortunately, I don't think the provided Using the code section of the README.md
file inside transfer_model
directory is making any sense to me since I cannot find the mentioned main.py
script. Trying to run transfer_model.py
or __main__.py
(though it seems to be something prepared for building a package or something and not for directly running but that seems to be where some argument parsing happens) inside transfer_model
with the same arguments only gets me errors like
Traceback (most recent call last):
File "transfer_model/transfer_model.py", line 26, in <module>
from .utils import get_vertices_per_edge
ModuleNotFoundError: No module named '__main__.utils'; '__main__' is not a package
Traceback (most recent call last):
File "transfer_model/__main__.py", line 30, in <module>
from .config import parse_args
ModuleNotFoundError: No module named '__main__.config'; '__main__' is not a package
which are indicating that's just not right.
What am I doing wrong? I'm using Python 3.7 virtualenv on Windows 10.
Thanks in advance! @vchoutas
Hey, I noticed that using global_orient in SMPL will also slightly translate the model as opposed to only rotating it. Is that intended?
Below you can see a comparison of two sets of mesh vertices, one rotated using global_orient param in SMPL.foward() [teal], and the other one rotated by applying scipy.Rotation.from_rotvec on the vertices [orange]. I also compared the results with cv2.Rodrigues which leads to similar results as scipy.
In line 705 of body_models.py the lbs() function takes self.betas as an argument, whereas I think it should be betas.
In its current form, the betas parameter in the forward method of SMPL-H is being ignored.
after doing
pip install smplx[all]
and then
git clone https://github.com/vchoutas/smplx
do i need to do cd smplx
and then do python setup.py install
? because otherwise I get the python: can't open file 'setup.py': [Errno 2] No such file or directory
erorr
I am trying to implement this on colab. I am getting this error when I am trying it on my own image. I already created the folder for images and keypoints. And I manually provided the image and the son file for that image in the directory as suggested.
File "/content/smplify-x/smplifyx/fit_single_frame.py", line 188, in fit_single_frame
vposer, _ = load_vposer(vposer_ckpt, vp_model='snapshot')
File "/usr/local/lib/python3.6/dist-packages/human_body_prior/tools/model_loader.py", line 56, in load_vposer
ps, trained_model_fname = expid2model(expr_dir)
File "/usr/local/lib/python3.6/dist-packages/human_body_prior/tools/model_loader.py", line 31, in expid2model
best_model_fname = sorted(glob.glob(os.path.join(expr_dir, 'snapshots', '*.pt')), key=os.path.getmtime)[-1]
IndexError: list index out of range
How can I use NTU-120's 3D 25-joint information instead of using OpenPose?
In body_models.py#L2053, https://github.com/vchoutas/smplx/blob/f4206853a4746139f61bdcf58571f2cea0cbebad/smplx/body_models.py#L2053
self.batch_size ==> batch_size.
It may caused errors when using multi-GPU.
"""""""
lmk_bary_coords = self.lmk_bary_coords.unsqueeze(dim=0).repeat(
self.batch_size, 1, 1)
""""""Should be"""
lmk_bary_coords = self.lmk_bary_coords.unsqueeze(dim=0).repeat(
batch_size, 1, 1)
"""""""
I use hotmail.com to register your website ,but nothing ,I want to test it
Hey, thank you so much for the fantastic work! Is there a documentation of all the joints name obtained from SMPLX model: https://github.com/vchoutas/smplx/blob/master/examples/demo.py#L43
I can only find the following 20 joints index name: https://github.com/vchoutas/smplx/blob/master/smplx/vertex_ids.py#L48-L69
But joints like neck, spine, are missing from the list.
A list of names for each joint index will be really helpful! Thank you!
It looks likes you have not finish the code in your smplx/utils.py ,the function rot_mat_to_euler implementation is incomplete.
How can I generate a joint model of a person along with feet and hands? I have no need for a 3D mesh.
hi
thanks for this repo and your amazing work.
So I have installed the repo and the pyrender and trimesh dependencies without trouble I think.
A ran the demo.py exemple mentioned in the readme and I got a 3D T-Pose 3D model, neutral, female or male depending the parameter specified.
So great, but , how do I specify the image I want the 3D full model (hands, face and body) to fit? I do not see any image input argument in the demo.py code nor any output argument for the results to be written.
I am probably missing something obvious but please can you help on how to inder the model on an input image and where to find the output results/objects?
thanks for your clarifications.
https://github.com/vchoutas/smplx/blob/master/smplx/vertex_ids.py
In this file it says that **Joint name to vertex mapping. SMPL/SMPL-H/SMPL-X vertices **, however it just contains SMPL-H and SMPL-X vertex mapping. Is SMPL mapping missing here?
if i can modify the model mesh to change the bone length
Could you please share the link to download the EHF dataset as mentioned in the paper and this video?
https://www.youtube.com/watch?v=XyXIEmapWkw
Hi,
Thanks for the great work.
(Not really an issue but) I would like to ask if there is a way for using smplx as a 3DMM for the head (i.e. use only head data and parameters).
Is there a fast out-of-the-box way to do this or should I try to separate the head of the model (and their corresponding parameters) according to the head joints?
Thank you very much.
Hello,
Thank you for creating a wonderful human model.
I am doing research about an inverse kinematic SMPL model that predicts SMPL joint angle [global orient + plus pose] from a temporal sequence of 3d relative keypoints [with respect to mid-hip]. Because the model overfits very fast, I need to add some augmented rotation to training.
The idea is that each smplx sequence will be rotated by a random angle, at the center of the whole sequence, around the vertical axis.
So far I have tried to apply a delta rotation to the global_orient and trans parts of the sequence, but the result is not correct.
Could you please tell me what would be a correct way to implement that?
I look forward to hearing from you soon.
Here is the code of my experiment so far
poses = data["poses"]
trans = data["trans"]
aug_angle = np.pi * 0.2 # augmentation angle
aug_axis = [0.0, 1.0, 0.0] # augmentation axis
# axis-angle augmentation rotation
org_aa = torch.from_numpy(poses[:, :3])
aug_aa = torch.tensor(np.array(aug_axis) * aug_angle).unsqueeze(0).repeat((org_aa.shape[0], 1))
# apply augmentaion to the original global_orient, through multiplication of two quaternions
final_qu = angle_axis_to_quaternion(aug_aa) * angle_axis_to_quaternion(org_aa)
# convert back to angle axis representation
global_orient = quaternion_to_angle_axis(final_qu)
# set the global orient
data["poses"][:, :3] = global_orient
# now apply the sampe rotation to the global translation part of the SMPL sequence
aug_rot = transform.Rotation.from_rotvec(np.array(aug_axis) * aug_angle)
new_trans = aug_rot.apply(trans)
data["trans"] = new_trans
When I run the demo, I found the SMPL model has 45 joints? It should be 24?
when i run the python examples/demo.py --model-folder /home/lxz/mygrade2/smplx/smplx/models/ --plot-joints=True --gender="neutral"
something may be wrong:
**python examples/demo.py --model-folder /home/lxz/mygrade2/smplx/smplx/models/ --plot-joints=True --gender="neutral"
SMPLX(
Gender: NEUTRAL
Number of joints: 55
Betas: 10
Number of PCA components: 6
Flat hand mean: False
Number of Expression Coefficients: 10
(vertex_joint_selector): VertexJointSelector()
)
Vertices shape = (10475, 3)
Joints shape = (127, 3)
Traceback (most recent call last):
File "examples/demo.py", line 180, in
use_face_contour=use_face_contour)
File "examples/demo.py", line 80, in main
pyrender.Viewer(scene, use_raymond_lighting=True)
File "/home/lxz/anaconda3/envs/smplx/lib/python3.6/site-packages/pyrender/viewer.py", line 347, in init
self._init_and_start_app()
File "/home/lxz/anaconda3/envs/smplx/lib/python3.6/site-packages/pyrender/viewer.py", line 997, in _init_and_start_app
height=self._viewport_size[1])
File "/home/lxz/anaconda3/envs/smplx/lib/python3.6/site-packages/pyglet/window/xlib/init.py", line 173, in init
super(XlibWindow, self).init(*args, kwargs)
File "/home/lxz/anaconda3/envs/smplx/lib/python3.6/site-packages/pyglet/window/init.py", line 603, in init
config = screen.get_best_config(config)
File "/home/lxz/anaconda3/envs/smplx/lib/python3.6/site-packages/pyglet/canvas/base.py", line 194, in get_best_config
raise window.NoSuchConfigException()
pyglet.window.NoSuchConfigException
I created a smplx object by set use_face_contour as true, then i forward the beta para and the expression para(all batch size is 20), but i got an error as fllows:
RunTime Error: invalid argument 7, equal number of batches expected at /torch/aten/src/THC/generic/THCTensorMathBlas.cu:523. If i set use_face_contour is false, this error disappeared.
Hello, thank you for your great work!
I use neural network to learn the parameters of smplx, and use gt_vertices and gt_parameters and keypoints as supervision, but the output parameters generate the following human body
When I only use the gt_parameters as the supervision, the waist posture will not be abnormal, but in other cases, the waist posture always has different degrees of abnormality.
In addition, in various training situations, the surface of the face and hand is always not smooth.
Can you give me some advice? thank you
Can you please let me know whether I can run this code on CPU?
Hi,
Thanks for your perfect work. I wanna ask how to compute the skinning weight? Because I wanna use different template. Thank you very much.
Best,
Jin
Hello,
I'm trying to run smlpx on Ubuntu 20.04.1 LTS
I installed smplx using pypi, and now trying to download the SMPL-X model from your website.
But I don't know how to download the file from your website.
I signed up using Windows10, so I already have the account.
But, as I copied the download link and used it with 'wget' on linux, an error message says "Unable to locally verify the issuer's authority."
So I tried again with '--no-check-certificate' option, and it did downloaded a file, but the file cannot be unzipped using 'unzip'.
Also, the size of the downloaded zip file is too small, so I guess it's not an intact file.
Can you plz help me with this?
Could you please share the npz files for SMPL-H model?
I get the following error:
[jalal@goku smplx]$ python examples/demo.py --model-folder /scratch3/3d_pose/humanpose/SMPL_models/models --model-type="smplh" --plot-joints=True --gender="female" --use-face-contour=True
Traceback (most recent call last):
File "examples/demo.py", line 138, in <module>
use_face_contour=use_face_contour)
File "examples/demo.py", line 33, in main
ext=ext)
File "/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/smplx/body_models.py", line 99, in create
return SMPLH(model_path, **kwargs)
File "/scratch/sjn-p3/anaconda/anaconda3/lib/python3.6/site-packages/smplx/body_models.py", line 468, in __init__
smplh_path)
AssertionError: Path /scratch3/3d_pose/humanpose/SMPL_models/models/smplh/SMPLH_FEMALE.npz does not exist!
[jalal@goku smplx]$ ls /scratch3/3d_pose/humanpose/SMPL_models/models/smplh/
total 254740
drwxrwx---. 4 jalal cs-grad 4096 Sep 24 17:33 ..
-rwx------. 1 jalal cs-grad 130422893 Sep 24 17:33 SMPLH_male.pkl
drwxr-xr-x. 2 jalal cs-grad 64 Sep 24 17:34 .
-rwx------. 1 jalal cs-grad 130423021 Sep 24 17:34 SMPLH_female.pkl
I have been using SMPL-based projects continually, such as SURREAL, HMR/HMMR.
I found it's convenient to inspect SMPL in 3D software (e.g. Blender and Maya) by FBX model .
I guess it may be possible for one to produce an FBX model from .pkl and .npy files.
It would be great if we can use an FBX model for SMPL-X.
Since dataclass is available only in recent version of python, it would be great to add it to rquirements.txt, so that older version of python is also supported.
context: open-mmlab/mmpose#184 (comment)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.