Hi Dr. Moon, May I ask whether you can open-source the NeuralAnnot? I would like to generate NeuralAnnotation for foot parameters on MSCOCO. Thanks in advance.

I find your offered pseudo-GT file (MSCOCO_train_MANO_NeuralAnnot.json) contain pose, shape, trans for mano, focal ,princpt for camera.
I want to know how to get trans, focal, princpt.

Thank you very much for the parameters you shared!!

I am very interested in Running SMPLify-X on 3D joint coordinates. Will you disclose the modified code of this part?

I learned about this part of your work from the supplementary material of your article. I also want to learn and write the corresponding code, but I don’t know how to start better, so I would like to ask whether this part of the code will be made public.

Looking forward to your reply.

Thanks for this valuable work.
It seems that InstaVariety data only contains images while in the demo file it loads them. Could you please add it to the repository, if possible?

Thank you very much for your work.
Here are some questions I hope you can answer
(1)What are the meanings of the red boxes

(2) why we should merge root pose and camera rotation
` # apply camera extrinsic (rotation)
# 合并根姿势和摄影机旋转 merge root pose and camera rotation

    root_pose = root_pose.numpy()
    root_pose, _ = cv2.Rodrigues(root_pose)#旋转矩阵转换为旋转向量
    root_pose, _ = cv2.Rodrigues(np.dot(R,root_pose))
    root_pose = torch.from_numpy(root_pose).view(1,3)`

@mks0601
Hello! NeuralAnnot is a wonderful work. I have a question about the experiment.

For SPIN used 3D dataset labels, that is, h36m mosh labels and mpi-inf-3dhp_mview_fits, will substituting NeuralAnnot pseudo labels with them raise the score (i.e. indirect 3d annotation error) while keeping ITW datasets labels with the same 2D GT annots without GT SMPL supervision?

Dear author,

it seems the download link for the dataset is not working, could you take a look when you have time to see if it is possible to fix it.

Thanks!

Hi mks0601 !
I am trying to visualize the SMPLXmesh driven with parameters provided in MPI-INF-3DHP_SMPLX_NeuralAnnot.json, and I used the bounding box and camera parameters provided in MPI-INF-3DHP_camera_1k.json. The pose of SMPLX mesh seemed to be right, but the rendered image had incorrect projected mesh.

Thank you for your excellent work! The SMPL parameter is based on a size-reduced image, so does it match the 2D and 3D keypoint annotation provided by the original MPII-3D dataset?

Hi,

May I ask what is the 3D joint positions stored in "Human36M_subject*_joint_3d.json"? Is that the 3D joint label provided in the original datasets or provide by NeuralAnnot?

Thanks,
Yufei

Hello! @mks0601

I am trying to reproduce three recipes based on Pose2Pose. I am not quite sure about the loss implementation. Is that the same as NeuralAnnot (i.e. same as below)?

loss['joint_img'] = self.coord_loss(joint_img, smpl.reduce_joint_set(targets['joint_img']), smpl.reduce_joint_set(meta_info['joint_trunc']), meta_info['is_3D'])
loss['smpl_joint_img'] = self.coord_loss(joint_img, smpl.reduce_joint_set(targets['smpl_joint_img']), smpl.reduce_joint_set(meta_info['smpl_joint_trunc']))
loss['smpl_pose'] = self.param_loss(smpl_pose, targets['smpl_pose'], meta_info['smpl_pose_valid']) # computing loss with rotation matrix instead of axis-angle can avoid ambiguity of axis-angle. current: compute loss with axis-angle. should be fixed.
loss['smpl_shape'] = self.param_loss(smpl_shape, targets['smpl_shape'], meta_info['smpl_shape_valid'][:,None])
loss['joint_proj'] = self.coord_loss(joint_proj, targets['joint_img'][:,:,:2], meta_info['joint_trunc'])
loss['joint_cam'] = self.coord_loss(joint_cam, targets['joint_cam'], meta_info['joint_valid'] * meta_info['is_3D'][:,None,None])
loss['smpl_joint_cam'] = self.coord_loss(joint_cam, targets['smpl_joint_cam'], meta_info['smpl_joint_valid'])
loss['regularizer_shape'] = self.regular_loss(smpl_shape)/100
loss['regularizer_pose'] = self.regular_loss(vposer_para)/100

where regular_loss is the regularize l2 loss (i.e. torch.pow(para,2))

Besides, original 3dpw dataset dataloader don't offer full targets like targets = {'joint_img': joint_img, 'smpl_joint_img': smpl_joint_img, 'joint_cam': joint_cam, 'smpl_joint_cam': smpl_joint_cam, 'smpl_pose': smpl_pose, 'smpl_shape': smpl_shape}. Can you offer the preprocess script to get the joint_img and joint_cam in 3dpw?

When I try to load the insta_train_db.py and insta_test_db.py using torch.load , i got error in load pickle.
I tried with every pytorch version. 1.1 to 1.11
_pickle.UnpicklingError: invalid load key, '\xc0'

How can I load it?

@mks0601

And is the num of NeuralAnnot MPI-INF-3DHP pseudo GT the same as the fits used in SPIN?

Hi Gyeongsik,

I ran the demo_smplx.py and it failed because the shape size in smplx model is 16 while the data from the json file is 10. I added the following code to solve the issue:

shape = torch.cat([shape, torch.zeros((1, 6))], dim=-1)

Is this inconsistency caused by a different version of smplx repository?

Hello, thank you for your outstanding work. I recently encountered some problems in making datasets. I would like to know how you labeled those datasets. Will the code for dataset annotation be made public?

Hi mks0610!
I am reproduce your paper but i am stuck at got 2D joints images from 3D joint coordinates Jˆ3D to the image plane with predicted camera parameters.
Could u public this function and λwild,θ parameter for training SMPL model
Many thanks

Hi，you do such great work!
I find two links released of you.
SMPL from SMPLify-x
NeuralAnnot
I find this two links question about data size.
just like Human36M_subject1_SMPL_NeuralAnnot.json(114.9) vs Human36M_subject1_smpl_param.json(34.8MB)
Is there something diferent? I want to get some detail difference about these two file. Thanks!

Hi, thanks for your excellent work!
I notice that the number of pseudo labels on MSCOCO datasets you provided is 262461, while SPIN and EFT only provide 28344 and 74834 samples, respectively. Could you please tell me what cause the difference? BTW, will the number of samples greatly affect the performance?

Thank you for your contribution, I have a question. How to extract files, such as images_1k.tar.gzaj.gzaj의 사본, how to use passwords to decompress in Ubuntu?

Hello.

Is there any demo code for InterHand2.6M dataset?

Hi,

Thanks making this great work. When checking the data, the action name defined in annotation file does not match with the real action in the video frames. For example, action 4 is marked as Eating in the annotation while the corresponding image frames actually show sitting down. Do I access the data correctly?

Yufei

Dear Authors,
Thank you for your inspiring work.
Will there be a code release?