Provide code (not data) for Human3.6m + SMPL training about graphcmr HOT 12 CLOSED

The code for training is already available on this repo. You just need to choose the FullDataset in the command line options.
We include code to prepare the test data of Human3.6M for evaluation. You can use this code to create the .npz file for the train data with only trivial modifications. You will need though to manually add the SMPL betas and shape parameters in the npz file. Our dataset class should be able to handle all that with no changes.

from graphcmr.

That's great thanks -- I'll close issue for now :)

from graphcmr.

I'm not sure this is quite trivial yet... The version of SMPL parameters I have contains a series of pkl files named (action)_cam(id)_aligned.pkl. In these files there is a 'new_poses' and 'betas' entry.

I'm struggling at present to get my SMPL renders to line up with the 3D key-points from S24 which originally come from 'D3_Positions_mono'.

I render these with code like:

poses = smpl_data['new_poses'][frame_i / 5]
betas = smpl_data['betas']
verts, joints_3d = SMPL(betas[None, :], poses[None, :])

Then to get them to even nearly align, I have to flip the y-coordinate:

verts *= np.array([1, -1, 1])
joints_3d *= np.array([1, -1, 1])

However, the result still looks wrong to me:

I wonder if you have any tips on getting this to work?

from graphcmr.

Did you try to align them? E.g. subtract the 3D location of the same joint for the 2 sets of joints? This should remove any spatial offset.

from graphcmr.

That would work in theory, although (as far as I know) your code only expects a set of shape & pose parameters for SMPL... I wouldn't be able to provide these offsets downstream?

It would be very useful to see the code your team used in this dataloader (which creates a .npz file) if at all possible.

from graphcmr.

This is the standard process used in most implementations. We do not explicitly regress depth or the precise translation of the object in the camera coordinate system. In the training and evaluation code as you can see, I align them at the root joint. So it should work as intended with the pose and beta parameters we have.

from graphcmr.

I see... In this case, does it not matter that the 3D joints generated by the SMPL model (via shape + pose) do not align with the 3D key points from S24 (via D3_positions_mono) in the preprocessing script?

from graphcmr.

Yes it does not matter. But you might have to check this issue, where they had to reflect the pose.

from graphcmr.

Thanks so much for the help, and the tip about the 'upside-down' people. I'll close the issue again and train tonight.

from graphcmr.

As a very brief final question... is it strange that I can't see any 2D positions marked on the Human3.6m images when I view in tensorboard? It was this that began my debugging exercise -- I assumed I had them positioned incorrectly.

See example of joints identified in top row and not in human3.6m bottom row:

from graphcmr.

You should also populate the training .npz files with the correct 2D joints. This is probably why you can't see the 2D locations here.
Also, for the SMPL parameters, you can confirm that they are correct by generating the mesh and projecting it to the image. Then you can easily evaluate the alignment between the model and the image.

from graphcmr.

Hi @nkolot , what is the different processing between *.pkl and the *_camx_aligned.pkl, or could you tell me where I can find the process scripts? I want to get the continuous smpl mesh.

from graphcmr.