hwangdonghyun / vnect-tensorflow-keras Goto Github PK

As the paper said there shoud be two intermidate at res4d and res5a, should I add the whole block of bone length and deltax,y,z and then spit and compute the l2 loss? Or I just go from res4d and res5a to get the heatmap,ignore the bone length and deltas to compute the l2 loss?
I tried both, it seems if I use the whole block wth bone length and deltas, it will never get
converged.
Or should I add a loss funtion for the deltas and bone length?
Can you give me any idea about this?
thank u

Hi there - I am trying to train the 2d heatmap using data that I generated myself using the kinect for windows sdk. I am also using images that are 540x540px and created 2d heatmaps (68x68px) from that data, but I am not seeing results that would lead me to believe that I am on the right path. Can you further elaborate on what you changed about the model's structure when training the 2d heatmap (in addition to what you've referenced in your bullet points)?

hello, I recently study vnect, and find that your code do not use bone length information as intermediate supervision. However，paper mention this bone length as intermediate supervison,
so I am confused.

Hi there. I acquired a dataset that has accurate joint positions in a different coordinate space that I want to use to train this model. I transformed the joints to be root-relative (to the pelvis) as I understood from the vnect paper. However, I am struggling to understand how one should produce the location maps. I understand x and y fairly well, but I am a bit confused what I should do with z outside of the region that would be covered by the heatmap. The loss function should send everything to 0 outside of that region, but am I missing something? Do you have any guidance?

Furthermore, I successfully trained the 2D heatmap already (though I did it with individual points for the joints, not a gaussian distribution) - when you train for 3D, do you include the 2D heatmap as a loss term with mean squared error again - and should the 2d heatmap be created using a gaussian? The paper alludes to this but it is not explicitly stated.

Thanks in advance.