chris10m / ev2hands Goto Github PK

Hi @Chris10M,

Thank you for the great work and for making the code available.

To get an insight into the Ev2Hands-R data, I plotted the ground-truth joint annotations on the event frames (plot_data.py). I observed that for some frames, the joint annotations used to train the Ev2Hands model are not accurate; the projected 2D skeleton does not coincide accurately with the event cloud, as seen in the attached images (column 1: input RGB image, column 2: input event frame, column 3: input event frame + ground-truth 2D joint skeleton). The fingers are straight in the input RGB and event frames, but the annotated skeleton appears to have bent fingers. This would have adversely affected the network training, leading to suboptimal accuracy.

Is this inaccuracy in the ground-truth joint annotations due to the motion tracking system (Captury)? Or could this be due to some misalignment in the synchronization of the event and RGB streams?

If possible, is there any alternative to using torch-mesh-isect? I found torch-mesh-isect not easy to install.

Hi @Chris10M,

Thank you for providing the synchronized RGB streams and the MANO parameters. I had a query regarding the number of samples in this data. Below are the number of samples for each subject.

Could you tell me the correspondence across these datasets? Specifically, given a sample index in rgb_mano_params.pkl, what is the corresponding frame in subject_{id}_rgb.mp4? Also, for subjects 1 and 5, given a sample index in event_mano_params.pkl what is the corresponding sample index in rgb_mano_params.pkl?