Hi I have a question about hand detection module about deep-prior HOT 8 CLOSED

Hi @mks0601 ,
The docom=True calculates the center-of-mass of the hand which is then used for the crop. For the results reported in the paper, I used the center-of-mass for testing as well. You can set docom=False to use the ground truth location, but of course this is not very realistic.

from deep-prior.

However, as in the 319 and 320 line of src/util/handdetector.py,

com[0] += xstart
com[1] += ystart

com is calculated using xstart and ystart which are calculated from the groundtruth joint position(gtorig[0]) as line 305 of src/util/handdetector.py,

xstart, xend, ystart, yend, zstart, zend = self.comToBounds(com, size)

In summary,

1. You use GT joint position(gtorig[0]) to calculate xstart and ystart (In this code, you used gtorig[0] as center-of-mass)
2. And the calculated xstart and ystart are used to calculate a new center-of-mass

So, you used GT joint position to calculate the new center-of-mass even in test stage. Is that right?

from deep-prior.

Hi @mks0601 ,
Yes, as you described. The reason to do this is: (1) it is much faster than running the full detection pipeline, (2) it is equivalent to running the detection pipeline and iterating the com several times.

from deep-prior.

Thank you for your answer. I have a one more question. Can you explain me why did you add xstart and ystart to the com[0] and com[1] respectively (line 319, 320 of src/util/handdetector.py)?

com[0] += xstart
com[1] += ystart

from deep-prior.

Hi @mks0601 ,
The two lines are required, since the com is calculated from the cropped image (com is in image coordinates + depth), and the com is projected back to the full depth image using this offset.

from deep-prior.

Thank you for your answer! It would be a fair comparison if someone just use gt joint position as CoM?

from deep-prior.

+I have an another question
If i follow your pre-processing, some joints are not included in some frames of depth map because of cropping. I think the reason is some noisy pixels which result in biased CoM. Did you notice this situation? How did you handle this situation?

from deep-prior.

Hi @mks0601
Yes I noticed, this can happen, esp with an inaccurate detection. One way is to adjust the crop cube in the importer.py: config = {'cube':(250,250,250)} and make it larger. Or you use the refinement of the CoM. In my experience it is not crucial, if it only effects some frames.

from deep-prior.