head pose estimation interpretation about deepgaze HOT 9 CLOSED

Hi @ahmelshi I think your problem is that you did not isolate the face frame. You cannot pass the entire image to Deepgaze, the face requires to be isolated. You can do it using any face detector (e.g. haar-cascade in OpenCV). The Prima dataset includes a text file with the coordinates of the faces, you can use that coordinates to isolate the faces. It is necessary to pass a square frame.

Massimiliano

from deepgaze.

In that image the cropping may be correct (would be better to be close to the chin on the bottom).

But I notice that there is a distorsion in the face proportion. Probably because it was a rectangular frame which has been squashed in a square frame without an appropriate crop. This kind of aberrattions may negatevely effect the performance of the classifier. When the frame is rectangular (let's say of size R x C) would be better to take the longest side (let's suppose it is R) and use it to crop a squared frame (in this example of size R x R) centered on the face.

Another thing to take into account is the image format. Deepgaze uses BGR convention like in OpenCV, if you are reading the image with another library it is possible that you get an image in RGB format which can cause trouble to the CNN. I suggest you to use the OpenCV imread() function to read the images.

The head pose estimator module does not require any other particular pre-processing operation. An internal normalisation is performed at pixel level and appropriate resize is applied to images larger than 64x64 pixels.

Massimiliano

from deepgaze.

Hi,

The last image you sent looks good. The type of cropping I generally use is similar to the one in this image.

The output of the network should be quite accurate for yaw angles in the range [+60, -60] degrees.
Unfortunately the performances slightly degradate when the face is completely turned. This is due to the fact that the images used for training have a low number of faces for those positions.

Massimiliano

from deepgaze.

Regarding pitch and roll there are some limitations as well. The roll estimator has been trained in the range [-25, +25] degrees, whereas the pitch span from -45 to +45 degrees. Also in this case when you move to the end of the range the performance deteriorate. These limits are due to the AFLW dataset which is the one used for the training. In this moment there is not another dataset in-the-wild having the same amount of images. The positive thing is that the aferementioned ranges allow estimating the pose of most of the images released in social networks and media.

Massimiliano

from deepgaze.

Hi Massimiliano,

Thanks for your fast reply.

just to confirm, would a CelebA-unaligned image like so https://ibb.co/dFMPuQ is considered "correct square frame" to be passed to deepgaze-head pose estimation ? also any more specifications to consider when using deepgaze head pose estimation ?

Thanks in advance :)

from deepgaze.

Hi Massimiliano,

Thanks alot for your detailed feedback. I appreciate your help.

To follow your instructions i tried with (http://conradsanderson.id.au/lfwcrop/) and random pic from that db would like this (https://ibb.co/dFz3M5) (img_dim is 64x64) (read with cv2.imread) i guess this fits the description, however deepgaze produces Estimated [roll, pitch, yaw] ..... [-1.51543,7.9395,49.6683], i just visually compare it to (https://ibb.co/njF4Tk) and wouldn't this should be a yaw of (+90) ? or what do you think is wrong in addressing this interpretation from my side?

Thanks in advance :)

from deepgaze.

Hi Massimiliano,

Thank you for the quick feedback and reply.

Any other "performance know about", regarding the other two angles (pitch and roll) like you mention deepgaze works perfect for yaw angles in the range [+60, -60] degrees. what about (pitch and roll) ranges ?

Many thanks to you.

from deepgaze.

Thanks a lot Massimiliano for your help.

from deepgaze.

You are welcome.

from deepgaze.