Gaze Estimation Bounding Boxes Left/Right eye drawings about depthai-experiments HOT 6 OPEN

Thank you for documenting this! I was wondering why the boxes are so big...

from depthai-experiments.

the bounding boxes for eyes are generated on the UI, so they can be changed. However, they are now reflecting what is being sent to the neural network. It accepts eye images scaled to 60x60, and from landmarks neural network I receive a single eye point, so using padded_point(..., padding=30, ...) is producing a correct bounding box that I can send further.

This is also why this bounding box is sometimes too big / too small - as it doesn't take the face dimensions into account.

One fix for this, that would be fairly easy to do, is to somehow correlate the face dimensions with eye size, so that for given face width/height we can estimate the eyes width/height. This way, we could draw a better bounding box and the scaled image for nn would contain less non-eye parts, so it may also improve efficiency

Sharing also here for readability

from depthai-experiments.

Drawing Bbox should have as main objective to be used as debugging.
The gaze model requires a 60x60 px input for the eyes.
Having too small or a large bbox, help understand why the model works well or badly. (If we are too close or too far).

To improve the cropped area (and thus have a better bbox for the eyes), we must find a way to adapt the resolution of the cropped face, using multiple resolutions/zoom for example ...

Changing the drawing of the bbox, to adapt well, change the effect and not the cause ...
So, I agree with @VanDavv about it.

from depthai-experiments.

So am I right that currently we are feeding the wrong 'size' into the NN? I mean we don't take the face size into account?

from depthai-experiments.

Yes, rescaling for the near range (depending on input resolution) and using the zoom feature (not yet implemented / finalized) for far distances should improve the result.

from depthai-experiments.

Ideally, we would have access to the training dataset and try to crop it the same (for eyes) ... ;-)

from depthai-experiments.