With less facial points, can face tracking time be shortened? about openface HOT 6 CLOSED

It is very likely that the tracking would be faster with fewer points being tracked, but it might become a bit less accurate.

Another way to speed up the model is to perform fewer scales when tracking and looking at smaller window sizes. Try playing around with window_sizes_small and window_sizes_init in the LandmarkDetectorParameters, changing window_sizes_small will allow you to control how many scales are run during the tracking and how big is the region of interest.

Thanks,
Tadas

from openface.

Thank you for your helpful comment!
I appreciate it.

from openface.

Could you please describe the meaning of the parameters, window_sizes_small and window_sizes_init, in more detail? How do they relate to the number of scales and the size of the ROI?

Per default the parameters are

window_sizes_init.at(0) = 11;
window_sizes_init.at(1) = 9;
window_sizes_init.at(2) = 7;
window_sizes_init.at(3) = 5;/

window_sizes_small[0] = 0;
window_sizes_small[1] = 9;
window_sizes_small[2] = 7;
window_sizes_small[3] = 0;

Is there a special reason to use this ordering, [0,9,7,9], for the window_sizes_small parameters?

Many Thanks!

from openface.

Each of the numbers in window_sizes_small and window_sizes_init describes the search area (region of interest in pixels, e.g. 11x11).

The vector describes the regions of interest for each scale, so for example of the current windows are set to window_sizes_init, then the algorithm will search an area of 11x11 for scale 1, area of 9x9 for scale 2, area of 7x7 for scale 3, and finally area of 5x5 at scale 4.

In case the window size is set to 0 (as is the case for window_sizes_small in scales 1 and 4), that scale will be skipped. That means the algorithm will only look at a 9x9 area for scale 2 and a 7x7 for scale 3.

Hope this helps.

Thanks,
Tadas

from openface.

Many thanks for the explanation! This helps a lot, but what is meant with "scale 1" or "scale 2"? Where are the scale pyramids defined? Or is it just sampling every pixel (scale = 1), every second (scale = 2) ... ?

from openface.

OpenFace performs landmark detection through 4 pyramid scales, each patch expert is trained to deal with face images of a particular size/scale. The area of interest for each patch expert (centered around current best landmark eztimate) is actually resized and in-plane rotation corrected using bilinear interpolation instead of just pixel resampling. This is done in the following call in PatchExperts.cpp:
cv::warpAffine(grayscale_image, area_of_interest, sim, area_of_interest.size(), cv::WARP_INVERSE_MAP + CV_INTER_LINEAR);

from openface.