Eye tracking about clmtrackr HOT 9 CLOSED

I've been working on a prototype for this, and so far results are satisfactory.

Here's my approach:

a) Face detection using clmtrackr.js
b) extracting eye points using the getCurrentPosition() method
c) convert cropped eye area into grayscale, apply a bit of colour balance, and use a threshold algorithm.

Now, this is where it gets "tricky"... After a lot of reading I decided to go with correlation coefficient of two images. I'm comparing the current webcam frame with another frame taken 100ms before, and checking the difference in the image. then based on the correlation coefficient (how different are the images) I decide to mark it as a blink.

Problems with this approach, its not 100% accurate, requires calibration based on room light, skin tone, etc.

Will publish this online at some point (getUserMedia only works over https, or localhost)

I was wondering @auduno , if we could have the same in the eyes as we have in the mouth... We can detect if the mouth gets open or not, how hard is to do the same in the eyes? would be great if the library detected the eyes closing... even if its just an average calculation...

from clmtrackr.

Hi, you're welcome to try, but I don't think clmtrackr is precise enough to be able to do eye tracking.

from clmtrackr.

@andrevenancio Did you ever end up posting the tutorial on this? My lab and I are working on using CLMTrakr for this very purpose and we would love to implement a blink detector to help out with our eye tracking algorithm

from clmtrackr.

I recently heard about http://webgazer.cs.brown.edu which seems to accomplish this

from clmtrackr.

That's what I am working on. Our current blink detector is rather simplistic and we are looking for ways to improve it.

from clmtrackr.

Hi, I haven't posted a tutorial yet, because I'm launching a campaign for Audi, where such technology is implemented. I'll share it over the next few days.
I've done a couple of changes to the tracker though, as it wasn't optimised to run at 60fps.

I'll drop by and comment with the code over the next few days.

from clmtrackr.

Hmm, @andrevenancio you optimized it to run at 60FPS? Could that possible solve #70 as well? If so, I would love to mess around with that fork of the tracker.

from clmtrackr.

Hey @Skylion007 I just published a draft of the tutorial on my website.

https://andrevenancio.com/eye-blink-detection/

Let me know if there's anything I'm missing.

from clmtrackr.

@andrevenancio Wow, thank you! This is a really impressive tutorial. Feel free to take a look at WebGazer when you have the chance; seems like you might have a lot of useful insights on the project.

from clmtrackr.