pupil-labs / pupil-detectors Goto Github PK

View Code? Open in Web Editor NEW

23.0 9.0 18.0 7.31 MB

License: GNU Lesser General Public License v3.0

Python 1.92% C++ 86.10% Shell 0.10% CMake 2.08% Cython 9.80%

pupil-detectors's Introduction

https://img.shields.io/badge/skeleton-2022-informational

pupil-detectors

This Python package contains the standalone 2D pupil detectors for the Pupil Core software stack.

Install via PyPI

pip install pupil-detectors

Usage

Here's a quick example on how to detect and draw an ellipse.

import cv2
from pupil_detectors import Detector2D

detector = Detector2D()

# read image as numpy array from somewhere, e.g. here from a file
img = cv2.imread("pupil.png")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

result = detector.detect(gray)
ellipse = result["ellipse"]

# draw the ellipse outline onto the input image
# note that cv2.ellipse() cannot deal with float values
# also it expects the axes to be semi-axes (half the size)
cv2.ellipse(
   img,
   tuple(int(v) for v in ellipse["center"]),
   tuple(int(v / 2) for v in ellipse["axes"]),
   ellipse["angle"],
   0, 360, # start/end angle for drawing
   (0, 0, 255) # color (BGR): red
)
cv2.imshow("Image", img)
cv2.waitKey(0)

Developers

Building from Source

Installing the dependencies

macOS: brew install eigen opencv
Windows: choco install eigen opencv
Ubuntu: apt-get install libeigen3-dev libeigen3-dev

Building the Python package

# Clone repository
git clone [email protected]:pupil-labs/pupil-detectors.git
cd pupil-detectors

# Install from source
pip install .

pupil-detectors's People

Contributors

Stargazers

Watchers

Forkers

romanroibu pfaion jojodevel vedb jasionkit elmadjian kbarkevich rizwanbinyasin allycasablancas erenatas marklescroart ankurrajw longmarcher cristianrodrigoguarachiibanez ajinkyapuar fourier-times shadow-johnson bsun1220

pupil-detectors's Issues

error during installation

I have problems during installation of pupil-detectors...
I'm currently using Windows 10 with python version 3.8.5
I have the following error that I think is related tu Numpy version.. ( the version istalled is numpy==1.19.3)

C:\Users\anton\Desktop\pupil-detectors-master>pip install pupil-detectors
Collecting pupil-detectors
  Downloading pupil-detectors-2.0.1.tar.gz (238 kB)
     |â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ| 238 kB 364 kB/s
  Installing build dependencies ... done
 ** On entry to DGEBAL parameter number  3 had an illegal value
 ** On entry to DGEHRD  parameter number  2 had an illegal value
 ** On entry to DORGHR DORGQR parameter number  2 had an illegal value
 ** On entry to DHSEQR parameter number  4 had an illegal value
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: 'c:\users\anton\appdata\local\programs\python\python38\python.exe' 'c:\users\anton\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pep517\_in_process.py' get_requires_for_build_wheel 'C:\Users\anton\AppData\Local\Temp\tmpvnrrlwvo'
       cwd: C:\Users\anton\AppData\Local\Temp\pip-install-me6l4as2\pupil-detectors
  Complete output (20 lines):
  Traceback (most recent call last):
    File "c:\users\anton\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 280, in <module>
      main()
    File "c:\users\anton\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "c:\users\anton\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 114, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\anton\AppData\Local\Temp\pip-build-env-y19lrlt2\overlay\Lib\site-packages\setuptools\build_meta.py", line 149, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "C:\Users\anton\AppData\Local\Temp\pip-build-env-y19lrlt2\overlay\Lib\site-packages\setuptools\build_meta.py", line 130, in _get_build_requires
      self.run_setup()
    File "C:\Users\anton\AppData\Local\Temp\pip-build-env-y19lrlt2\overlay\Lib\site-packages\setuptools\build_meta.py", line 145, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 20, in <module>
      import numpy as np
    File "C:\Users\anton\AppData\Local\Temp\pip-build-env-y19lrlt2\overlay\Lib\site-packages\numpy\__init__.py", line 305, in <module>
      _win_os_check()
    File "C:\Users\anton\AppData\Local\Temp\pip-build-env-y19lrlt2\overlay\Lib\site-packages\numpy\__init__.py", line 302, in _win_os_check
      raise RuntimeError(msg.format(__file__)) from None
  RuntimeError: The current Numpy installation ('C:\\Users\\anton\\AppData\\Local\\Temp\\pip-build-env-y19lrlt2\\overlay\\Lib\\site-packages\\numpy\\__init__.py') fails to pass a sanity check due to a bug in the windows runtime. See this issue for more information: https://tinyurl.com/y3dm3h86
  ----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\anton\appdata\local\programs\python\python38\python.exe' 'c:\users\anton\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pep517\_in_process.py' get_requires_for_build_wheel 'C:\Users\anton\AppData\Local\Temp\tmpvnrrlwvo' Check the logs for full command output.

ERROR: Failed building wheel for pupil-detectors

I tired to build pupil-detectors in Raspberry Pi 3 Model B Rev 1.2 with ubuntu-18.04.5-preinstalled-server-arm64+raspi3 server but could not get it to work.

ubuntu@ubuntu:~/pupil-detectors$ sudo pip3 install ".[dev]"
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
WARNING: The directory '/home/ubuntu/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Processing /home/ubuntu/pupil-detectors
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: numpy in /home/ubuntu/.local/lib/python3.6/site-packages (from pupil-detectors==1.1.1) (1.19.4)
Requirement already satisfied: cysignals in /home/ubuntu/.local/lib/python3.6/site-packages (from pupil-detectors==1.1.1) (1.10.2)
Requirement already satisfied: pytest; extra == "dev" in /home/ubuntu/.local/lib/python3.6/site-packages (from pupil-detectors==1.1.1) (6.1.2)
Requirement already satisfied: tox; extra == "dev" in /home/ubuntu/.local/lib/python3.6/site-packages (from pupil-detectors==1.1.1) (3.20.1)
Requirement already satisfied: Cython>=0.28 in /home/ubuntu/.local/lib/python3.6/site-packages (from cysignals->pupil-detectors==1.1.1) (0.29.21)
Requirement already satisfied: iniconfig in /home/ubuntu/.local/lib/python3.6/site-packages (from pytest; extra == "dev"->pupil-detectors==1.1.1) (1.1.1)
Requirement already satisfied: toml in /home/ubuntu/.local/lib/python3.6/site-packages (from pytest; extra == "dev"->pupil-detectors==1.1.1) (0.10.2)
Requirement already satisfied: packaging in /home/ubuntu/.local/lib/python3.6/site-packages (from pytest; extra == "dev"->pupil-detectors==1.1.1) (20.4)
Requirement already satisfied: pluggy<1.0,>=0.12 in /home/ubuntu/.local/lib/python3.6/site-packages (from pytest; extra == "dev"->pupil-detectors==1.1.1) (0.13.1)
Requirement already satisfied: attrs>=17.4.0 in /home/ubuntu/.local/lib/python3.6/site-packages (from pytest; extra == "dev"->pupil-detectors==1.1.1) (20.2.0)
Requirement already satisfied: importlib-metadata>=0.12; python_version < "3.8" in /home/ubuntu/.local/lib/python3.6/site-packages (from pytest; extra == "dev"->pupil-detectors==1.1.1) (2.0.0)
Requirement already satisfied: py>=1.8.2 in /home/ubuntu/.local/lib/python3.6/site-packages (from pytest; extra == "dev"->pupil-detectors==1.1.1) (1.9.0)
Requirement already satisfied: filelock>=3.0.0 in /home/ubuntu/.local/lib/python3.6/site-packages (from tox; extra == "dev"->pupil-detectors==1.1.1) (3.0.12)
Requirement already satisfied: virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 in /home/ubuntu/.local/lib/python3.6/site-packages (from tox; extra == "dev"->pupil-detectors==1.1.1) (20.1.0)
Requirement already satisfied: six>=1.14.0 in /home/ubuntu/.local/lib/python3.6/site-packages (from tox; extra == "dev"->pupil-detectors==1.1.1) (1.15.0)
Requirement already satisfied: pyparsing>=2.0.2 in /home/ubuntu/.local/lib/python3.6/site-packages (from packaging->pytest; extra == "dev"->pupil-detectors==1.1.1) (2.4.7)
Requirement already satisfied: zipp>=0.5 in /home/ubuntu/.local/lib/python3.6/site-packages (from importlib-metadata>=0.12; python_version < "3.8"->pytest; extra == "dev"->pupil-detectors==1.1.1) (3.4.0)
Requirement already satisfied: appdirs<2,>=1.4.3 in /home/ubuntu/.local/lib/python3.6/site-packages (from virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0->tox; extra == "dev"->pupil-detectors==1.1.1) (1.4.4)
Requirement already satisfied: importlib-resources>=1.0; python_version < "3.7" in /home/ubuntu/.local/lib/python3.6/site-packages (from virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0->tox; extra == "dev"->pupil-detectors==1.1.1) (3.3.0)
Requirement already satisfied: distlib<1,>=0.3.1 in /home/ubuntu/.local/lib/python3.6/site-packages (from virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0->tox; extra == "dev"->pupil-detectors==1.1.1) (0.3.1)
Building wheels for collected packages: pupil-detectors
  Building wheel for pupil-detectors (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /home/ubuntu/.local/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmp4g4275hz
       cwd: /tmp/pip-req-build-3qjqemjf
  Complete output (31 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-aarch64-3.6
  creating build/lib.linux-aarch64-3.6/pupil_detectors
  copying src/pupil_detectors/__init__.py -> build/lib.linux-aarch64-3.6/pupil_detectors
  copying src/pupil_detectors/utils.py -> build/lib.linux-aarch64-3.6/pupil_detectors
  creating build/lib.linux-aarch64-3.6/pupil_detectors/detector_3d
  copying src/pupil_detectors/detector_3d/__init__.py -> build/lib.linux-aarch64-3.6/pupil_detectors/detector_3d
  creating build/lib.linux-aarch64-3.6/pupil_detectors/detector_2d
  copying src/pupil_detectors/detector_2d/__init__.py -> build/lib.linux-aarch64-3.6/pupil_detectors/detector_2d
  running build_ext
  building 'pupil_detectors.detector_base' extension
  creating build/temp.linux-aarch64-3.6
  creating build/temp.linux-aarch64-3.6/src
  creating build/temp.linux-aarch64-3.6/src/pupil_detectors
  aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c src/pupil_detectors/detector_base.cpp -o build/temp.linux-aarch64-3.6/src/pupil_detectors/detector_base.o -w -std=c++11 -O2
  aarch64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-aarch64-3.6/src/pupil_detectors/detector_base.o -o build/lib.linux-aarch64-3.6/pupil_detectors/detector_base.cpython-36m-aarch64-linux-gnu.so
  building 'pupil_detectors.detector_2d.detector_2d' extension
  creating build/temp.linux-aarch64-3.6/src/pupil_detectors/detector_2d
  creating build/temp.linux-aarch64-3.6/src/singleeyefitter
  creating build/temp.linux-aarch64-3.6/src/singleeyefitter/ImageProcessing
  aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Isrc -Isrc/pupil_detectors/detector_2d -Isrc/shared_cpp/include -Isrc/singleeyefitter/ -I/tmp/pip-build-env-0hp57gr0/overlay/lib/python3.6/site-packages/numpy/core/include -I/usr/local/opt/opencv/include -I/usr/local/opt/opencv@3/include -I/usr/local/include/opencv4 -I/usr/include/opencv4 -I/usr/local/include/eigen3 -I/usr/include/eigen3 -I/usr/include/python3.6m -c src/pupil_detectors/detector_2d/detector_2d.cpp -o build/temp.linux-aarch64-3.6/src/pupil_detectors/detector_2d/detector_2d.o -w -std=c++11 -O2
  aarch64-linux-gnu-gcc: internal compiler error: Killed (program cc1plus)
  Please submit a full bug report,
  with preprocessed source if appropriate.
  See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
  /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'zip_save'
    warnings.warn(msg)
  error: command 'aarch64-linux-gnu-gcc' failed with exit status 4
ERROR: Failed building wheel for pupil-detectors
Failed to build pupil-detectors
ERROR: Could not build wheels for pupil-detectors which use PEP 517 and cannot be installed directly.

Edit by @papr: Added log formatting

refactor "algorithm" debug ?

The use of algorithm view is necessary to adjust the pupil intensity threshold.
I think there would be added value to return the algorithm debug info as objects rather than embedding it in a color image.
The advantages would be :

avoid having to convert to rgb when the native frame is not rgb (grayscale, yuv, ...), for instance just np.tile to convert from our gray input to rgb is making a 250 fps to drop dramatically, causing frame loss/lags/overflow. ( Can't use numpy tricks as the array needs to be c-contiguous.)
display debug in pupil by drawing over the captured image, prettier when windows scaled, and likely faster than rasterizing on CPU.
allowing external program to analyze the debug info.

WDYT?

Add small examples that show how to use the detector with pyav and OpenCV

Documentation of Detect2D options

Hello, thanks for the very useful code. I'm looking for some guidance regarding the options that can be fed to the Detect2D class as a properties dict. I see them enumerated in the code (here: https://github.com/pupil-labs/pupil-detectors/blob/master/src/pupil_detectors/detector_2d/detector_2d.pyx), but I haven't yet found any help on them anywhere. Some seem straightforward (e.g. pupil_size_min) but others (e.g. ellipse_roundness_ratio) don't. Is the latter a minimum ratio between the two axes, perhaps? In general, just looking for an explanation of parameters to tweak given our specific eye data. Thanks for your time.

ImportError: DLL load failed while importing detector_2d

I build pupil-detectors on WIndows, it seems everything is OK.

But when i run the example code, it throw error:

Documentation on Approach

Is there any documentation or paper that explains the method used here?

Is it the same approach as is outlined in https://dl.acm.org/doi/10.1145/2638728.2641695?:

Eye image converted to grayscale, user region of interest (white stroke rectangle), and initial estimation of pupil region (white square and dashed line square.)
Canny edge detection (green lines.)
Define “dark” region as offset from lowest spike in histogram within eye image.
Filter edges to exclude spectral reflections (yellow.) and not inside “dark” areas (blue)
Remaining edges extracted into contours using connected components and split into sub-contours based on curvature continuity criteria (multi colored lines).
Candidate pupil ellipses (blue) are formed using ellipse fitting
Final ellipse fit found through an augmented combinatorial search ( finally ellipse with center in red) - supporting edge pixels drawn in white.

Utilized OpenCV Version

When installing pupil-detectors via pip install pupil-detectors I get the following error when importing with
from pupil_detectors import Detector2D, Detector3D:

/usr/local/lib/python3.6/dist-packages/pupil_detectors/detector_2d/detector_2d.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN2cv12findContoursERKNS_11_InputArrayERKNS_12_OutputArrayEiiNS_6Point_IiEE

I guess my OpenCV version (3.4.6) is not compatible. What version do I need?

Detect 2 pupils

Hi, how can I use Detector2D() to detect 2 pupils in one image?
Is there any suggested way of implementing this? Thanks

Detector2D detection parameters

Hi,

the performance of Detector2D on LPW dataset seems to have problems with parameters where only the pupil_size_max and pupil_size_min were adapted.

I used single image Detector2D with the following parameters:

    detector = Detector2D({'2d': {
        "coarse_detection": True,
        "coarse_filter_min": 128,
        "coarse_filter_max": 280,
        "intensity_range": 23,
        "blur_size": 5,
        "canny_treshold": 160,
        "canny_ration": 2,
        "canny_aperture": 5,
        "pupil_size_max": 200,
        "pupil_size_min": 60,
        "strong_perimeter_ratio_range_min": 0.6,
        "strong_perimeter_ratio_range_max": 1.1,
        "strong_area_ratio_range_min": 0.8,
        "strong_area_ratio_range_max": 1.1,
        "contour_size_min": 5,
        "ellipse_roundness_ratio": 0.09,
        "initial_ellipse_fit_treshhold": 4.3,
        "final_perimeter_ratio_range_min": 0.5,
        "final_perimeter_ratio_range_max": 1.0,
        "ellipse_true_support_min_dist": 3.0,
        "support_pixel_ratio_exponent": 2.0,
    }})
    result = detector.detect(gray)

And this high res image:

The detector did not find the pupil at all.

Another example is a lower res image with the following parameters where also no pupil is found

detector = Detector2D({'2d': {
        "coarse_detection": True,
        "coarse_filter_min": 128,
        "coarse_filter_max": 280,
        "intensity_range": 23,
        "blur_size": 5,
        "canny_treshold": 160,
        "canny_ration": 2,
        "canny_aperture": 5,
        "pupil_size_max": 50,
        "pupil_size_min": 15,
        "strong_perimeter_ratio_range_min": 0.6,
        "strong_perimeter_ratio_range_max": 1.1,
        "strong_area_ratio_range_min": 0.8,
        "strong_area_ratio_range_max": 1.1,
        "contour_size_min": 5,
        "ellipse_roundness_ratio": 0.09,
        "initial_ellipse_fit_treshhold": 4.3,
        "final_perimeter_ratio_range_min": 0.5,
        "final_perimeter_ratio_range_max": 1.0,
        "ellipse_true_support_min_dist": 3.0,
        "support_pixel_ratio_exponent": 2.0,
    }})
    result = detector.detect(gray)

Is it possible to detect the pupil with different parameters?
Can you tell me which parameters might be most important for tuning here?

Segmentation Fault when Passing Color Image to detect

eg, running this on a color image:

import cv2
from pupil_detectors import Detector2D
detector = Detector2D()
detector.detect(cv2.cvtColor(cv2.imread('/Users/me/Downloads/iris.png', cv2.IMREAD_COLOR), cv2.COLOR_BGR2RGB))

Causes a segmentation fault.