Comments (6)
Thank you for documenting this! I was wondering why the boxes are so big...
from depthai-experiments.
the bounding boxes for eyes are generated on the UI, so they can be changed. However, they are now reflecting what is being sent to the neural network. It accepts eye images scaled to 60x60, and from landmarks neural network I receive a single eye point, so using padded_point(..., padding=30, ...) is producing a correct bounding box that I can send further.
This is also why this bounding box is sometimes too big / too small - as it doesn't take the face dimensions into account.
One fix for this, that would be fairly easy to do, is to somehow correlate the face dimensions with eye size, so that for given face width/height we can estimate the eyes width/height. This way, we could draw a better bounding box and the scaled image for nn would contain less non-eye parts, so it may also improve efficiency
Sharing also here for readability
from depthai-experiments.
Drawing Bbox should have as main objective to be used as debugging.
The gaze model requires a 60x60 px input for the eyes.
Having too small or a large bbox, help understand why the model works well or badly. (If we are too close or too far).
To improve the cropped area (and thus have a better bbox for the eyes), we must find a way to adapt the resolution of the cropped face, using multiple resolutions/zoom for example ...
Changing the drawing of the bbox, to adapt well, change the effect and not the cause ...
So, I agree with @VanDavv about it.
from depthai-experiments.
So am I right that currently we are feeding the wrong 'size' into the NN? I mean we don't take the face size into account?
from depthai-experiments.
Yes, rescaling for the near range (depending on input resolution) and using the zoom feature (not yet implemented / finalized) for far distances should improve the result.
from depthai-experiments.
Ideally, we would have access to the training dataset and try to crop it the same (for eyes) ... ;-)
from depthai-experiments.
Related Issues (20)
- OAK-D-SR Camera save color image and depth image by record.py HOT 3
- [gen2-yolo/device-decoding] Visualization doesn't show camera footage or detections HOT 2
- To add mesh generation for gen2-pointcloud and Multiple camera HOT 2
- How to visualize when I record data with gen2-record-replay?
- Why pylance is complaining about NNData(), node, and ImageManipConfig() call in script.py from gen2-gaze-estimation project?(Solved) HOT 2
- Head posture estimation module HOT 2
- segmentation with yolov5
- Head pose estimation HOT 2
- How to load external video or webcam feed and process it on device side HOT 1
- New recommendations on head pose estimation model. HOT 1
- Support for YOLOP HOT 2
- Depth information in deep learning model HOT 3
- Camera extrinsic parameter HOT 1
- Issue with installing requirements for RTSP streaming HOT 1
- Deploy yolov8 on OAK-D HOT 6
- Challenges Installing PyGObject for OKA Camera RTSP Streaming HOT 1
- Can we specify the layers you are interested in for a custom nn ? HOT 8
- Yolo v8 on OAK-D camera HOT 1
- gen2-pedestrian-reidentification freezes while person is detected HOT 3
- Yolov8 on OAK-1 using depthai_sdk HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from depthai-experiments.