Giter Club home page Giter Club logo

glasses-detector's Introduction

Glasses Detector

Colab Docs PyPI Python CUDA DOI License

Banner

About

Package for processing images with different types of glasses and their parts. It provides a quick way to use the pre-trained models for 3 kinds of tasks, each divided into multiple categories, for instance, classification of sunglasses or segmentation of glasses frames.


Classification ๐Ÿ‘“ transparent ๐Ÿ•ถ๏ธ opaque ๐Ÿฅฝ any โžฟshadows
Detection ๐Ÿค“ worn ๐Ÿ‘“ standalone ๐Ÿ‘€ eye-area
Segmentation ๐Ÿ˜Ž full ๐Ÿ–ผ๏ธ frames ๐Ÿฆฟ legs ๐Ÿ” lenses ๐Ÿ‘ฅ shadows

$\color{gray}{\textit{Note: }\text{refer to}}$ Glasses Detector Features $\color{gray}{\text{for visual examples.}}$

Installation

Important

Minimum version of Python 3.12 is required. Also, you may want to install Pytorch (select Nightly for compatibility) in advance to select specific configuration for your device and environment.

Pip Package

If you only need the library with pre-trained models, just install the pip package and see Quick Start for usage (also check Glasses Detector Installation for more details):

pip install glasses-detector

You can also install it from the source:

git clone https://github.com/mantasu/glasses-detector
cd glasses-detector && pip install .

Local Project

If you want to train your own models on the given datasets (or on some other datasets), just clone the project and install training requirements, then see Running section to see how to run training and testing.

git clone https://github.com/mantasu/glasses-detector
cd glasses-detector && pip install -r requirements.txt

You can create a virtual environment for your packages via venv, however, if you have conda, then you can simply use it to create a new environment, for example:

conda create -n glasses-detector python=3.12
conda activate glasses-detector 

To set-up the datasets, refer to Data section.

Quick Start

Command Line

You can run predictions via the command line. For example, classification of a single image and segmentation of images inside a directory can be performed by running:

glasses-detector -i path/to/img.jpg -t classification -d cuda -f int # Prints 1 or 0
glasses-detector -i path/to/img_dir -t segmentation -f mask -e .jpg  # Generates masks

Tip

You can also specify things like --output-path, --size, --batch-size etc. Check the Glasses Detector CLI and Command Line Examples for more details.

Python Script

You can import the package and its models via the python script for more flexibility. Here is an example of how to classify people wearing sunglasses:

from glasses_detector import GlassesClassifier

# Generates a CSV with each line "<img_name.jpg>,<True|False>"
classifier = GlassesClassifier(size="small", kind="sunglasses")
classifier.process_dir("path/to/dir", "path/to/preds.csv", format="bool")

And here is a more efficient way to process a dir for detection task (only single bbox per image is currently supported):

from glasses_detector import GlassesDetector

# Generates dir_preds with bboxes as .txt for each img
detector = GlassesDetector(kind="eyes", device="cuda")
detector.process_dir("path/to/dir", ext=".txt", batch_size=64)

Tip

Again, there are a lot more things that can be specified, for instance, output_size and pbar. It is also possible to directly output the results or save them in a variable. See Glasses Detector API and Python Script Examples for more details.

Demo

Feel free to play around with some demo image files. For example, after installing through pip, you can run:

git clone https://github.com/mantasu/glasses-detector && cd glasses-detector/data
glasses-detector -i demo -o demo_labels.csv --task classification:eyeglasses

You can also check out the demo notebook which can be also accessed via Google Colab.

Data

Before downloading the datasets, please install unrar package, for example if you're using Ubuntu (if you're using Windows, just install WinRAR):

sudo apt-get install unrar

Also, ensure the scripts are executable:

chmod +x scripts/*

Once you download all the datasets (or some that interest you), process them:

python scripts/preprocess.py --root data -f -d

Tip

You can also specify only certain tasks, e.g., --tasks classification segmentation would ignore detection datasets. It is also possible to change image size and val/test split fractions: use --help to see all the available CLI options.

After processing all the datasets, your data directory should have the following structure:

โ””โ”€โ”€ data                    # The data directory (root) under project
    โ”œโ”€โ”€ classification
    โ”‚   โ”œโ”€โ”€ anyglasses      # Datasets with any glasses as positives
    โ”‚   โ”œโ”€โ”€ eyeglasses      # Datasets with transparent glasses as positives
    โ”‚   โ”œโ”€โ”€ shadows         # Datasets with visible glasses frames shadows as positives
    โ”‚   โ””โ”€โ”€ sunglasses      # Datasets with semi-transparent/opaque glasses as positives 
    โ”‚
    โ”œโ”€โ”€ detection
    โ”‚   โ”œโ”€โ”€ eyes            # Datasets with bounding boxes for eye area 
    โ”‚   โ”œโ”€โ”€ solo            # Datasets with bounding boxes for standalone glasses
    โ”‚   โ””โ”€โ”€ worn            # Datasets with bounding boxes for worn glasses
    โ”‚
    โ””โ”€โ”€ segmentation
        โ”œโ”€โ”€ frames          # Datasets with masks for glasses frames
        โ”œโ”€โ”€ full            # Datasets with masks for full glasses (frames + lenses)
        โ”œโ”€โ”€ legs            # Datasets with masks for glasses legs (part of frames)
        โ”œโ”€โ”€ lenses          # Datasets with masks for glasses lenses
        โ”œโ”€โ”€ shadows         # Datasets with masks for eyeglasses frames cast shadows
        โ””โ”€โ”€ smart           # Datasets with masks for glasses frames and lenses if opaque

Almost every dataset will have train, val and test sub-directories. These splits for classification datasets are further divided to <category> and no_<category>, for detection - to images and annotations, and for segmentation - to images and masks sub-sub-directories. By default, all the images are 256x256.

Note

Instead of downloading the datasets manually one-by-one, here is a Kaggle Dataset that you could download which already contains everything.

Download Instructions

Download the following files and place them all inside the cloned project under directory data which will be your data --root (please note for some datasets you need to have created a free Kaggle account):

Classification datasets:

  1. From CMU Face Images download cmu+face+images.zip
  2. From Specs on Faces download original images.rar and metadata.rar
  3. From Sunglasses / No Sunglasses download archive.zip and rename to sunglasses-no-sunglasses.zip
  4. From Glasses and Coverings download archive.zip and rename to glasses-and-coverings.zip
  5. From Face Attributes Grouped download archive.zip and rename to face-attributes-grouped.zip
  6. From Face Attributes Extra download archive.zip and rename to face-attributes-extra.zip
  7. From Glasses No Glasses download archive.zip and rename to glasses-no-glasses.zip
  8. From Indian Facial Database download An Indian facial database highlighting the Spectacle.zip
  9. From Face Attribute 2 download FaceAttribute 2.v2i.multiclass.zip (choose v2 and Multi Label Classification format)
  10. From Glasses Shadows Synthetic download archive.zip and rename to glasses-shadows-synthetic.zip

Detection datasets:

  1. From AI Pass download AI-Pass.v6i.coco.zip (choose v6 and COCO format)
  2. From PEX5 download PEX5.v4i.coco.zip (choose v4 and COCO format)
  3. From Sunglasses Glasses Detect download sunglasses_glasses_detect.v1i.coco.zip (choose v1 and COCO format)
  4. From Glasses Detection download Glasses Detection.v2i.coco.zip (choose v2 and COCO format)
  5. From Glasses Image Dataset download glasses.v1-glasses_2022-04-01-8-12pm.coco.zip (choose v1 and COCO format)
  6. From EX07 download Ex07.v1i.coco.zip (choose v1 and COCO format)
  7. From No Eyeglass download no eyeglass.v3i.coco.zip (choose v3 and COCO format)
  8. From Kacamata-Membaca download Kacamata-Membaca.v1i.coco.zip (choose v1 and COCO format)
  9. From Only Glasses download onlyglasses.v1i.coco.zip (choose v1 and COCO format)

Segmentation datasets:

  1. From CelebA Mask HQ download CelebAMask-HQ.zip and from CelebA Annotations download annotations.zip
  2. From Glasses Segmentation Synthetic Dataset download archive.zip and rename to glasses-segmentation-synthetic.zip
  3. From Face Synthetics Glasses download archive.zip and rename to face-synthetics-glasses.zip
  4. From Eyeglass download eyeglass.v10i.coco-segmentation.zip (choose v10 and COCO Segmentation format)
  5. From Glasses Lenses Segmentation download glasses lenses segmentation.v7-sh-improvments-version.coco.zip (choose v7 and COCO format)
  6. From Glasses Lens download glasses lens.v6i.coco-segmentation.zip (choose v6 and COCO Segmentation format)
  7. From Glasses Segmentation Cropped Faces download glasses segmentation cropped faces.v2-segmentation_models_pytorch-s_1st_version.coco-segmentation.zip (choose v2 and COCO Segmentation format)
  8. From Spects Segmentation download Spects Segementation.v3i.coco-segmentation.zip (choose v3 and COCO Segmentation)
  9. From KINH download kinh.v1i.coco.zip (choose v1 and COCO format)
  10. From Capstone Mini 2 download CAPSTONE_MINI_2.v1i.coco-segmentation.zip (choose v1 and COCO Segmentation format)
  11. From Sunglasses Color Detection download Sunglasses Color detection roboflow.v2i.coco-segmentation.zip (choose v2 and COCO Segmentation format)
  12. From Sunglasses Color Detection 2 download Sunglasses Color detection 2.v3i.coco-segmentation.zip (choose v3 and COCO Segmentation format)
  13. From Glass Color download Glass-Color.v1i.coco-segmentation.zip (choose v1 and COCO Segmentation format)

The table below shows which datasets are used for which tasks and their categories. Feel free to pick only the ones that interest you.

Task Category Dataset IDs
Classification anyglasses 1, 3, 4, 5, 6, 7, 8, 9, 14, 15, 16
Classification eyeglasses 2, 4, 5, 6, 11, 12, 13, 14, 15
Classification sunglasses 1, 2, 3, 4, 5, 6, 11, 12, 13, 14, 15
Classification shadows 10
Detection eyes 14, 15, 16, 17
Detection solo 18, 19
Detection worn 11, 12, 13, 14, 15, 16
Segmentation frames 21, 23
Segmentation full 20, 27, 28
Segmentation legs 29, 30, 31
Segmentation lenses 23, 24, 25, 26, 30, 31, 32
Segmentation shadows 21
Segmentation smart 22

Running

To run custom training and testing, it is first advised to familiarize with how Pytorch Lightning works and briefly check its CLI documentation. In particular, take into account what arguments are accepted by the Trainer class and how to customize your own optimizer and scheduler via command line. Prerequisites:

  1. Clone the repository
  2. Install the requirements
  3. Download and preprocess the data

Training

You can run simple training as follows (which is the default):

python scripts/run.py fit --task classification:anyglasses --size medium 

You can customize things like batch-size, num-workers, as well as trainer and checkpoint arguments:

python scripts/run.py fit --batch-size 64 --trainer.max_epochs 300 --checkpoint.dirname ckpt

It is also possible to overwrite default optimizer and scheduler:

python scripts/run.py fit --optimizer Adam --optimizer.lr 1e-3 --lr_scheduler CosineAnnealingLR

Testing

To run testing, specify the trained model and the checkpoint to it:

python scripts/run.py test -t classification:anyglasses -s small --ckpt_path path/to/model.ckpt

Or you can also specify the pth file to pre-load the model with weights:

python scripts/run.py test -t classification:anyglasses -s small -w path/to/weights.pth

If you get UserWarning: No positive samples in targets, true positive value should be meaningless, increase the batch size.

Credits

For references and citation, please see Glasses Detector Credits.

glasses-detector's People

Contributors

dobiasd avatar mantasu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

glasses-detector's Issues

Could you help me?

Hi!
I'm trying to use, I tried first to clone this source code, but I'm using Py 3.11 and I receive a lot of errors, so now I'm using pip install glasses-detector (compiled package).

but I'm not haven error codes that I had when I use src code, but I think it's not finding something that need to download.

Thanks! I'm searching for an glasses detector for long time, hope to can use this.

glasses-detector -i .\fotos\foto_alexandre.jpg --kind anyglasses-classifier
Downloading: "https://github.com/mantasu/glasses-detector/releases/download/eyeglasses_classifier_shufflenet_v2_x0_5.pth" to C:\Users\Alexandre/.cache\torch\hub\checkpoints\eyeglasses_classifier_shufflenet_v2_x0_5.pth
Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Program Files\Python311\Scripts\glasses-detector.exe_main
.py", line 7, in
File "C:\Program Files\Python311\Lib\site-packages\glasses_detector_main
.py", line 175, in main
model = model_cls(base_model=base_model, pretrained=True).to(device).eval()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\site-packages\glasses_detector\classifiers.py", line 195, in init
self.eyeg_classifier = EyeglassesClassifier(base_model[0], pretrained[0])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\site-packages\glasses_detector\classifiers.py", line 73, in init
super().init(base_model, pretrained)
File "C:\Program Files\Python311\Lib\site-packages\glasses_detector\bases\base_classifier.py", line 72, in init
super().init(*args, **kwargs)
File "C:\Program Files\Python311\Lib\site-packages\glasses_detector\bases\base_model.py", line 134, in init
self._init_from_base_model(base_model, pretrained)
File "C:\Program Files\Python311\Lib\site-packages\glasses_detector\bases\base_model.py", line 265, in _init_from_base_model
self._load_weights_from_url(self.kind, self.name, version)
File "C:\Program Files\Python311\Lib\site-packages\glasses_detector\bases\base_model.py", line 280, in _load_weights_from_url
weights = torch.hub.load_state_dict_from_url(url, map_location=device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\site-packages\torch\hub.py", line 766, in load_state_dict_from_url
download_url_to_file(url, cached_file, hash_prefix, progress=progress)
File "C:\Program Files\Python311\Lib\site-packages\torch\hub.py", line 620, in download_url_to_file
u = urlopen(req)
^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 525, in open
response = meth(req, response)
^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 634, in http_response
response = self.parent.error(
^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 563, in error
return self._call_chain(*args)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\urllib\request.py", line 643, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

Error load pretrained weight

Hi, I'm try run predict.py to test your weight, but i got the error:
"RuntimeError: Error(s) in loading state_dict for SunglassesClassifier:
Missing key(s) in state_dict: "base_model.features.0.0.weight", "base_model.features.0.1.weight", ...."

classify sunglasses

Hello author, I really need a model to classify sunglasses (the same task you provided). Can you upload the code of this task earlier. I really appreciate that..Thank you so much

Eyeglass classifier

Is the Eyeglass classifier available at the moment ?
Would it possible to share a single checkpoint for eyeglass classifier.

Thanks in advance!

Video input

Hello, i wonder this can be use with video input?
Really looking forward!

Python Script is not working

I tried running this script

from glasses_detector import SunglassesClassifier

classifier = SunglassesClassifier(model_type="small", pretrained=True).eval()

classifier.predict(
input_path="path/to/dir",
output_path="path/to/output.csv",
label_type="int",
)

but it didn't work

so I fixed it like this.

from glasses_detector import SunglassesClassifier

classifier = SunglassesClassifier(base_model="small", pretrained=True).eval()

classifier.process(
input_path="path/to/dir",
output_path="path/to/output.csv",
label_type="int",
)

Getting HTTP 404 when trying to download models

Hello,

I am having difficulty running the classifier locally. I'm getting a 404 error while using command line as well as in a Python script. This is the URL it is trying to hit:
Downloading: "https://github.com/mantasu/glasses-detector/releases/download/eyeglasses_classifier_efficientnet_b0.pth"

Is anyone else facing the same problem?

Alternatively, where can I find the different models that are used here?

TODO for v1.0.0 (Late February 2024 Update)

Features and Improvements for v1.0.0

Data

  • Update scripts to process dataset-by-dataset instead of task-by-task
  • Find or add more datasets for classification, detection, and segmentation
  • Write processing scripts, update preprocess.py CLI
  • Upload the datasets as a single big one to Kaggle

Architecture

  • Add prediction interface to handle predictions and conversion + saving to different formats
  • Update base model to be extendable with glasses classifier, detector, and segmenter children
  • Change class-based model instantiation to arg-based, create default models for each specification
  • Require task for CLI, add size alias, detect device when None

Features

  • Add detection dataset, tiny detection model, and binary detection wrapper
  • Identify the best hyperparameters and set them as default for each wrapper
  • Train models of different sizes and kinds for different tasks (36 in total)

Documentation

  • Add a demo file and a collab notebook for quick examples
  • Crop only the eye area in the tables, add more example images to showcase features
  • Add the results for each model + their sizes, document training parameters
  • Document data download and preprocessing steps
  • Add type hints and documentation for the new classes/functions
  • Add dataset + model architecture references

Training Dataset

Hi, i want to train with my dataset but i got an error like this. Can you help me? Thanks

image

image

image

Eyeglasses Classifier

Setting up Eyeglasses Classification model

About

Some people asked for a generic glasses classifier (#3, #7, #8), so I released weights for the small eyeglasses classification model. This means you can classify Eyeglasses (sunglasses are excluded) and Anyglasses (sunglasses are included).

Model

The same model architecture was used as for the small sunglasses classifier, i.e., Shufflenet. The datasets used were Face Attributes Grouped, Face Attributes Extra, and Glasses and Coverings. The results on the test set are as follows:

Model type BCE loss $\downarrow$ F1 score $\uparrow$ ROC-AUC score $\uparrow$ Num params $\downarrow$ Model size $\downarrow$
Eyeglasses classifier small 0.1726 0.9217 0.9908 342.82 k 1.34 Mb

Note

Please note that this is just a temporary solution. As noted in #2, the package should be updated in February (non-beta release) with more features, newer models, and better accuracies. Additionally, there will be some architectural changes, thus don't rely on imports or any code used in this beta release - it will be incompatible with v1.0.0.

Setup

Prepare the weights

  1. Download eyeglasses_classifier_shufflenet_v2_x0_5.zip
  2. Unzip and place eyeglasses_classifier_shufflenet_v2_x0_5.pth inside .cache/torch/hub/checkpoints.

Note: your torch hub directory may be elsewhere, you can check that by trying to classify sunglasses. All the models are automatically downloaded there, so just place eyeglasses_classifier_shufflenet_v2_x0_5.pth alongside them.

Run the classifier

  1. You can just run from command-line:

    glasses-detector -i data/demo --kind anyglasses-classifier
  2. OR create a custom script, e.g., classify_if_glasses.py:

    from glasses_detector import AnyglassesClassifier, EyeglassesClassifier
    
    # Constants - change if needed
    CLASSIFIER_CLS = AnyglassesClassifier
    REL_PATH = "data/demo"
    
    # Instantiate and run the classifier (only "small" is available)
    classifier = CLASSIFIER_CLS(base_model="small", pretrained=True).eval()
    classifier.process(REL_PATH)

    and then just run:

    python classify_if_glasses.py

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.