didi / maskdetection Goto Github PK

License: Apache License 2.0

CMake 1.23% C++ 98.77%

maskdetection's Introduction

MaskDetection

What is MaskDetection?

To further help combat the coronavirus, Didi Chuxing decides to open up its face mask detection technology to the public for free. Developed by DiDi AI team, the mask detection technology is based on DFS face detection algorithm and the face attributes recognition algorithm DiDi employs on its platform. The model has overcome several difficulties like complex lighting changes in a day, face pose variety, face scales, etc. It uses weighted loss function and data augmentation methods to deal with different mask types and uneven mask data during the day and the night.

The system can identify non-mask drivers using their uploaded images with 99.5 per cent accuracy, and achieves 98 per cent accuracy during DiDi’s spot inspection with in-vehicle cameras. The model was trained on a dataset of 200,000 faces to ensure its robustness.

This quick detection system can be widely used in travel scenes, with mobile phone photos, surveillance images etc., and is able to work round the clock.

Requirements

google/protobuf
openblas or atlas
opencv
cuda/cudnn(if GPU is used)
Caffe

Installation

Clone the repository. We'll call the directory that you cloned as MASK_ROOT.

git clone [email protected]:didi/maskdetection.git

Build Caffe,see:http://caffe.berkeleyvision.org

Usage

In this part, we assume you are in the directory $MASK_ROOT/

Install requirements (remember the install path to configure CMakeLists.txt)
Configure CMakeLists.txt、 test/CMakeLists.txt、 lib/CMakeLists.txt
Compile

mkdir build
cd build
cmake ..
make

After compilation, executable files will be generated in directory: build/bin
Detect the face bounding box, and then expand the bounding box to a certain proportion to get a detected face.
Run executable files with the detected face to judge if the face is wearing mask.

Tips：

The detected face bounding box is expanded to reduce the effect of detection errors.
Each side of the original face bounding box is expanded by 40% for mask detection. The proportion may be adjusted to the actual scenario/context（e.g., reduced to fit the actual scene in case of a dense crowd).

Model

Our model is pretrained by public ResNet50-caffemodel.
Trained by our collected private data.
Introduce attention mechanism.

License

maskdetection is licensed under the terms of the Apache license. See LICENSE for more information.

Note

DiDi's mask-recognition service is designed to better protect users from public health risks. An attention-learning mechanism is built in the technology to focus on recognizing the existence of a mask while weakening the recognition of other face areas. The service is subject to various sources of error including brightness, posture or partial image capture. We will continue to improve on the accuracy of the technology. Thank you for your support.

maskdetection's People

Contributors

Stargazers

Watchers

maskdetection's Issues

Mask recognize

Unable to Complete the compilation to the executable

[ 70%] Building CXX object lib/CMakeFiles/face_mask.dir/src/util/xml/xmlfunc.cpp.o
[ 80%] Linking CXX shared library libface_mask.so
[ 80%] Built target face_mask
Scanning dependencies of target mask
[ 90%] Building CXX object test/CMakeFiles/mask.dir/main_face_mask.cpp.o
make[2]: *** No rule to make target '../test/../thirdpart/lib/caffe/libcaffe.so', needed by 'bin/mask'. Stop.
make[1]: *** [CMakeFiles/Makefile2:142: test/CMakeFiles/mask.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

this is the error that I'm getting now and I have no clue how to go about it... Someone help Please :) thanks in advance

Problem after converting from caffe to keras in number of layers

I am new to C++. Can you please shed some light on what they are and how I should tweak them to make the repo in working condition?

Also, is it possible to use the .caffemodel and .prototxt files inside model folder for inference in Python?

Update : I used this to convert the .caffemodel and .prototxt file into .hdf5 file in Keras. Now when trying to load resnet50 inbuilt in keras with the computed .hdf5 file, I am getting the following error.

It is telling there are more number of layers in your architecture. Could you please assist me on this?

My code
from keras.applications import resnet
model = resnet.ResNet50(include_top=True, weights='mask_dete1.h5', input_tensor=None, input_shape=(224, 224, 3), pooling=None, classes=2)

Error

ValueError Traceback (most recent call last)
in ()
----> 1 model = resnet.ResNet50(include_top=True, weights='mask_dete1.h5', input_tensor=None, input_shape=(224, 224, 3), pooling=None, classes=2)

6 frames
/usr/local/lib/python3.6/dist-packages/keras/applications/init.py in wrapper(*args, **kwargs)
18 kwargs['models'] = models
19 kwargs['utils'] = utils
---> 20 return base_fun(*args, **kwargs)
21
22 return wrapper

/usr/local/lib/python3.6/dist-packages/keras/applications/resnet.py in ResNet50(*args, **kwargs)
12 @keras_modules_injection
13 def ResNet50(*args, **kwargs):
---> 14 return resnet.ResNet50(*args, **kwargs)
15
16

/usr/local/lib/python3.6/dist-packages/keras_applications/resnet_common.py in ResNet50(include_top, weights, input_tensor, input_shape, pooling, classes, **kwargs)
433 input_tensor, input_shape,
434 pooling, classes,
--> 435 **kwargs)
436
437

/usr/local/lib/python3.6/dist-packages/keras_applications/resnet_common.py in ResNet(stack_fn, preact, use_bias, model_name, include_top, weights, input_tensor, input_shape, pooling, classes, **kwargs)
411 model.load_weights(weights_path)
412 elif weights is not None:
--> 413 model.load_weights(weights)
414
415 return model

/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in load_wrapper(*args, **kwargs)
490 os.remove(tmp_filepath)
491 return res
--> 492 return load_function(*args, **kwargs)
493
494 return load_wrapper

/usr/local/lib/python3.6/dist-packages/keras/engine/network.py in load_weights(self, filepath, by_name, skip_mismatch, reshape)
1228 else:
1229 saving.load_weights_from_hdf5_group(
-> 1230 f, self.layers, reshape=reshape)
1231 if hasattr(f, 'close'):
1232 f.close()

/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in load_weights_from_hdf5_group(f, layers, reshape)
1207 'containing ' + str(len(layer_names)) +
1208 ' layers into a model with ' +
-> 1209 str(len(filtered_layers)) + ' layers.')
1210
1211 # We batch weight value assignments in a single backend call

ValueError: You are trying to load a weight file containing 112 layers into a model with 107 layers.

how to compile for cpu? CMake Error at /usr/share/cmake-3.10/Modules/FindCUDA.cmake:682 (message): Specify CUDA_TOOLKIT_ROOT_DIR Call Stack (most recent call first): lib/CMakeLists.txt:4 (find_package)

I don't have GPU, I am using CPU. When I was following the instructions given in the README to compile.
$ cmake ..
I got the following errors.

-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /usr/share/cmake-3.10/Modules/FindCUDA.cmake:682 (message):
  Specify CUDA_TOOLKIT_ROOT_DIR
Call Stack (most recent call first):
  lib/CMakeLists.txt:4 (find_package)


-- Configuring incomplete, errors occurred!

Can anyone tell me how to fix it?

CopyTrainedLayersFrom undefined reference

/home/wangmenghu/maskdetection/build/lib/libface_mask.so：对‘caffe::Net::CopyTrainedLayersFrom(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)’未定义的引用
/home/wangmenghu/maskdetection/build/lib/libface_mask.so：对‘cv::error(int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, char const*, char const*, int)’未定义的引用
/home/wangmenghu/maskdetection/build/lib/libface_mask.so：对‘cv::Mat::updateContinuityFlag()’未定义的引用
/home/wangmenghu/maskdetection/build/lib/libface_mask.so：对‘cv::imread(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int)’未定义的引用

Compilation on Windows 10 64 Bits

Hi, Im interested to test your code.
I want to know if it is possible to compile the code on Windows 10 64 Bits.
I have Visual Studio Community 2017 and cmake.
But I get this message:

CMake Error at C:/Program Files/CMake/share/cmake-3.17/Modules/FindCUDA.cmake:715 (message):
Specify CUDA_TOOLKIT_ROOT_DIR
Call Stack (most recent call first):
lib/CMakeLists.txt:4 (find_package)

Thanks.

Sergio

Dataset

Please could you upload the dataset?

Importing trained model directly using python opencv

Python: 3.6.9
opencv-python==4.2.0.34

Rather than trying to build this project with so little documentation, I decided to directly import the model in cv2 python but no success so far.

import cv2

proto_path = 'deploy.prototxt'
model_path = 'face_mask.caffemodel'

model = cv2.dnn.readNetFromCaffe(proto_path, model_path)

image_file = 'sample_images/0000.jpg'
img = cv2.imread(image_file)

image_blob = cv2.dnn.blobFromImage(img)

model.setInput(image_blob)

prediction = model.forward()

Error:
cv2.error: OpenCV(4.2.0) /io/opencv/modules/dnn/src/dnn.cpp:3084: error: (-215:Assertion failed) total(os[i]) > 0 in function 'getLayerShapesRecursively'

Did anyone had any success doing this or what else can be done to use the pre-trained model?

link to opencv fails

I build opencv from scratch, and everything seems ok. I was able to build caffe right afterwards.

Now after updating all the CMakefile.txt's, I started building maskdetect. It went all the way thru to
[100%] Linking CXX executable ../bin/mask
but throw a bunch of opencv related errors

[100%] Linking CXX executable ../bin/mask
/home/bruce/maskdetection/build/lib/libface_mask.so: undefined reference to cv::_OutputArray::_OutputArray(cv::Mat&)' /home/bruce/maskdetection/build/lib/libface_mask.so: undefined reference to cv::split(cv::Mat const&, std::vector<cv::Mat, std::allocatorcv::Mat >&)'
/home/bruce/maskdetection/build/lib/libface_mask.so: undefined reference to cv::_InputArray::_InputArray(cv::Mat const&)' /home/bruce/maskdetection/build/lib/libface_mask.so: undefined reference to vtable for cv::_InputArray'
collect2: error: ld returned 1 exit status
test/CMakeFiles/mask.dir/build.make:105: recipe for target 'bin/mask' failed

it appears that it can not find the right opencv object files to link to. in particular _OutputArray is a part of core. But I have that object file spelled out in test/CMakefile.txt

TARGET_LINK_LIBRARIES(mask
/usr/local/lib/libopencv_core.so

and I checked many time to make sure that path and creation date are all correct.

Why does the linker keep complaining about this?

This can support opencv directly infer ?

Not able to biuld in google clab "/content/drive/My Drive/Colab Notebooks/ai/mask_detection/maskdetection/lib/include/FaceMaskDetector.hpp:9:10: fatal error: caffe/caffe.hpp: No such file or directory #include "caffe/caffe.hpp" compilation terminated."

I was able to successfully run cmake
!cmake ..

-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found CUDA: /usr/local/cuda (found version "10.1") 
-- Configuring done
-- Generating done
-- Build files have been written to: /content/drive/My Drive/Colab Notebooks/ai/mask_detection/maskdetection/build

but when I did make
!make
I got the following error

Scanning dependencies of target face_mask
[-10%] Building CXX object lib/CMakeFiles/face_mask.dir/src/FaceMaskDetector.cpp.o
In file included from /content/drive/My Drive/Colab Notebooks/ai/mask_detection/maskdetection/lib/src/FaceMaskDetector.cpp:1:0:
/content/drive/My Drive/Colab Notebooks/ai/mask_detection/maskdetection/lib/include/FaceMaskDetector.hpp:9:10: fatal error: caffe/caffe.hpp: No such file or directory
 #include "caffe/caffe.hpp"
          ^~~~~~~~~~~~~~~~~
compilation terminated.
lib/CMakeFiles/face_mask.dir/build.make:62: recipe for target 'lib/CMakeFiles/face_mask.dir/src/FaceMaskDetector.cpp.o' failed
make[2]: *** [lib/CMakeFiles/face_mask.dir/src/FaceMaskDetector.cpp.o] Error 1
CMakeFiles/Makefile2:90: recipe for target 'lib/CMakeFiles/face_mask.dir/all' failed
make[1]: *** [lib/CMakeFiles/face_mask.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

by the way caffe-gpu is installed via sudo apt install caffe-cuda