junhwanjang / face_landmark_dnn Goto Github PK
View Code? Open in Web Editor NEWFace Landmark Detector based on Mobilenet V1
License: MIT License
Face Landmark Detector based on Mobilenet V1
License: MIT License
hi,
thanks to your sharing codes, it's very useful.
can you test the speed on predicting the images or video? such like ** fps? or ** ms per image?
Have you ported the model to android, can you provide a link to the android demo?
Do you have a demo of model by any chance? I was having trouble getting the code to run.
OpenCV: Couldn't read video stream from file "samples/IU.avi" Traceback (most recent call last): File "test_video.py", line 82, in <module> single_main() File "test_video.py", line 29, in single_main mark_detector = MarkDetector() File "/face_landmark_dnn/testing/mark_detector.py", line 22, in __init__ self.face_detector = FaceDetector() File "ace_landmark_dnn/testing/mark_detector.py", line 162, in __init__ self.face_net = cv2.dnn.readNetFromCaffe(dnn_proto_text, dnn_model) cv2.error: OpenCV(3.4.4) /Users/travis/build/skvark/opencv-python/opencv/modules/dnn/src/caffe/caffe_io.cpp:1121: error: (-2:Unspecified error) FAILED: fs.is_open(). Can't open "/Users/seerslab/projects/face_landmark_dnn/src/assets/deploy.prototxt" in function 'ReadProtoFromTextFile'
Hi junhwangjang,
Last week I run your model and looked at your code, but I don't know how do you can extract landmarks when an image push to mobilenet network. Please kindly share me some documents.
Thank you very much!
Code on tensorflow for mobile:
const int image_channels = 4;
assert(image_channels >= wanted_input_channels);
tensorflow::Tensor image_tensor(
tensorflow::DT_FLOAT,
tensorflow::TensorShape(
{1, wanted_input_height, wanted_input_width, wanted_input_channels}));
auto image_tensor_mapped = image_tensor.tensor<float, 4>();
tensorflow::uint8 *in = sourceStartAddr;
float *out = image_tensor_mapped.data();
for (int y = 0; y < wanted_input_height; ++y) {
float *out_row = out + (y * wanted_input_width * wanted_input_channels);
for (int x = 0; x < wanted_input_width; ++x) {
const int in_x = (y * image_width) / wanted_input_width;
const int in_y = (x * image_height) / wanted_input_height;
tensorflow::uint8 *in_pixel =
in + (in_y * image_width * image_channels) + (in_x * image_channels);
float *out_pixel = out_row + (x * wanted_input_channels);
for (int c = 0; c < wanted_input_channels; ++c) {
out_pixel[c] = (in_pixel[c] * (1.0/255.0));// - input_mean) / input_std;
}
}
}
CVPixelBufferUnlockBaseAddress(pixelBuffer, unlockFlags);
if (tf_session.get()) {
std::vectortensorflow::Tensor outputs;
tensorflow::Status run_status = tf_session->Run(
{{input_layer_name, image_tensor}}, {output_layer_name}, {}, &outputs);
if (!run_status.ok()) {
LOG(ERROR) << "Running model failed:" << run_status;
Please kindly give me some suggestion. Thanks~
Hello, I am reading your code, but I see your modeling /train_mobilenets.py
file,
loss=smoothL1
, but smoothL1 is undefined, what is going on?
How do I train on custom data? can you please share the script to create npy files?
i think box_image = cv2.resize(box_image, (output_size, output_size))
landmarks = (landmarks - (x1 - p_x, y1 - p_y)) / (new_size, new_size) should change into
box_image = cv2.resize(box_image, (new_size, new_size))
landmarks = (landmarks - (x1 - p_x, y1 - p_y))
# / (new_size, new_size)
because / (new_size, new_size), the offset of landmarks will be (0-1), it is wrong
Hi,
First, i would like to thank you for creating such a nice project which helps me lot for further research. I am training a model for my custom input images. I converted the trained keras model using tfjs in model.json format.
Can you please tell me how can I further load it in tfjs, or any tutorial link which you have done????
When i am loading it in tfjs, getting error as "Uncaught (in promise) Error: Error when checking : expected input_1 to have shape [null,64,64,1] but got array with shape [1,64,64,3]."
Please help me out in this.
hello author,
which dataset did you use for generating augmentation data. i have ibug images and pts files, the code gives error. what format was your dataset?
thank you
HI, Run the code, thanks for your share, the result on the test-dataset is good, but when i run the same model with the real-image i found from internet, no longer the one from the dataset, the result becomes bad, do you have meet the same issue? how did you fix the bug?
Thanks!
Hi,
When I am using my coreml model (Which I build by mobinet version 2) on ios I cannot detect landmarks point as your Mobilenet_v1.mlcoreml. (I predicted on PC normally )
Please kindly let me know why?
Thanks!
Hello, I have a face data of 98 key points, the image channel is 3 channels, I modified num_outputs=196
, batch_szie=16
, but I got the following error:
Caused by op 'metrics/mean_absolute_percentage_error/sub', defined at:
File "train_mobilenets.py", line 463, in <module>
main()
File "train_mobilenets.py", line 437, in main
model.compile(loss=smoothL1, optimizer=keras.optimizers.Adam(lr=1e-3), metrics=['mape'])
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/training.py", line 934, in compile
handle_metrics(output_metrics)
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/training.py", line 913, in handle_metrics
mask=masks[i])
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/training.py", line 429, in weighted
score_array = fn(y_true, y_pred)
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/losses.py", line 22, in mean_absolute_percentage_error
diff = K.abs((y_true - y_pred) / K.clip(K.abs(y_true),
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 847, in binary_op_wrapper
return func(x, y, name=name)
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 8009, in sub
"Sub", x=x, y=y, name=name)
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3414, in create_op
op_def=op_def)
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1740, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Incompatible shapes: [16,10] vs. [16,196]
[[Node: metrics/mean_absolute_percentage_error/sub = Sub[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"](_arg_fc_14_target_0_2/_1001, fc_14/BiasAdd)]]
[[Node: metrics/mean_absolute_percentage_error/Mean_1/_1083 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_10473_metrics/mean_absolute_percentage_error/Mean_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
In addition, I want to use imagenet weights. When I modify weights="imagenet",
I get the following error:
raceback (most recent call last):
File "train_mobilenets.py", line 402, in <module>
,include_top=False,weights="imagenet",pooling='max',shallow=True)
File "train_mobilenets.py", line 349, in MobileNet
model.load_weights(weights_path)
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/topology.py", line 2656, in load_weights
f, self.layers, reshape=reshape)
File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/topology.py", line 3354, in load_weights_from_hdf5_group
str(len(filtered_layers)) + ' layers.')
ValueError: You are trying to load a weight file containing 54 layers into a model with 34 layers.
May I ask what is going on here? Can you give me some guidance, thank you!
Hi,
When I am using Mobilenet_v1.mlmodel on ios I cannot detect landmarks point, please kindly let me know why?
Thanks!
Hello @junhwanjang, Thank you for the code, I'm on Python 3.5 and Keras 2.2.3 and I'm having this error when running tets_video.py
Traceback (most recent call last):
File "test_video.py", line 4, in
from mark_detector import MarkDetector
File "C:\Users\mwdar\Desktop\face_landmark_dnn-master\testing\mark_detector.py", line 6, in from utils import smoothL1, relu6, DepthwiseConv2D, mask_weights
File "C:\Users\mwdar\Desktop\face_landmark_dnn-master\testing\utils.py", line 11, in
from keras.applications.imagenet_utils import decode_predictions, _obtain_in
put_shape
ImportError: cannot import name '_obtain_input_shape'
I suspect it is a version problem and I tried different Keras versions with no luck.
Thank you.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.