junhwanjang / face_landmark_dnn Goto Github PK

hi，
thanks to your sharing codes, it's very useful.

can you test the speed on predicting the images or video? such like ** fps? or ** ms per image?

Have you ported the model to android, can you provide a link to the android demo?

Do you have a demo of model by any chance? I was having trouble getting the code to run.

OpenCV: Couldn't read video stream from file "samples/IU.avi" Traceback (most recent call last): File "test_video.py", line 82, in <module> single_main() File "test_video.py", line 29, in single_main mark_detector = MarkDetector() File "/face_landmark_dnn/testing/mark_detector.py", line 22, in __init__ self.face_detector = FaceDetector() File "ace_landmark_dnn/testing/mark_detector.py", line 162, in __init__ self.face_net = cv2.dnn.readNetFromCaffe(dnn_proto_text, dnn_model) cv2.error: OpenCV(3.4.4) /Users/travis/build/skvark/opencv-python/opencv/modules/dnn/src/caffe/caffe_io.cpp:1121: error: (-2:Unspecified error) FAILED: fs.is_open(). Can't open "/Users/seerslab/projects/face_landmark_dnn/src/assets/deploy.prototxt" in function 'ReadProtoFromTextFile'

Hi junhwangjang,
Last week I run your model and looked at your code, but I don't know how do you can extract landmarks when an image push to mobilenet network. Please kindly share me some documents.
Thank you very much!

Hi friends,
After I converted keras model to tensorflow model, I run it on tensorflow for mobile. I got an error:

camera/CameraExampleViewController.mm:320] Running model failed:Invalid argument: Shape [-1,112,112,1] has negative dimensions
[[Node: input_1 = Placeholderdtype=DT_FLOAT, shape=[?,112,112,1], _device="/job:localhost/replica:0/task:0/cpu:0"]]

Code on tensorflow for mobile:
const int image_channels = 4;

assert(image_channels >= wanted_input_channels);
tensorflow::Tensor image_tensor(
tensorflow::DT_FLOAT,
tensorflow::TensorShape(
{1, wanted_input_height, wanted_input_width, wanted_input_channels}));
auto image_tensor_mapped = image_tensor.tensor<float, 4>();
tensorflow::uint8 *in = sourceStartAddr;
float *out = image_tensor_mapped.data();
for (int y = 0; y < wanted_input_height; ++y) {
float *out_row = out + (y * wanted_input_width * wanted_input_channels);
for (int x = 0; x < wanted_input_width; ++x) {
const int in_x = (y * image_width) / wanted_input_width;
const int in_y = (x * image_height) / wanted_input_height;
tensorflow::uint8 *in_pixel =
in + (in_y * image_width * image_channels) + (in_x * image_channels);
float *out_pixel = out_row + (x * wanted_input_channels);
for (int c = 0; c < wanted_input_channels; ++c) {
out_pixel[c] = (in_pixel[c] * (1.0/255.0));// - input_mean) / input_std;
}
}
}

CVPixelBufferUnlockBaseAddress(pixelBuffer, unlockFlags);

if (tf_session.get()) {
std::vectortensorflow::Tensor outputs;
tensorflow::Status run_status = tf_session->Run(
{{input_layer_name, image_tensor}}, {output_layer_name}, {}, &outputs);
if (!run_status.ok()) {
LOG(ERROR) << "Running model failed:" << run_status;

Please kindly give me some suggestion. Thanks~

Hello, I am reading your code, but I see your modeling /train_mobilenets.py file,
loss=smoothL1, but smoothL1 is undefined, what is going on?

How do I train on custom data? can you please share the script to create npy files?

i think box_image = cv2.resize(box_image, (output_size, output_size))
landmarks = (landmarks - (x1 - p_x, y1 - p_y)) / (new_size, new_size) should change into

box_image = cv2.resize(box_image, (new_size, new_size))
landmarks = (landmarks - (x1 - p_x, y1 - p_y))
# / (new_size, new_size)

because / (new_size, new_size), the offset of landmarks will be (0-1), it is wrong

Hi,
First, i would like to thank you for creating such a nice project which helps me lot for further research. I am training a model for my custom input images. I converted the trained keras model using tfjs in model.json format.

Can you please tell me how can I further load it in tfjs, or any tutorial link which you have done????

When i am loading it in tfjs, getting error as "Uncaught (in promise) Error: Error when checking : expected input_1 to have shape [null,64,64,1] but got array with shape [1,64,64,3]."

Please help me out in this.

hello author,

which dataset did you use for generating augmentation data. i have ibug images and pts files, the code gives error. what format was your dataset?

thank you

HI, Run the code, thanks for your share, the result on the test-dataset is good, but when i run the same model with the real-image i found from internet, no longer the one from the dataset, the result becomes bad, do you have meet the same issue? how did you fix the bug?

Thanks!

Hi,
When I am using my coreml model (Which I build by mobinet version 2) on ios I cannot detect landmarks point as your Mobilenet_v1.mlcoreml. (I predicted on PC normally )
Please kindly let me know why?
Thanks!

Hello, I have a face data of 98 key points, the image channel is 3 channels, I modified num_outputs=196, batch_szie=16, but I got the following error:

Caused by op 'metrics/mean_absolute_percentage_error/sub', defined at:
  File "train_mobilenets.py", line 463, in <module>
    main()
  File "train_mobilenets.py", line 437, in main
    model.compile(loss=smoothL1, optimizer=keras.optimizers.Adam(lr=1e-3), metrics=['mape'])
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/training.py", line 934, in compile
    handle_metrics(output_metrics)
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/training.py", line 913, in handle_metrics
    mask=masks[i])
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/training.py", line 429, in weighted
    score_array = fn(y_true, y_pred)
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/losses.py", line 22, in mean_absolute_percentage_error
    diff = K.abs((y_true - y_pred) / K.clip(K.abs(y_true),
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 847, in binary_op_wrapper
    return func(x, y, name=name)
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 8009, in sub
    "Sub", x=x, y=y, name=name)
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3414, in create_op
    op_def=op_def)
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1740, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Incompatible shapes: [16,10] vs. [16,196]
         [[Node: metrics/mean_absolute_percentage_error/sub = Sub[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"](_arg_fc_14_target_0_2/_1001, fc_14/BiasAdd)]]
         [[Node: metrics/mean_absolute_percentage_error/Mean_1/_1083 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_10473_metrics/mean_absolute_percentage_error/Mean_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

In addition, I want to use imagenet weights. When I modify weights="imagenet", I get the following error:

raceback (most recent call last):
  File "train_mobilenets.py", line 402, in <module>
    ,include_top=False,weights="imagenet",pooling='max',shallow=True)
  File "train_mobilenets.py", line 349, in MobileNet
    model.load_weights(weights_path)
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/topology.py", line 2656, in load_weights
    f, self.layers, reshape=reshape)
  File "/atlas/home/hzhou/.local/lib/python3.6/site-packages/keras/engine/topology.py", line 3354, in load_weights_from_hdf5_group
    str(len(filtered_layers)) + ' layers.')
ValueError: You are trying to load a weight file containing 54 layers into a model with 34 layers.

May I ask what is going on here? Can you give me some guidance, thank you!

Hi,
When I am using Mobilenet_v1.mlmodel on ios I cannot detect landmarks point, please kindly let me know why?
Thanks!

Hello @junhwanjang, Thank you for the code, I'm on Python 3.5 and Keras 2.2.3 and I'm having this error when running tets_video.py

Traceback (most recent call last):
File "test_video.py", line 4, in
from mark_detector import MarkDetector
File "C:\Users\mwdar\Desktop\face_landmark_dnn-master\testing\mark_detector.py", line 6, in from utils import smoothL1, relu6, DepthwiseConv2D, mask_weights
File "C:\Users\mwdar\Desktop\face_landmark_dnn-master\testing\utils.py", line 11, in
from keras.applications.imagenet_utils import decode_predictions, _obtain_in
put_shape
ImportError: cannot import name '_obtain_input_shape'

I suspect it is a version problem and I tried different Keras versions with no luck.

Thank you.