jeffxtang / yolov2_tf_ios Goto Github PK

Object Detection with YOLOv2 and TensorFlow on iOS

Objective-C 5.90% Objective-C++ 90.97% C++ 3.14%

yolov2_tf_ios's Introduction

What

This repo contains a sample iOS app that runs with a 80-class COCO trained tiny-yolo model and the TensorFlow libraries to do object detection and show the detected results and the bounding boxes. An iOS app based on the repo has been published and available for free download here.

Please check out my blog for the latest info on this app and tutorials on other TensorFlow mobile apps.

How

Download the TensorFlow 1.2 source here;
Build the TensorFlow iOS libraries by following the instructions here. After that, you'll see libprotobuf.a and libprotobuf-lite.a in tensorflow/contrib/makefile/gen/protobuf_ios/lib/, and libtensorflow-core.a in tensorflow/contrib/makefile/gen/lib;
Clone this repo and then move the whole folder yolov2_tf_ios to tensorflow/contrib/ios_examples/ (so the TF libraries built in step 2 will be available to the project) and open the project in Xcode;
Download the OpenCV 3.2.0 iOS pack here and then drag and drop the opencv2.framework to the Xcode project - this is used for drawing bounding boxes but of course you can do so (easily) with the iOS API too. Still I included OpenCV in the project to be ready for other more advanced OpenCV based CV processing;
Run the app in Xcode on simulator or device and tap the button to see results as the screenshot below.

Notes

This repo includes two pre-trained and TF-quantized models (to reduce the model size significantly): quantized-tiny-yolo.pb and quantized-tiny-yolo-voc.pb. The first one was originally trained using the PASCAL VOC dataset for 20-class object detection, and the second one the COCO dataset for 80-class object detection. I created the pb files using Darkflow then ran bazel-bin/tensorflow/tools/quantization/quantize_graph --input=darkflow/built_graph/tiny-yolo.pb --output_node_names=output --output=quantized_tiny-yolo.pb --mode=weights and bazel-bin/tensorflow/tools/quantization/quantize_graph --input=darkflow/built_graph/tiny-yolo-voc.pb --output_node_names=output --output=quantized_tiny-yolo-voc.pb --mode=weights. Both tiny-yolo(-vos).pb and its quantized versions can run successfully on device;
I also tested the yolo.pb and its quantized version - they can run OK on simulator but would crash on actual device.

Credits

The preprocessing and postprocessing of the input image tensor and output tensor in iOS with TensorFlow C++ API was finally made right after many hours of debugging, review of the original Yolo9000 paper, and the help of the following sources:

Darkflow source and README;
The Real-time object detection with YOLO blog;
The Tensorflow Android source code of TF Detect.

yolov2_tf_ios's People

Contributors

Stargazers

Watchers

Forkers

bluehackrano biranchi2018 offbye hradynarski tvkpz 307509256 dongdyang drahfa medredha ifyoung

yolov2_tf_ios's Issues

Quantization tool in tensorflow

You are using quantize_graph tool to quantize the model.
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite
On this website, they are suggesting freeze the graph using .pb and .ckpt, then convert it to .tflite.
I think freeze_graph tool can also generate a quantized model.
Do you know what is the difference?
Where is the .ckpt file for darkflow?

Quantization problem

Do you know the output of your .pb file from quantization is quantized or float?

Protocol buffers version mismatch

I followed the installation steps, but I get a runtime exception when running on the device:

[libprotobuf FATAL /Volumes/build-storage/build/master_iOS-mac/opencv/3rdparty/protobuf/src/google/protobuf/stubs/common.cc:67] This program requires version 3.2.0 of the Protocol Buffer runtime library, but the installed version is 3.1.0. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "/Users/Peter/Downloads/tensorflow-1.2.0/tensorflow/contrib/makefile/gen/proto/tensorflow/core/util/test_log.pb.cc".) libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: This program requires version 3.2.0 of the Protocol Buffer runtime library, but the installed version is 3.1.0. Please update your library. If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library. (Version verification failed in "/Users/Peter/Downloads/tensorflow-1.2.0/tensorflow/contrib/makefile/gen/proto/tensorflow/core/util/test_log.pb.cc".)

I tried building libprotobuf 3.2 from source but then I have a similar issue that says the library was build against 3.1.
Help would be appreciated, could you perhaps share your project?

I can't find the file of quantize_graph

I have no quantization folder inside bazel-bin/tensorflow/tools. However, I have it inside tensorflow-master/tensorflow/tools/quantization. I just removed the bazel-bin and simply put tensorflow/tools/quantization/quantize_graph and it tells me can not find the directory or file.
Do you know why it happen.
How am I able to get a quantization folder inside bazel-bin?

Remove the imagenet_comp_graph_label_strings reference

The xcodeproject references a file that does not exist and is not used within the project.
But this fie is missing which causes a compile error. The reference should probably be removed therefore:

imagenet_comp_graph_label_strings

This program requires version 3.2.0 of the Protocol Buffer runtime library, but the installed version is 3.1.0.

I followed your instructions to compile TensorFlow for iOS and then put your project into the example folder.

When I compile your project now, I get this error: