pinto0309 / pinto_model_zoo Goto Github PK

A repository for storing models that have been inter-converted between various frameworks. Supported frameworks are TensorFlow, PyTorch, ONNX, OpenVINO, TFJS, TFTRT, TensorFlowLite (Float32/16/INT8), EdgeTPU, CoreML.

Home Page: https://qiita.com/PINTO

License: MIT License

Shell 15.55% Python 83.88% C++ 0.24% Dockerfile 0.02% Jupyter Notebook 0.31%

tensorflow tensorflow-lite openvino edgetpu mediapipe coreml tensorflowjs tf-trt onnx pytorch

pinto_model_zoo's Introduction

PINTO_model_zoo

Please read the contents of the LICENSE file located directly under each folder before using the model. My model conversion scripts are released under the MIT license, but the license of the source model itself is subject to the license of the provider repository.

Contributors

Made with contrib.rocks.

TensorFlow Lite, OpenVINO, CoreML, TensorFlow.js, TF-TRT, MediaPipe, ONNX [.tflite, .h5, .pb, saved_model, tfjs, tftrt, mlmodel, .xml/.bin, .onnx]

I have been working on quantization of various models as a hobby, but I have skipped the work of making sample code to check the operation because it takes a lot of time. I welcome a pull request from volunteers to provide sample code. 😄

[Note Jan 05, 2020] Currently, the MobileNetV3 backbone model and the Full Integer Quantization model do not return correctly.

[Note Jan 08, 2020] If you want the best performance with RaspberryPi4/3, install Ubuntu 19.10 aarch64 (64bit) instead of Raspbian armv7l (32bit). The official Tensorflow Lite is performance tuned for aarch64. On aarch64 OS, performance is about 4 times higher than on armv7l OS.

My article

List of pre-quantized models

* WQ = Weight Quantization ** OV = OpenVINO IR *** CM = CoreML **** DQ = Dynamic Range Quantization

1. Image Classification

No.	Model Name	Link	FP32	FP16	INT8	DQ	TPU	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
004	Efficientnet	■■■	⚫		⚫			⚫
010	Mobilenetv3	■■■	⚫		⚫			⚫
011	Mobilenetv2	■■■	⚫		⚫	⚫		⚫
016	Efficientnet-lite	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫
070	age-gender-recognition	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
083	Person_Reidentification	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	248,277,286,287,288,300
087	DeepSort	■■■	⚫	⚫	⚫			⚫	⚫	⚫		⚫	⚫
124	person-attributes-recognition-crossroad-0230	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
125	person-attributes-recognition-crossroad-0234	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
126	person-attributes-recognition-crossroad-0238	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
175	face-recognition-resnet100-arcface-onnx	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	RGB/BGR,112x112,[1,512]
187	vehicle-attributes-recognition-barrier-0039	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	72x72
188	vehicle-attributes-recognition-barrier-0042	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	72x72
191	anti-spoof-mn3	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	128x128
192	open-closed-eye-0001	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	32x32
194	face_recognizer_fast	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	112x112
195	person_reid_youtu	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	256x128, ReID
199	NSFW	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	224x224
244	FINNger	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	96x96
256	SFace	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	112x112
257	PiCANet	■■■							⚫				⚫	BDDA,SAGE/224x224
259	Emotion_FERPlus	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	64x64
290	AdaFace	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	112x112
317	MobileOne	■■■											⚫	224x224
346	facial_expression_recognition_mobilefacenet	■■■	⚫	⚫	⚫	⚫		⚫					⚫	112x112
379	PP-LCNetV2	■■■	⚫	⚫									⚫	224x224
429	OSNet	■■■											⚫	256x128, ReID
430	FastReID	■■■											⚫	384x128, ReID
431	NITEC	■■■											⚫	224x224, Gaze Estimation
432	face-reidentification-retail-0095	■■■	⚫	⚫									⚫	128x128, FaceReID

2. 2D Object Detection

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
002	Mobilenetv3-SSD	■■■	⚫		⚫			⚫		⚫
006	Mobilenetv2-SSDlite	■■■	⚫		⚫	⚫		⚫
008	Mask_RCNN_Inceptionv2	■■■	⚫	⚫				⚫
018	EfficientDet	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
023	Yolov3-nano	■■■	⚫	⚫				⚫					⚫
024	Yolov3-lite	■■■	⚫	⚫	⚫			⚫	⚫				⚫
031	Yolov4	■■■	⚫	⚫	⚫	⚫		⚫					⚫
034	SSD_Mobilenetv2_mnasfpn	■■■	⚫	⚫	⚫	⚫		⚫					⚫
038	SSDlite_MobileDet_edgetpu	■■■	⚫	⚫	⚫	⚫		⚫			⚫	⚫	⚫
039	SSDlite_MobileDet_cpu	■■■	⚫	⚫	⚫			⚫
042	Centernet	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
045	SSD_Mobilenetv2_oid_v4	■■■	⚫	⚫	⚫	⚫		⚫			⚫
046	Yolov4-tiny	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
047	SpineNetMB_49	■■■	⚫	⚫	⚫			⚫					⚫	Mobile RetinaNet
051	East_Text_Detection	■■■	⚫	⚫	⚫	⚫		⚫	⚫		⚫	⚫	⚫
054	KNIFT	■■■	⚫	⚫	⚫			⚫	⚫				⚫	MediaPipe
056	TextBoxes++ with dense blocks, separable convolution and Focal Loss	■■■	⚫	⚫	⚫			⚫	⚫	⚫		⚫	⚫
058	keras-retinanet	■■■	⚫	⚫				⚫	⚫		⚫	⚫	⚫	resnet50_coco_best_v2.1.0.h5,320x320
059	Yolov5	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
072	NanoDet	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	issue #274
073	RetinaNet	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
074	Yolact	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
085	Yolact_Edge	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	21/10/05 new MobileNetV2(550x550)
089	DETR	■■■	⚫	⚫	⚫			⚫					⚫	256x256
103	EfficientDet_lite	■■■	⚫	⚫	⚫	⚫		⚫	⚫		⚫	⚫	⚫	lite0,lite1,lite2,lite3,lite4
116	DroNet	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	DroNet,DroNetV3
123	YOLOR	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫		⚫	ssss_s2d/320x320,640x640,960x960,1280x1280
132	YOLOX	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	nano,tiny,s,m,l,x/256x320,320x320,416x416,480x640,544x960,736x1280,1088x1920
143	RAPiD	■■■	⚫	⚫	⚫			⚫	⚫				⚫	Fisheye, cepdof/habbof/mw_r, 608x608/1024x1024
145	text_detection_db	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	480x640
151	object_detection_mobile_object_localizer	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	192x192
169	spaghettinet_edgetpu	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	⚫	⚫	320x320,S/M/L
174	PP-PicoDet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	S/M/L,320x320/416x416/640x640
178	vehicle-detection-0200	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	256x256,PriorBoxClustered->ndarray(0.npy)
179	person-detection-0202	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	512x512,PriorBoxClustered->ndarray(0.npy)
180	YOLOv5-Lite	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	S:640x640,TensorRT rtx3070 Float16
183	pedestrian-detection-adas-0002	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	384x672,PriorBox->ndarray(0.npy)
184	pedestrian-and-vehicle-detector-adas-0001	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	384x672,PriorBox->ndarray(0.npy)
185	person-vehicle-bike-detection-crossroad-0078	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	1024x1024,PriorBoxClustered->ndarray(0.npy)
186	person-vehicle-bike-detection-crossroad-1016	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	512x512,PriorBoxClustered->ndarray(0.npy)
189	vehicle-license-plate-detection-barrier-0106	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	300x300,PriorBoxClustered->ndarray(0.npy)
190	person-detection-asl-0001	■■■	⚫	⚫	⚫		⚫	⚫	⚫				⚫	320x320
197	yolact-resnet50-fpn	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	RGB,550x550
198	YOLOF	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	BGR/RGB,608x608
221	YOLACT-PyTorch	■■■	⚫	⚫	⚫		⚫	⚫	⚫		⚫	⚫	⚫	180x320,240x320,320x480,480x640,544x544,720x1280
226	CascadeTableNet	■■■	⚫	⚫			⚫	⚫	⚫				⚫	General,320x320 only
262	ByteTrack	■■■											⚫	YOLOX/nano,tiny,s,m,l,x,mot17,ablation/128x320,192x320,192x448,192x640,256x320,256x448,256x640,384x640,512x1280,736x1280
264	object_localization_network	■■■											⚫	180x320,240x320,270x480,360x480,360x480,360x640,480x640,720x1280
307	YOLOv7	■■■	⚫	⚫	⚫		⚫	⚫	⚫		⚫		⚫	YOLOv7,YOLOv7-tiny
308	FastestDet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	180x320,256x320,320x480,352x352,352x640,480x640,736x1280
329	YOLOX-PAI	■■■											⚫
331	YOLOv5L6_Ball	■■■											⚫
332	CrowdDet	■■■											⚫
334	DAMO-YOLO	■■■											⚫
336	PP-YOLOE-Plus	■■■											⚫
337	FreeYOLO	■■■											⚫
341	YOLOv6	■■■	⚫	⚫	⚫		⚫	⚫					⚫
356	EdgeYOLO	■■■											⚫
376	RT-DETR	■■■											⚫	ResNet50,ResNet101,HgNetv2-L,HgNetv2-X
386	naruto_handsign_detection	■■■	⚫	⚫	⚫		⚫						⚫
422	Gold-YOLO-Head-Hand	■■■											⚫	Head,Hand
424	Gold-YOLO-Body	■■■											⚫	Body
425	Gold-YOLO-Body-Head-Hand	■■■											⚫	Body,Head,Hand
426	YOLOX-Body-Head-Hand	■■■	⚫	⚫	⚫								⚫	Body,Head,Hand, tflite float16 XNNPACK boost (ARMv8.2)
434	YOLOX-Body-Head-Hand-Face	■■■											⚫	Body,Head,Hand,Face
441	YOLOX-Body-Head-Hand-Face-Dist	■■■											⚫	Body,Head,Hand,Face,Complex Distorted
442	YOLOX-Body-Head-Face-HandLR-Dist	■■■											⚫	Body,Head,Hands,Left-Hand,Right-Hand,Face,Complex Distorted
444	YOLOX-Foot-Dist	■■■											⚫	Foot,Complex Distorted
445	YOLOX-Body-Head-Face-HandLR-Foot-Dist	■■■											⚫	Body,Head,Face,Hands,Left-Hand,Right-Hand,Foot,Complex Distorted
446	YOLOX-Body-With-Wheelchair	■■■											⚫	Body with WheelChair
447	YOLOX-Wholebody-with-Wheelchair	■■■											⚫	Wholebody with WheelChair

3. 3D Object Detection

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
036	Objectron	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	MediaPipe/camera,chair,chair_1stage,cup,sneakers,sneakers_1stage,ssd_mobilenetv2_oidv4_fp16
063	3D BoundingBox estimation for autonomous driving	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	YouTube
107	SFA3D	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
263	EgoNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫
321	DID-M3D	■■■											⚫
363	YOLO-6D-Pose	■■■	⚫								⚫		⚫	Texas Instruments ver, PINTO Special ver

4. 2D/3D Face Detection

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
025	Head_Pose_Estimation	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
030	BlazeFace	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	MediaPipe
032	FaceMesh	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	MediaPipe
040	DSFD_vgg	■■■	⚫	⚫				⚫
041	DBFace	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	MobileNetV2/V3, 320x320,480x640,640x960,800x1280
043	Face_Landmark	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
049	Iris_Landmark	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	MediaPipe
095	CenterFace	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
096	RetinaFace	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
106	WHENet	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	Real-time Fine-Grained Estimation for Wide Range Head Pose
129	SCRFD	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	All types
130	YOLOv5_Face	■■■	⚫	⚫				⚫	⚫	⚫	⚫		⚫	yolov5n_0.5,yolov5n_face,yolov5s_face/256x320,480x640,736x1280
134	head-pose-estimation-adas-0001	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	60x60
144	YuNet	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	120x160
227	face-detection-adas-0001	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	384x672,PriorBox->ndarray(0.npy)
250	Face-Mask-Detection	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	PriorBox->ndarray(0.npy)
282	face_landmark_with_attention	■■■	⚫	⚫	⚫		⚫	⚫	⚫		⚫		⚫	MediaPipe,192x192
289	face-detection-0100	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	256x256,PriorBoxClustered->ndarray(0.npy)
293	Lightweight-Head-Pose-Estimation	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	HeadPose, 224x224
300	6DRepNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫		⚫		⚫	6D HeadPose, 224x224
301	YOLOv4_Face	■■■											⚫	480x640
302	SLPT	■■■											⚫	decoder=6/12,256x256
303	FAN	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	Face Alignment,128x128/256x256
304	SynergyNet	■■■											⚫	6D HeadPose,224x224
305	DMHead	■■■											⚫	6D HeadPose,Multi-Model-Fused,224x224,PINTO's custom models
311	HHP-Net	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	6D HeadPose,No-LICENSE
319	ACR-Loss	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	Face Alignment
322	YOLOv7_Head	■■■											⚫	PINTO's custom models
345	YOLOv8	■■■											⚫
383	DirectMHP	■■■	⚫	⚫									⚫
387	YuNetV2	■■■	⚫	⚫	⚫		⚫						⚫	640x640
390	BlendshapeV2	■■■	⚫	⚫									⚫	1x146x2,Nx146x2,MediaPipe
399	RetinaFace_MobileNetv2	■■■											⚫
410	FaceMeshV2	■■■	⚫	⚫	⚫		⚫		⚫				⚫	MediaPipe
414	STAR	■■■	⚫	⚫	⚫		⚫		⚫				⚫
421	Gold-YOLO-Head	■■■											⚫	Head (not Face)
423	6DRepNet360	■■■											⚫	6D HeadPose, FullRange, 224x224
433	FaceBoxes.PyTorch	■■■											⚫	2D Face
435	MobileFaceNet	■■■											⚫	Face Alignment,112x112
436	Peppa_Pig_Face_Landmark	■■■											⚫	Face Alignment,128x128,256x256
437	PIPNet	■■■											⚫	Face Alignment,256x256
443	Opal23_HeadPose	■■■											⚫	6D HeadPose, FullRange, 128x128

5. 2D/3D Hand Detection

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
027	Minimal-Hand	■■■	⚫	⚫	⚫			⚫		⚫	⚫	⚫	⚫
033	Hand_Detection_and_Tracking	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	MediaPipe
094	hand_recrop	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	MediaPipe
403	trt_pose_hand	■■■											⚫	2D
420	Gold-YOLO-Hand	■■■											⚫	2D
438	PeCLR	■■■											⚫	2D+3D

6. 2D/3D Human/Animal Pose Estimation

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
003	Posenet	■■■	⚫	⚫	⚫	⚫		⚫	⚫
007	Mobilenetv2_Pose_Estimation	■■■	⚫	⚫	⚫	⚫		⚫	⚫		⚫	⚫
029	Human_Pose_Estimation_3D	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	RGB,180x320,240x320,360x640,480x640,720x1280
053	BlazePose	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	MediaPipe
065	ThreeDPoseUnityBarracuda	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	YouTube
080	tf_pose_estimation	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
084	EfficientPose	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	SinglePose
088	Mobilenetv3_Pose_Estimation	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
115	MoveNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	lightning,thunder
137	MoveNet_MultiPose	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	lightning,192x192,192x256,256x256,256x320,320x320,480x640,720x1280,1280x1920
156	MobileHumanPose	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	3D
157	3DMPPE_POSENET	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	3D,192x192/256x256/320x320/416x416/480x640/512x512
265	PoseAug	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	2D->3D/GCN,MLP,STGCN,VideoPose/Nx16x2
268	Lite-HRNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	COCO,MPII/Top-Down
269	Higher-HRNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	192x320,256x320,320x480,384x640,480x640,512x512,576x960,736x1280/Bottom-Up
271	HRNet	■■■											⚫	COCO,MPII/Top-Down
333	E2Pose	■■■	⚫	⚫	⚫		⚫						⚫	COCO/CrowdPose,End-to-End
350	P-STMO	■■■											⚫	2D->3D,in_the_wild
355	MHFormer	■■■	⚫	⚫									⚫	2D->3D
365	HTNet	■■■	⚫	⚫									⚫	2D->3D
392	STCFormer	■■■	⚫	⚫									⚫	2D->3D
393	RTMPose_WholeBody	■■■	⚫	⚫	⚫		⚫						⚫	2D
394	RTMPose_Animal	■■■	⚫	⚫	⚫		⚫						⚫	2D
402	trt_pose	■■■											⚫	2D
412	pytorch_cpn	■■■	⚫	⚫									⚫	2D
427	RTMPose_Hand	■■■											⚫	2D
440	ViTPose	■■■											⚫	2D

7. Depth Estimation from Monocular/Stereo Images

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
009	Multi-Scale Local Planar Guidance for Monocular Depth Estimation	■■■	⚫
014	tf-monodepth2	■■■	⚫	⚫	⚫			⚫	⚫	⚫		⚫	⚫
028	struct2depth	■■■	⚫	⚫				⚫					⚫
064	Dense Depth	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
066	Footprints	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
067	MiDaS	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
081	MiDaS v2	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
135	CoEx	■■■							⚫				⚫	WIP, onnx/OpenVINO only
142	HITNET	■■■	⚫	⚫				⚫					⚫	WIP issue1,issue2,flyingthings_finalpass_xl/eth3d/middlebury_d400,120x160/240x320/256x256/480x640/720x1280
146	FastDepth	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	128x160,224x224,256x256,256x320,320x320,480x640,512x512,768x1280
147	PackNet-SfM	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	ddad/kitti,Convert all ResNet18 backbones only
148	LapDepth	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	kitti/nyu,192x320/256x320/368x640/480x640/720x1280
149	depth_estimation	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	nyu,180x320/240x320/360x640/480x640/720x1280
150	MobileStereoNet	■■■												WIP. Conversion script only.
153	MegaDepth	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	192x256,384x512
158	HR-Depth	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫
159	EPCDepth	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫
160	msg_chn_wacv20	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	192x320,240x320,256x256,352x480,368x480,368x640,480x640,720x1280,1280x1920
162	PyDNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫
164	MADNet	■■■	⚫	⚫				⚫	⚫	⚫	⚫	⚫	⚫	Real-time-self-adaptive-deep-stereo (perform only inference mode, no-backprop, kitti)
165	RealtimeStereo	■■■	⚫	⚫			⚫	⚫	⚫		⚫	⚫	⚫	180x320,216x384,240x320,270x480,360x480,360x640,480x640,720x1280
166	Insta-DM	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	192x320,256x320,256x832,384x640,480x640,736x1280
167	DPT	■■■	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	⚫	dpt-hybrid,480x640,ViT,ONNX 96x128/256x320/384x480/480x640
173	MVDepthNet	■■■							⚫				⚫	256x320
202	stereoDNN	■■■	⚫	⚫			⚫	⚫	⚫	⚫		⚫	⚫	NVSmall_321x1025,NVTiny_161x513,ResNet18_321x1025,ResNet18_2d_257x513
203	SRHNet	■■■											⚫	finetune2_kitti/sceneflow,maxdisp192,320x480/480x640
210	SC_Depth_pl	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	kitti/nyu,320x320,320x480,480x640,640x800
211	Lac-GwcNet	■■■											⚫	kitti,240x320,320x480,480x640,720x1280
219	StereoNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫			⚫	⚫	Left/180x320,240x320,320x480,360x640,480x640
235	W-Stereo-Disp	■■■											⚫	Kitti,Sceneflow/320x480,384x576,480x640
236	A-TVSNet	■■■	⚫	⚫			⚫	⚫					⚫	Stereo only/192x320,256x320,320x480,480x640
239	CasStereoNet	■■■											⚫	Stereo KITTI only/256x320,384x480,480x640,736x1280
245	GLPDepth	■■■							⚫				⚫	Kitti,NYU/192x320,320x480,384x640,480x640,736x1280,non-commercial use only
258	TinyHITNet	■■■							⚫				⚫	180x320,240x320,300x400,360x640,384x512,480x640,720x960,720x1280
266	ACVNet	■■■							⚫				⚫	sceneflow,kitti/240x320,320x480,384x640,480x640,544x960,720x1280
280	GASDA	■■■											⚫	No-LICENSE
284	CREStereo	■■■											⚫	ITER2,ITER5,ITER10,ITER20/240x320,320x480,360x640,480x640,480x640,720x1280
292	Graft-PSMNet	■■■											⚫	192x320,240x320,320x480,368x640,480x640,720x1280
294	FSRE-Depth	■■■							⚫				⚫	192x320,256x320,320x480,368x640,480x640,736x1280
296	MGNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫		⚫		⚫	240x320,360x480,360x640,360x1280,480x640,720x1280
312	NeWCRFs	■■■											⚫	384x384,384x576,384x768,384x960,576x768,768x1344
313	PyDNet2	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	Mono-Depth
327	EMDC	■■■											⚫	RGB+SarseDepth
338	Fast-ACVNet	■■■											⚫	Stereo/grid_sample opset=16,no_grid_sample opset=11
358	CGI-Stereo	■■■	⚫	⚫									⚫	Stereo
362	ZoeDepth	■■■											⚫	Mono-Depth
364	IGEV	■■■											⚫	Stereo
371	Lite-Mono	■■■											⚫	Mono
384	TCMonoDepth	■■■											⚫	Mono
397	MiDaSv3.1	■■■											⚫	Mono
415	High-frequency-Stereo-Matching-Network	■■■											⚫	Stereo
439	Depth-Anything	■■■											⚫	Mono

8. Semantic Segmentation

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
001	deeplabv3	■■■	⚫					⚫
015	Faster-Grad-CAM	■■■	⚫		⚫			⚫
020	EdgeTPU-Deeplab	■■■	⚫	⚫	⚫			⚫
021	EdgeTPU-Deeplab-slim	■■■	⚫	⚫	⚫			⚫
026	Mobile-Deeplabv3-plus	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
035	BodyPix	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	MediaPipe,MobileNet0.50/0.75/1.00,ResNet50
057	BiSeNetV2	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
060	Hair Segmentation	■■■	⚫					⚫			⚫	⚫	⚫	WIP,MediaPipe
061	U^2-Net	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
069	ENet	■■■	⚫	⚫				⚫			⚫	⚫	⚫	Cityscapes,512x1024
075	ERFNet	■■■	⚫	⚫	⚫	⚫		⚫	⚫		⚫	⚫	⚫	Cityscapes,256x512,384x786,512x1024
078	MODNet	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	128x128,192x192,256x256,512x512
082	MediaPipe_Meet_Segmentation	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	MediaPipe,128x128,144x256,96x160
104	DeeplabV3-plus	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫		⚫	cityscapes,200x400,400x800,800x1600
109	Selfie_Segmentation	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	256x256
136	road-segmentation-adas-0001	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
138	BackgroundMattingV2	■■■	⚫	⚫				⚫	⚫				⚫	720x1280,2160x4096
181	models_edgetpu_checkpoint_and_tflite_vision_segmentation-edgetpu_tflite_default_argmax	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫
182	models_edgetpu_checkpoint_and_tflite_vision_segmentation-edgetpu_tflite_fused_argmax	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫
196	human_segmentation_pphumanseg	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫
201	CityscapesSOTA	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	180x320,240x320,360x640,480x640,720x1280
206	Matting	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	PaddleSeg/modnet_mobilenetv2,modnet_hrnet_w18,modnet_resnet50_vd/256x256,384x384,512x512,640x640
228	Fast-SCNN	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	192x384,384x384,384x576,576x576,576x768,768x1344
238	SUIM-Net	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫			⚫	RSB,VGG/240x320,256x320,320x480,360x640,384x480,384x640,480x640,720x1280
242	RobustVideoMatting	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	Mbnv3,ResNet50/192x320,240x320,320x480,384x640,480x640,720x1280,1088x1920,2160x3840
246	SqueezeSegV3	■■■	⚫	⚫			⚫	⚫	⚫	⚫	⚫		⚫	21,53/180x320,240x320,320x480,360x640,480x640,720x1280
267	LIOT	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	180x320,240x320,320x480,360x640,480x640,540x960,720x1280,1080x1920
287	Topformer	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	Tiny,Small,Base/448x448,512x512
295	SparseInst	■■■							⚫				⚫	r50_giam_aug/192x384,384x384,384x576,384x768,576x576,576x768,768x1344
299	DGNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫
313	IS-Net	■■■											⚫	180x320,240x320,320x480,360x640,480x640,720x1280,1080x1920,1080x2048,2160x4096,N-batch,Dynamic-HeightxWidth
335	PIDNet	■■■											⚫	Cityscapes,CamVid/Dynamic-HeightxWidth
343	PP-MattingV2	■■■	⚫	⚫									⚫	HumanSeg
347	RGBX_Semantic_Segmentation	■■■											⚫
369	Segment_Anything	■■■											⚫
380	Skin-Clothes-Hair-Segmentation-using-SMP	■■■	⚫	⚫									⚫
391	MagicTouch	■■■	⚫	⚫			⚫						⚫	MediaPipe
405	Ear_Segmentation	■■■											⚫	Ear
417	PopNet	■■■											⚫	Saliency

9. Anomaly Detection

No.	Model Name	Link	FP32	FP16	INT8	TPU	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
005	One_Class_Anomaly_Detection	■■■	⚫		⚫	⚫	⚫
099	Efficientnet_Anomaly_Detection_Segmentation	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫

10. Artistic

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
017	Artistic-Style-Transfer	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫		⚫
019	White-box-Cartoonization	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
037	First_Neural_Style_Transfer	■■■	⚫					⚫					⚫
044	Selfie2Anime	■■■	⚫	⚫	⚫			⚫	⚫	⚫			⚫
050	AnimeGANv2	■■■	⚫	⚫	⚫			⚫	⚫	⚫		⚫	⚫
062	Facial Cartoonization	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
068	Colorful_Image_Colorization	■■■	⚫	⚫				⚫	⚫	⚫	⚫		⚫	experimental
101	arbitrary_image_stylization	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	magenta
113	Anime2Sketch	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
161	EigenGAN-Tensorflow	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	Anime,CelebA
193	CoCosNet	■■■	⚫	⚫			⚫	⚫	⚫	⚫	⚫		⚫	RGB,256x256

11. Super Resolution

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
012	Fast_Accurate_and_Lightweight_Super-Resolution	■■■	⚫		⚫			⚫
022	Learning_to_See_Moving_Objects_in_the_Dark	■■■	⚫	⚫				⚫					⚫
071	Noise2Noise	■■■	⚫	⚫	⚫	⚫		⚫					⚫	srresnet/clear only
076	Deep_White_Balance	■■■	⚫	⚫				⚫	⚫	⚫	⚫	⚫	⚫
077	ESRGAN	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	50x50->x4, 100x100->x4
079	MIRNet	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image Enhancement/40x40,80x80,120x120,120x160,120x320,120x480,120x640,120x1280,180x480,180x640,180x1280,180x320,240x320,240x480,360x480,360x640,480x640,720x1280
086	Defocus Deblurring Using Dual-Pixel	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
090	Ghost-free_Shadow_Removal	■■■	⚫	⚫	⚫			⚫	⚫	⚫			⚫	256x256
111	SRN-Deblur	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	240x320,480x640,720x1280,1024x1280
112	DeblurGANv2	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	inception/mobilenetv2:256x256,320x320,480x640,736x1280,1024x1280
114	Two-branch-dehazing	■■■	⚫	⚫	⚫			⚫	⚫		⚫	⚫	⚫	240x320,480x640,720x1280
133	Real-ESRGAN	■■■	⚫	⚫				⚫	⚫	⚫	⚫	⚫	⚫	16x16,32x32,64x64,128x128,240x320,256x256,320x320,480x640
152	DeepLPF	■■■	⚫	⚫				⚫	⚫				⚫
170	Learning-to-See-in-the-Dark	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	sony/fuji, 240x320,360x480,360x640,480x640
171	Fast-SRGAN	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫			⚫	120x160,128x128,240x320,256x256,480x640,512x512
172	Real-Time-Super-Resolution	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	64x64,96x96,128x128,256x256,240x320,480x640
176	StableLLVE	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image/Video Enhancement,180x240,240x320,360x640,480x640,720x1280
200	AGLLNet	■■■	⚫	⚫				⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image/Video Enhancement,256x256,256x384,384x512,512x640,768x768,768x1280
204	HINet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	DeBlur,DeNoise,DeRain/256x320,320x480,480x640
205	MBLLEN	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image/Video Enhancement,180x320,240x320,360x640,480x640,720x1280
207	GLADNet	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image/Video Enhancement,180x320,240x320,360x640,480x640,720x1280,No-LICENSE
208	SAPNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	DeRain,180x320,240x320,360x640,480x640,720x1280
209	MSBDN-DFF	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	Dehazing,192x320,240x320,320x480,384x640,480x640,720x1280,No-LICENSE
212	GFN	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	DeBlur+SuperResolution,x4/64x64,96x96,128x128,192x192,240x320,256x256,480x640,720x1280
213	TBEFN	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image Enhancement/180x320,240x320,320x480,360x640,480x640,720x1280
214	EnlightenGAN	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image Enhancement/192x320,240x320,320x480,368x640,480x640,720x1280
215	AOD-Net	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	DeHazing/180x320,240x320,320x480,360x640,480x640,720x1280
216	Zero-DCE-TF	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image Enhancement/180x320,240x320,320x480,360x640,480x640,720x1280
217	RUAS	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image Enhancement/180x320,240x320,320x480,360x640,480x640,720x1280,No-LICENSE
218	DSLR	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	Low-light Image Enhancement/256x256,256x384,256x512,384x640,512x640,768x1280
220	HEP	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image Enhancement/180x320,240x320,320x480,360x640,480x640
222	LFT	■■■											⚫	Transformer/2x,4x/65x65
223	DA_dahazing	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	DeHazing/192x320,240x320,320x480,360x640,480x640,720x1280,No-LICENSE
224	Y-net	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	DeHazing/192x320,240x320,320x480,384x640,480x640,720x1280
225	DRBL	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	DeHazing/192x320,240x320,320x480,384x640,480x640,720x1280
230	Single-Image-Desnowing-HDCWNet	■■■	⚫	⚫			⚫	⚫			⚫		⚫	DeSnowing/512x672
231	DRBL	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image Enhancement/180x320,240x320,320x480,360x640,480x640,720x1280,No-LICENSE
232	MIMO-UNet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	DeBlur/180x320,240x320,320x480,360x640,480x640,720x1280,No-LICENSE
234	FBCNN	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	DeNoise/180x320,240x320,320x480,360x640,480x640,720x1280
240	BSRGAN	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	x2,x4/64x64,96x96,128x128,160x160,180x320,240x320,No-LICENSE
241	SCL-LLE	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image Enhancement/180x320,240x320,320x480,480x640,720x1280,No-LICENSE
243	Zero-DCE-improved	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫		⚫	Low-light Image Enhancement/180x320,240x320,320x480,360x640,480x640,720x1280,academic use only
249	Real-CUGAN	■■■								⚫			⚫	2x,3x,4x/64x64,96x96,128x128,120x160,160x160,180x320,240x320
251	AU-GAN	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	Low-light Image Enhancement/128x256,240x320,240x640,256x512,480x640,512x1024,720x1280
253	TransWeather	■■■							⚫				⚫	DeRain,DeHaizing,DeSnow/192x320,256x320,320x480,384x640,480x640,736x1280
261	EfficientDerain	■■■											⚫	v4_SPA,v4_rain100H,v4_rain1400/192x320,256x320,320x480,384x640,480x640,608x800,736x1280
270	HWMNet	■■■											⚫	Low-light Image Enhancement/192x320,256x320,320x480,384x640,480x640,544x960,720x1280
275	FD-GAN	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	DeHaizing/192x320,256x320,384x640,480x640,720x1280,1080x1920,No-LICENSE
277	EDN-GTM	■■■	⚫	⚫			⚫	⚫	⚫	⚫	⚫		⚫	DeHaizing/192x320,240x320,384x480,480x640,512x512,720x1280,1088x1920
281	IMDN	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	x4/64x64,96x96,128x128,120x160,160x160,180x320,192x192,256x256,180x320,240x320,360x640,480x640
283	UIE-WD	■■■											⚫	Underwater Image Enhancement/WIP issue #97/192x320,240x320,320x480,360x640,480x640,720x1280,1080x1920
285	Decoupled-Low-light-Image-Enhancement	■■■											⚫	Low-light Image Enhancement/180x320,240x320,360x480,360x640,480x640,720x1280
286	SCI	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	Low-light Image Enhancement/180x320,240x320,360x480,360x640,480x640,720x1280
315	Illumination-Adaptive-Transformer	■■■											⚫	Low-light Image Enhancement
316	night_enhancement	■■■											⚫	Low-light Image Enhancement
320	Dehamer	■■■											⚫	Dehazing
323	Stripformer	■■■											⚫	DeBlur
325	DehazeFormer	■■■											⚫	Dehazing
344	XYDeblur	■■■											⚫	DeBlur
348	Bread	■■■											⚫	Low-light Image Enhancement
348	PMN	■■■											⚫	DeNoise, Low-light Image Enhancement
351	RFDN	■■■											⚫	x4
352	MAXIM	■■■	⚫	⚫									⚫	Dehaze only
353	ShadowFormer	■■■											⚫	Shadow Removal
354	DEA-Net	■■■	⚫	⚫									⚫	DeHaze
359	MSPFN	■■■	⚫	⚫									⚫	DeRain
361	KBNet	■■■											⚫	Real Image Denoising
367	FLW-Net	■■■											⚫	Low-light Image Enhancement
368	C2PNet	■■■											⚫	DeHaze
370	Semantic-Guided-Low-Light-Image-Enhancement	■■■											⚫	Low-light Image Enhancement
372	URetinex-Net	■■■											⚫	Low-light Image Enhancement
375	SCANet	■■■											⚫	DeHaze
377	DRSformer	■■■											⚫	DeRain
385	PairLIE	■■■											⚫	Low-light Image Enhancement
389	WGWS-Net	■■■											⚫	DeRain,DeRainDrop,DeHaize,DeSnow
396	MixDehazeNet	■■■											⚫	DeHaize
400	CSRNet	■■■											⚫	Low-light Image Enhancement
404	HDR-Transformer	■■■											⚫
409	nighttime_dehaze	■■■											⚫	DeHaze
411	UDR-S2Former_deraining	■■■											⚫	DeRain
418	Diffusion-Low-Light	■■■	⚫	⚫									⚫	Diffusion, Low-light Image Enhancement

12. Sound Classifier

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
013	ml-sound-classifier	■■■	⚫	⚫	⚫			⚫	⚫		⚫	⚫
097	YAMNet	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
098	SPICE	■■■	⚫	⚫	⚫			⚫			⚫	⚫
118	Speech-enhancement	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	WIP,EdgeTPU(LeakyLeRU)
120	FRILL	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	nofrontend
177	BirdNET-Lite	■■■	⚫	⚫	⚫		⚫	⚫				⚫		non-flex
381	Whisper	■■■										⚫
382	Light-SERNet	■■■	⚫	⚫	⚫

13. Natural Language Processing

No.	Model Name	Link	FP32	FP16	WQ	OV	CM	TFJS	TF-TRT	ONNX
048	Mobile_BERT	■■■	⚫	⚫	⚫	⚫	⚫	⚫		⚫
121	GPT2/DistillGPT2	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫
122	DistillBert	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫

14. Text Recognition

No.	Model Name	Link	FP32	FP16	INT8	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
052	Handwritten_Text_Recognition	■■■	⚫	⚫		⚫
055	Handwritten_Japanese_Recognition	■■■	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
093	ocr_japanese	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	120x160

15. Action Recognition

No.	Model Name	Link	FP32	FP16	INT8	TPU	WQ	OV	CM	TF-TRT	ONNX	Remarks
092	weld-porosity-detection-0001	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫
247	PoseC3D	■■■									⚫	Skeleton-based/FineGYM,NTU60_XSub,NTU120_XSub,UCF101,HMDB51/1x20x48x64x64
248	MS-G3D	■■■									⚫	Skeleton-based/Kinetics,NTU60,NTU120/1x3xTx25x2

16. Inpainting

No.	Model Name	Link	FP32	FP16	INT8	DQ	WQ	OV	CM	TFJS	ONNX
100	HiFill	■■■	⚫	⚫			⚫	⚫			⚫
163	MST_inpainting	■■■	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫	⚫
273	OPN(Onion-Peel Networks)	■■■									⚫
274	DeepFillv2	■■■	⚫	⚫		⚫	⚫	⚫			⚫

17. GAN

No.	Model Name	Link	FP32	FP16	INT8	TPU	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
105	MobileStyleGAN	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
310	attentive-gan-derainnet	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫		⚫	DeRain/180x320,240x320,240x360,320x480,360x640,480x640,720x1280

18. Transformer

No.	Model Name	Link	FP32	FP16	INT8	TPU	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
127	dino	■■■	⚫	⚫			⚫	⚫	⚫	⚫		⚫	experimental,dino_deits8/dino_deits16

19. Others

No.	Model Name	Link	FP32	FP16	INT8	TPU	DQ	WQ	OV	CM	TFJS	TF-TRT	ONNX	Remarks
091	gaze-estimation-adas-0002	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
102	Coconet	■■■	⚫	⚫				⚫			⚫			magenta
108	HAWP	■■■											⚫	Line Parsing,WIP
110	L-CNN	■■■											⚫	Line Parsing,WIP
117	DTLN	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫
119	M-LSD	■■■	⚫	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫
131	CFNet	■■■											⚫	256x256,512x768
139	PSD-Principled-Synthetic-to-Real-Dehazing-Guided-by-Physical-Priors	■■■	⚫	⚫				⚫	⚫	⚫	⚫	⚫	⚫
140	Ultra-Fast-Lane-Detection	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	288x800
141	lanenet-lane-detection	■■■	⚫	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	256x512
154	driver-action-recognition-adas-0002-encoder	■■■	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	⚫
155	driver-action-recognition-adas-0002-decoder	■■■	⚫	⚫			⚫	⚫	⚫	⚫	⚫	⚫	⚫
167	LSTR	■■■	⚫	⚫			⚫	⚫	⚫		⚫		⚫	180x320,240x320,360x640,480x640,720x1280
229	DexiNed	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫	⚫	⚫	160x320,320x480,368x640,480x640,720x1280
233	HRNet-for-Fashion-Landmark-Estimation	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	192x320,256x320,320x480,384x640,480x640,736x1280
237	piano_transcription	■■■							⚫				⚫	1x160000,Nx160000
252	RAFT	■■■											⚫	small,chairs,kitti,sintel,things/iters=10,20/240x320,360x480,480x640
254	FullSubNet-plus	■■■											⚫	1x1x257x100,200,500,1000,2000,3000,5000,7000,8000,10000
255	FILM	■■■	⚫	⚫			⚫	⚫	⚫		⚫		⚫	L1,Style,VGG/256x256,180x320,240x320,360x640,480x640,720x1280,1080x1920
260	KP2D	■■■	⚫	⚫			⚫	⚫	⚫				⚫	ResNet/128x320,192x320,192x448,192x640,256x320,256x448,256x640,320x448,384x640,480x640,512x1280,736x1280
272	CSFlow	■■■											⚫	chairs,kitti,things/iters=10,20/192x320,240x320,320x480,384x640,480x640,736x1280
276	HybridNets	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	anchor_HxW.npy/256x384,256x512,384x512,384x640,384x1024,512x640,768x1280,1152x1920
278	DWARF	■■■	⚫	⚫	⚫		⚫	⚫	⚫	⚫	⚫		⚫	StereoDepth+OpticalFlow,/192x320,256x320,384x640,512x640,512x640,768x1280
279	F-Clip	■■■	⚫	⚫	⚫		⚫	⚫	⚫		⚫		⚫	Line Parsing/ALL/192x320,256x320,320x480,384x640,480x640,736x1280
288	perceptual-reflection-removal	■■■	⚫	⚫	⚫		⚫	⚫	⚫		⚫		⚫	Reflection-Removal/180x320,240x320,360x480,360x640,480x640,720x1280
291	SeAFusion	■■■	⚫	⚫			⚫	⚫	⚫		⚫		⚫	180x320,240x320,360x480,360x640,480x640,720x1280
297	GazeNet	■■■											⚫	1x7x3x256x192/NxFx3x256x192
298	DEQ-Flow	■■■											⚫	AGPL-3.0 license
306	GMFlowNet	■■■											⚫	OpticalFlow/192x320,240x320,320x480,360x640,480x640,720x1280
309	ImageForensicsOSN	■■■	⚫	⚫	⚫		⚫	⚫	⚫		⚫		⚫	forgery detection/180x320,240x320,320x480,360x640,480x640,720x1280
318	pips	■■■											⚫
324	Ultra-Fast-Lane-Detection-v2	■■■											⚫
326	YOLOPv2	■■■	⚫	⚫			⚫	⚫	⚫				⚫
328	Stable_Diffusion	■■■											⚫
339	DeepLSD	■■■											⚫
342	ALIKE	■■■	⚫	⚫	⚫		⚫	⚫					⚫
357	Unimatch	■■■	⚫	⚫									⚫	OpticalFlow, StereoDepth
360	PARSeq	■■■	⚫	⚫									⚫	Scene Text Recognition
366	text_recognition_CRNN	■■■	⚫	⚫									⚫	CN/CH/EN
373	LiteTrack	■■■	⚫	⚫									⚫	Tracking
374	LaneSOD	■■■											⚫	Lane Segmentation
378	P2PNet_tfkeras	■■■	⚫	⚫									⚫
388	LightGlue	■■■											⚫	Keypoint Matching
398	L2CS-Net	■■■	⚫	⚫	⚫		⚫						⚫	Gaze Pose 448x448
401	CLRerNet	■■■											⚫	Lane Detection
406	DeDoDe	■■■	⚫	⚫									⚫	Keypoint Detection, Description, Matching
407	Generalizing_Gaze_Estimation	■■■	⚫	⚫									⚫	Gaze Pose 160x160
408	UAED	■■■											⚫	Edge Detectopm
413	DocShadow	■■■	⚫										⚫	Document Shadow Removal
416	GeoNet	■■■	⚫										⚫	MonoDepth, CameraPose, OpticalFlow
428	ISR	■■■											⚫	Person ReID

Sample.1 - Object detection by video file

RaspberryPi4 (CPU only)
Raspbian Buster 64bit
Tensorflow / Tensorflow Lite with multi-thread acceleration tuning for PythonAPI
MobileNetV2-SSDLite 300x300 Integer Quantization
Pascal-VOC Dataset (Japanese article)
MP4 30FPS, 640x360
Approximately 14FPS ~ 15FPS for all processes from pre-processing, inference, post-processing, and display

$ cd 006_mobilenetv2-ssdlite/02_voc/03_integer_quantization
$ ./download.sh && cd ..
$ python3 mobilenetv2ssdlite_movie_sync.py

Sample.2 - Object detection by USB Camera

RaspberryPi4 (CPU only)
Ubuntu 19.10 64bit
Tensorflow / Tensorflow Lite with multi-thread acceleration tuning for PythonAPI
MobileNetV2-SSDLite 300x300 Integer Quantization
Pascal-VOC Dataset (Japanese article)
USB Camera, 640x480
IPS 1080p HDMI Display
Approximately 12FPS for all processes from pre-processing, inference, post-processing, and display

$ cd 006_mobilenetv2-ssdlite/02_voc/03_integer_quantization
$ ./download.sh && cd ..
$ python3 mobilenetv2ssdlite_usbcam_sync.py

Sample.3 - Head Pose Estimation, Multi-stage inference with multi-model

RaspberryPi4 (CPU only)
Ubuntu 19.10 64bit
Tensorflow / Tensorflow Lite with multi-thread acceleration tuning for PythonAPI
[Model.1] MobileNetV2-SSDLite dm=0.5 300x300, Integer Quantization
[Model.2] Head Pose Estimation 128x128, Integer Quantization
WIDERFACE
USB Camera, 640x480
IPS 1080p HDMI Display
Approximately 13FPS for all processes from pre-processing, inference, post-processing, and display

$ cd 025_head_pose_estimation/03_integer_quantization
$ ./download.sh
$ python3 head_pose_estimation.py

Sample.4 - Semantic Segmentation, DeeplabV3-plus 256x256

RaspberryPi4 (CPU only)
Ubuntu 19.10 64bit
Tensorflow / Tensorflow Lite with multi-thread acceleration tuning for PythonAPI
DeeplabV3-plus (MobileNetV2) Decoder 256x256, Integer Quantization
USB Camera, 640x480
IPS 1080p HDMI Display
Approximately 8.5 FPS for all processes from pre-processing, inference, post-processing, and display

$ cd 026_mobile-deeplabv3-plus/03_integer_quantization
$ ./download.sh
$ python3 deeplabv3plus_usbcam.py

Sample.5 - MediaPipe/FaceMesh, face_detection_front_128_weight_quant, face_landmark_192_weight_quant

Ubuntu 18.04 x86_64
Tensorflow Lite
C/C++
OpenGL
USB Camera, 640x480
$ v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=YUYV
Test Code - tflite_gles_app - gl2facemesh - @terryky

Sample.6 - MediaPipe/Objectron, object_detection_3d_chair_640x480_weight_quant

Ubuntu 18.04 x86_64
Tensorflow Lite
C/C++
OpenGL
USB Camera, 640x480
$ v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=YUYV
Test Code - tflite_gles_app - gl2objectron - @terryky

Sample.7 - MediaPipe/Objectron, object_detection_3d_chair_640x480_openvino_FP32

Ubuntu 18.04 x86_64
OpenVINO 2020.2
Python
Core i7 (CPU only)
USB Camera, 640x480
Test Code - objectron-3d-object-detection-openvino - @yas-sim

Sample.8 - MediaPipe/BlazeFace, face_detection_front_128_integer_quant

RaspberryPi4
Ubuntu 19.10 aarch64
Tensorflow Lite
C/C++
OpenGL
USB Camera, 640x480
$ v4l2-ctl --set-fmt-video=width=640,height=480,pixelformat=YUYV
Test Code - tflite_gles_app - gl2blazeface - @terryky

Sample.9 - MediaPipe/Hand_Detection_and_Tracking(3D Hand Pose), hand_landmark_3d_256_integer_quant.tflite + palm_detection_builtin_256_integer_quant.tflite

RaspberryPi4
Tensorflow Lite
C/C++
OpenGL
Test Code - tflite_gles_app - gl2handpose - @terryky

Sample.10 - DBFace, 640x480_openvino_FP32

Ubuntu 18.04 x86_64
OpenVINO 2020.2
Python
Core i7 (CPU only)
USB Camera, 640x480
Test Code - DBFace-on-OpenVINO - @yas-sim

Sample.11 - Human_Pose_Estimation_3D, 640x480, Tensorflow.js + WebGL + Browser

Ubuntu 18.04 x86_64
Tensorflow.js
USB Camera, 640x480
Test Code - tfjs_webgl_app - @terryky

Sample.12 - BlazePose Full Body, 640x480, Tensorflow.js + WebGL + Browser

Ubuntu 18.04 x86_64
Tensorflow.js
USB Camera, 640x480
Test Code - tfjs_webgl_app - @terryky

Sample.13 - Facial Cartoonization, 640x480, OpenVINO Corei7 CPU only

Ubuntu 18.04 x86_64
OpenVINO
USB Camera, 640x480
Test Code

1. Environment

Ubuntu 18.04 x86_64
RaspberryPi4 Raspbian Buster 32bit / Raspbian Buster 64bit / Ubuntu 19.10 aarch64
Tensorflow-GPU v1.15.2 or Tensorflow v2.3.1+
OpenVINO 2020.2+
PyTorch 1.6.0+
ONNX Opset12
Python 3.6.8
PascalVOC Dataset
COCO Dataset
Cityscapes Dataset
Imagenette Dataset
CelebA Dataset
Audio file (.wav)
WIDERFACE
Google Colaboratory

2. Procedure

Procedure examples

2-1. MobileNetV3+DeeplabV3+PascalVOC

2-1-1. Preparation

$ cd ~
$ mkdir deeplab;cd deeplab
$ git clone --depth 1 https://github.com/tensorflow/models.git
$ cd models/research/deeplab/datasets
$ mkdir pascal_voc_seg

$ curl -sc /tmp/cookie \
  "https://drive.google.com/uc?export=download&id=1rATNHizJdVHnaJtt-hW9MOgjxoaajzdh" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie \
  "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1rATNHizJdVHnaJtt-hW9MOgjxoaajzdh" \
  -o pascal_voc_seg/VOCtrainval_11-May-2012.tar

$ sed -i -e "s/python .\/remove_gt_colormap.py/python3 .\/remove_gt_colormap.py/g" \
      -i -e "s/python .\/build_voc2012_data.py/python3 .\/build_voc2012_data.py/g" \
      download_and_convert_voc2012.sh

$ sh download_and_convert_voc2012.sh

$ cd ../..
$ mkdir -p deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/train
$ mkdir -p deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/eval
$ mkdir -p deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/vis

$ export PATH_TO_TRAIN_DIR=${HOME}/deeplab/models/research/deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/train
$ export PATH_TO_DATASET=${HOME}/deeplab/models/research/deeplab/datasets/pascal_voc_seg/tfrecord
$ export PYTHONPATH=${HOME}/deeplab/models/research:${HOME}/deeplab/models/research/deeplab:${HOME}/deeplab/models/research/slim:${PYTHONPATH}

# See feature_extractor.network_map for supported model variants.
# models/research/deeplab/core/feature_extractor.py

networks_map = {
    'mobilenet_v2': _mobilenet_v2,
    'mobilenet_v3_large_seg': mobilenet_v3_large_seg,
    'mobilenet_v3_small_seg': mobilenet_v3_small_seg,
    'resnet_v1_18': resnet_v1_beta.resnet_v1_18,
    'resnet_v1_18_beta': resnet_v1_beta.resnet_v1_18_beta,
    'resnet_v1_50': resnet_v1_beta.resnet_v1_50,
    'resnet_v1_50_beta': resnet_v1_beta.resnet_v1_50_beta,
    'resnet_v1_101': resnet_v1_beta.resnet_v1_101,
    'resnet_v1_101_beta': resnet_v1_beta.resnet_v1_101_beta,
    'xception_41': xception.xception_41,
    'xception_65': xception.xception_65,
    'xception_71': xception.xception_71,
    'nas_pnasnet': nas_network.pnasnet,
    'nas_hnasnet': nas_network.hnasnet,
}

2-1-2. "mobilenet_v3_small_seg" Float32 regular training

$ python3 deeplab/train.py \
    --logtostderr \
    --training_number_of_steps=500000 \
    --train_split="train" \
    --model_variant="mobilenet_v3_small_seg" \
    --decoder_output_stride=16 \
    --train_crop_size="513,513" \
    --train_batch_size=8 \
    --dataset="pascal_voc_seg" \
    --save_interval_secs=300 \
    --save_summaries_secs=300 \
    --save_summaries_images=True \
    --log_steps=100 \
    --train_logdir=${PATH_TO_TRAIN_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

2-1-3. "mobilenet_v3_large_seg" Float32 regular training

$ python3 deeplab/train.py \
    --logtostderr \
    --training_number_of_steps=1000000 \
    --train_split="train" \
    --model_variant="mobilenet_v3_large_seg" \
    --decoder_output_stride=16 \
    --train_crop_size="513,513" \
    --train_batch_size=8 \
    --dataset="pascal_voc_seg" \
    --save_interval_secs=300 \
    --save_summaries_secs=300 \
    --save_summaries_images=True \
    --log_steps=100 \
    --train_logdir=${PATH_TO_TRAIN_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

2-1-4. Visualize training status

$ tensorboard \
  --logdir ${HOME}/deeplab/models/research/deeplab/datasets/pascal_voc_seg/exp/train_on_train_set/train

2-2. MobileNetV3+DeeplabV3+Cityscaps - Post-training quantization

2-2-1. Preparation

$ cd ~
$ mkdir -p git/deeplab && cd git/deeplab
$ git clone --depth 1 https://github.com/tensorflow/models.git
$ cd models/research/deeplab/datasets
$ mkdir cityscapes && cd cityscapes

# Clone the script to generate Cityscapes Dataset.
$ git clone --depth 1 https://github.com/mcordts/cityscapesScripts.git
$ mv cityscapesScripts cityscapesScripts_ && \
  mv cityscapesScripts_/cityscapesscripts . && \
  rm -rf cityscapesScripts_

# Download Cityscapes Dataset.
# https://www.cityscapes-dataset.com/
# You will need to sign up and issue a userID and password to download the data set.
$ wget --keep-session-cookies --save-cookies=cookies.txt \
  --post-data 'username=(userid)&password=(password)&submit=Login' \
  https://www.cityscapes-dataset.com/login/
$ wget --load-cookies cookies.txt \
  --content-disposition https://www.cityscapes-dataset.com/file-handling/?packageID=1
$ wget --load-cookies cookies.txt \
  --content-disposition https://www.cityscapes-dataset.com/file-handling/?packageID=3
$ unzip gtFine_trainvaltest.zip && rm gtFine_trainvaltest.zip
$ rm README && rm license.txt
$ unzip leftImg8bit_trainvaltest.zip && rm leftImg8bit_trainvaltest.zip
$ rm README && rm license.txt

# Convert Cityscapes Dataset to TFRecords format.
$ cd ..
$ sed -i -e "s/python/python3/g" convert_cityscapes.sh
$ export PYTHONPATH=${HOME}/git/deeplab/models/research/deeplab/datasets/cityscapes:${PYTHONPATH}
$ sh convert_cityscapes.sh

# Create a checkpoint storage folder for training. If training is not required,
# there is no need to carry out.
$ cd ../..
$ mkdir -p deeplab/datasets/cityscapes/exp/train_on_train_set/train && \
  mkdir -p deeplab/datasets/cityscapes/exp/train_on_train_set/eval && \
  mkdir -p deeplab/datasets/cityscapes/exp/train_on_train_set/vis

# Download the DeepLabV3 trained model of the MobileNetV3 backbone.
$ curl -sc /tmp/cookie \
  "https://drive.google.com/uc?export=download&id=1f5ccaJmJBYwBmHvRQ77yGIUcXnqQIRY_" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie \
  "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1f5ccaJmJBYwBmHvRQ77yGIUcXnqQIRY_" \
  -o deeplab_mnv3_small_cityscapes_trainfine_2019_11_15.tar.gz
$ tar -zxvf deeplab_mnv3_small_cityscapes_trainfine_2019_11_15.tar.gz
$ rm deeplab_mnv3_small_cityscapes_trainfine_2019_11_15.tar.gz

$ curl -sc /tmp/cookie \
  "https://drive.google.com/uc?export=download&id=1QxS3G55rUQvuiBF-hztQv5zCkfPfwlVU" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie \
  "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1QxS3G55rUQvuiBF-hztQv5zCkfPfwlVU" \
  -o deeplab_mnv3_large_cityscapes_trainfine_2019_11_15.tar.gz
$ tar -zxvf deeplab_mnv3_large_cityscapes_trainfine_2019_11_15.tar.gz
$ rm deeplab_mnv3_large_cityscapes_trainfine_2019_11_15.tar.gz

$ export PATH_TO_INITIAL_CHECKPOINT=${HOME}/git/deeplab/models/research/deeplab_mnv3_small_cityscapes_trainfine/model.ckpt
$ export PATH_TO_DATASET=${HOME}/git/deeplab/models/research/deeplab/datasets/cityscapes/tfrecord
$ export PYTHONPATH=${HOME}/git/deeplab/models/research:${HOME}/git/deeplab/models/research/deeplab:${HOME}/git/deeplab/models/research/slim:${PYTHONPATH}

# Fix a bug in the data generator.
$ sed -i -e \
  "s/splits_to_sizes={'train_fine': 2975,/splits_to_sizes={'train': 2975,/g" \
  deeplab/datasets/data_generator.py

# Back up the trained model.
$ cd ${HOME}/git/deeplab/models/research
$ cp deeplab/export_model.py deeplab/export_model.py_org
$ cp deeplab_mnv3_small_cityscapes_trainfine/frozen_inference_graph.pb \
  deeplab_mnv3_small_cityscapes_trainfine/frozen_inference_graph_org.pb
$ cp deeplab_mnv3_large_cityscapes_trainfine/frozen_inference_graph.pb \
  deeplab_mnv3_large_cityscapes_trainfine/frozen_inference_graph_org.pb

# Customize "export_model.py" according to the input resolution. Must be (multiple of 8 + 1).
#   (example.1) 769 = 8 * 96 + 1
#   (example.2) 512 = 8 * 64 + 1
#   (example.3) 320 = 8 * 40 + 1
# And it is necessary to change from tf.uint8 type to tf.float32 type.
$ sed -i -e \
  "s/tf.placeholder(tf.uint8, \[1, None, None, 3\], name=_INPUT_NAME)/tf.placeholder(tf.float32, \[1, 769, 769, 3\], name=_INPUT_NAME)/g" \
  deeplab/export_model.py

2-2-2. Parameter sheet

# crop_size and image_pooling_crop_size are multiples of --decoder_output_stride + 1
# 769 = 8 * 96 + 1
# 513 = 8 * 64 + 1
# 321 = 8 * 40 + 1

# --initialize_last_layer=True initializes the final layer with the weight of
# tf_initial_checkpoint (inherits the weight)

# Named tuple to describe the dataset properties.
# deeplab/datasets/data_generator.py
DatasetDescriptor = collections.namedtuple(
    'DatasetDescriptor',
    [
        'splits_to_sizes',  # Splits of the dataset into training, val and test.
        'num_classes',  # Number of semantic classes, including the
                        # background class (if exists). For example, there
                        # are 20 foreground classes + 1 background class in
                        # the PASCAL VOC 2012 dataset. Thus, we set
                        # num_classes=21.
        'ignore_label',  # Ignore label value.
    ])

_CITYSCAPES_INFORMATION = DatasetDescriptor(
    splits_to_sizes={'train': 2975,
                     'train_coarse': 22973,
                     'trainval_fine': 3475,
                     'trainval_coarse': 23473,
                     'val_fine': 500,
                     'test_fine': 1525},
    num_classes=19,
    ignore_label=255,
)

_PASCAL_VOC_SEG_INFORMATION = DatasetDescriptor(
    splits_to_sizes={
        'train': 1464,
        'train_aug': 10582,
        'trainval': 2913,
        'val': 1449,
    },
    num_classes=21,
    ignore_label=255,
)

_ADE20K_INFORMATION = DatasetDescriptor(
    splits_to_sizes={
        'train': 20210,  # num of samples in images/training
        'val': 2000,  # num of samples in images/validation
    },
    num_classes=151,
    ignore_label=0,
)

_DATASETS_INFORMATION = {
    'cityscapes': _CITYSCAPES_INFORMATION,
    'pascal_voc_seg': _PASCAL_VOC_SEG_INFORMATION,
    'ade20k': _ADE20K_INFORMATION,
}

# A map from network name to network function. model_variant.
# deeplab/core/feature_extractor.py
networks_map = {
    'mobilenet_v2': _mobilenet_v2,
    'mobilenet_v3_large_seg': mobilenet_v3_large_seg,
    'mobilenet_v3_small_seg': mobilenet_v3_small_seg,
    'resnet_v1_18': resnet_v1_beta.resnet_v1_18,
    'resnet_v1_18_beta': resnet_v1_beta.resnet_v1_18_beta,
    'resnet_v1_50': resnet_v1_beta.resnet_v1_50,
    'resnet_v1_50_beta': resnet_v1_beta.resnet_v1_50_beta,
    'resnet_v1_101': resnet_v1_beta.resnet_v1_101,
    'resnet_v1_101_beta': resnet_v1_beta.resnet_v1_101_beta,
    'xception_41': xception.xception_41,
    'xception_65': xception.xception_65,
    'xception_71': xception.xception_71,
    'nas_pnasnet': nas_network.pnasnet,
    'nas_hnasnet': nas_network.hnasnet,
}

2-2-3. "mobilenet_v3_small_seg" Export Model

Generate Freeze Graph (.pb) with INPUT Placeholder changed from checkpoint file (.ckpt).

$ python3 deeplab/export_model.py \
    --checkpoint_path=./deeplab_mnv3_small_cityscapes_trainfine/model.ckpt \
    --export_path=./deeplab_mnv3_small_cityscapes_trainfine/frozen_inference_graph.pb \
    --num_classes=19 \
    --crop_size=769 \
    --crop_size=769 \
    --model_variant="mobilenet_v3_small_seg" \
    --image_pooling_crop_size="769,769" \
    --image_pooling_stride=4,5 \
    --aspp_convs_filters=128 \
    --aspp_with_concat_projection=0 \
    --aspp_with_squeeze_and_excitation=1 \
    --decoder_use_sum_merge=1 \
    --decoder_filters=19 \
    --decoder_output_is_logits=1 \
    --image_se_uses_qsigmoid=1 \
    --image_pyramid=1 \
    --decoder_output_stride=8

2-2-4. "mobilenet_v3_large_seg" Export Model

Generate Freeze Graph (.pb) with INPUT Placeholder changed from checkpoint file (.ckpt).

$ python3 deeplab/export_model.py \
    --checkpoint_path=./deeplab_mnv3_large_cityscapes_trainfine/model.ckpt \
    --export_path=./deeplab_mnv3_large_cityscapes_trainfine/frozen_inference_graph.pb \
    --num_classes=19 \
    --crop_size=769 \
    --crop_size=769 \
    --model_variant="mobilenet_v3_large_seg" \
    --image_pooling_crop_size="769,769" \
    --image_pooling_stride=4,5 \
    --aspp_convs_filters=128 \
    --aspp_with_concat_projection=0 \
    --aspp_with_squeeze_and_excitation=1 \
    --decoder_use_sum_merge=1 \
    --decoder_filters=19 \
    --decoder_output_is_logits=1 \
    --image_se_uses_qsigmoid=1 \
    --image_pyramid=1 \
    --decoder_output_stride=8

If you follow the Google Colaboratory sample procedure, copy the "deeplab_mnv3_small_cityscapes_trainfine" folder and "deeplab_mnv3_large_cityscapes_trainfine" to your Google Drive "My Drive". It is not necessary if all procedures described in Google Colaboratory are performed in a PC environment.

2-2-5. Google Colaboratory - Post-training quantization - post_training_integer_quant.ipynb

Weight Quantization
Integer Quantization
Full Integer Quantization

https://colab.research.google.com/drive/1TtCJ-uMNTArpZxrf5DCNbZdn08DsiW8F 　　

2-3. MobileNetV3+DeeplabV3+Cityscaps - Quantization-aware training

2-3-1. "mobilenet_v3_small_seg" Quantization-aware training

$ cd ${HOME}/git/deeplab/models/research
$ export PATH_TO_TRAINED_FLOAT_MODEL=${HOME}/git/deeplab/models/research/deeplab_mnv3_small_cityscapes_trainfine/model.ckpt
$ export PATH_TO_TRAIN_DIR=${HOME}/git/deeplab/models/research/deeplab/datasets/cityscapes/exp/train_on_train_set/train
$ export PATH_TO_DATASET=${HOME}/git/deeplab/models/research/deeplab/datasets/cityscapes/tfrecord

# deeplab_mnv3_small_cityscapes_trainfine
$ python3 deeplab/train.py \
    --logtostderr \
    --training_number_of_steps=5000 \
    --train_split="train" \
    --model_variant="mobilenet_v3_small_seg" \
    --train_crop_size="769,769" \
    --train_batch_size=8 \
    --dataset="cityscapes" \
    --initialize_last_layer=False \
    --base_learning_rate=3e-5 \
    --quantize_delay_step=0 \
    --image_pooling_crop_size="769,769" \
    --image_pooling_stride=4,5 \
    --aspp_convs_filters=128 \
    --aspp_with_concat_projection=0 \
    --aspp_with_squeeze_and_excitation=1 \
    --decoder_use_sum_merge=1 \
    --decoder_filters=19 \
    --decoder_output_is_logits=1 \
    --image_se_uses_qsigmoid=1 \
    --image_pyramid=1 \
    --decoder_output_stride=8 \
    --save_interval_secs=300 \
    --save_summaries_secs=300 \
    --save_summaries_images=True \
    --log_steps=100 \
    --tf_initial_checkpoint=${PATH_TO_TRAINED_FLOAT_MODEL} \
    --train_logdir=${PATH_TO_TRAIN_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

2-3-2. "mobilenet_v3_large_seg" Quantization-aware training

$ cd ${HOME}/git/deeplab/models/research
$ export PATH_TO_TRAINED_FLOAT_MODEL=${HOME}/git/deeplab/models/research/deeplab_mnv3_large_cityscapes_trainfine/model.ckpt
$ export PATH_TO_TRAIN_DIR=${HOME}/git/deeplab/models/research/deeplab/datasets/cityscapes/exp/train_on_train_set/train
$ export PATH_TO_DATASET=${HOME}/git/deeplab/models/research/deeplab/datasets/cityscapes/tfrecord

# deeplab_mnv3_large_cityscapes_trainfine
$ python3 deeplab/train.py \
    --logtostderr \
    --training_number_of_steps=4350 \
    --train_split="train" \
    --model_variant="mobilenet_v3_large_seg" \
    --train_crop_size="769,769" \
    --train_batch_size=8 \
    --dataset="cityscapes" \
    --initialize_last_layer=False \
    --base_learning_rate=3e-5 \
    --quantize_delay_step=0 \
    --image_pooling_crop_size="769,769" \
    --image_pooling_stride=4,5 \
    --aspp_convs_filters=128 \
    --aspp_with_concat_projection=0 \
    --aspp_with_squeeze_and_excitation=1 \
    --decoder_use_sum_merge=1 \
    --decoder_filters=19 \
    --decoder_output_is_logits=1 \
    --image_se_uses_qsigmoid=1 \
    --image_pyramid=1 \
    --decoder_output_stride=8 \
    --save_interval_secs=300 \
    --save_summaries_secs=300 \
    --save_summaries_images=True \
    --log_steps=100 \
    --tf_initial_checkpoint=${PATH_TO_TRAINED_FLOAT_MODEL} \
    --train_logdir=${PATH_TO_TRAIN_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

The orange line is "deeplab_mnv3_small_cityscapes_trainfine" loss. The blue line is "deeplab_mnv3_large_cityscapes_trainfine" loss. 　　

2-4. MobileNetV2+DeeplabV3+coco/voc - Post-training quantization

2-4-1. Preparation

$ cd ${HOME}/git/deeplab/models/research

$ wget http://download.tensorflow.org/models/deeplabv3_mnv2_dm05_pascal_trainaug_2018_10_01.tar.gz
$ tar -zxvf deeplabv3_mnv2_dm05_pascal_trainaug_2018_10_01.tar.gz
$ rm deeplabv3_mnv2_dm05_pascal_trainaug_2018_10_01.tar.gz

$ wget http://download.tensorflow.org/models/deeplabv3_mnv2_dm05_pascal_trainval_2018_10_01.tar.gz
$ tar -zxvf deeplabv3_mnv2_dm05_pascal_trainval_2018_10_01.tar.gz
$ rm deeplabv3_mnv2_dm05_pascal_trainval_2018_10_01.tar.gz

$ wget http://download.tensorflow.org/models/deeplabv3_mnv2_pascal_train_aug_2018_01_29.tar.gz
$ tar -zxvf deeplabv3_mnv2_pascal_train_aug_2018_01_29.tar.gz
$ rm deeplabv3_mnv2_pascal_train_aug_2018_01_29.tar.gz

$ sed -i -e \
  "s/tf.placeholder(tf.uint8, \[1, None, None, 3\], name=_INPUT_NAME)/tf.placeholder(tf.float32, \[1, 257, 257, 3\], name=_INPUT_NAME)/g" \
  deeplab/export_model.py

$ export PYTHONPATH=${HOME}/git/deeplab/models/research:${HOME}/git/deeplab/models/research/deeplab:${HOME}/git/deeplab/models/research/slim:${PYTHONPATH}

$ python3 deeplab/export_model.py \
  --checkpoint_path=./deeplabv3_mnv2_dm05_pascal_trainaug/model.ckpt \
  --export_path=./deeplabv3_mnv2_dm05_pascal_trainaug/frozen_inference_graph.pb \
  --model_variant="mobilenet_v2" \
  --crop_size=257 \
  --crop_size=257 \
  --depth_multiplier=0.5

$ python3 deeplab/export_model.py \
  --checkpoint_path=./deeplabv3_mnv2_dm05_pascal_trainval/model.ckpt \
  --export_path=./deeplabv3_mnv2_dm05_pascal_trainval/frozen_inference_graph.pb \
  --model_variant="mobilenet_v2" \
  --crop_size=257 \
  --crop_size=257 \
  --depth_multiplier=0.5

$ python3 deeplab/export_model.py \
  --checkpoint_path=./deeplabv3_mnv2_pascal_train_aug/model.ckpt-30000 \
  --export_path=./deeplabv3_mnv2_pascal_train_aug/frozen_inference_graph.pb \
  --model_variant="mobilenet_v2" \
  --crop_size=257 \
  --crop_size=257

2-5. MobileNetV3-SSD+coco - Post-training quantization

2-5-1. Preparation

$ cd ~
$ sudo pip3 install tensorflow-gpu==1.15.0
$ git clone --depth 1 https://github.com/tensorflow/models.git
$ cd models/research

$ git clone https://github.com/cocodataset/cocoapi.git
$ cd cocoapi/PythonAPI
$ make
$ cp -r pycocotools ../..
$ cd ../..
$ wget -O protobuf.zip https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip
$ unzip protobuf.zip
$ ./bin/protoc object_detection/protos/*.proto --python_out=.

$ sudo apt-get install -y protobuf-compiler python3-pil python3-lxml python3-tk
$ sudo -H pip3 install Cython contextlib2 jupyter matplotlib

$ export PYTHONPATH=${PWD}:${PWD}/object_detection:${PWD}/slim:${PYTHONPATH}

$ mkdir -p ssd_mobilenet_v3_small_coco_2019_08_14 && cd ssd_mobilenet_v3_small_coco_2019_08_14
$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1uqaC0Y-yRtzkpu1EuZ3BzOyh9-i_3Qgi" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1uqaC0Y-yRtzkpu1EuZ3BzOyh9-i_3Qgi" -o ssd_mobilenet_v3_small_coco_2019_08_14.tar.gz
$ tar -zxvf ssd_mobilenet_v3_small_coco_2019_08_14.tar.gz
$ rm ssd_mobilenet_v3_small_coco_2019_08_14.tar.gz
$ cd ..

$ mkdir -p ssd_mobilenet_v3_large_coco_2019_08_14 && cd ssd_mobilenet_v3_large_coco_2019_08_14
$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1NGLjKRWDQZ_kibQHlLZ7Eetuuz1waC7X" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1NGLjKRWDQZ_kibQHlLZ7Eetuuz1waC7X" -o ssd_mobilenet_v3_large_coco_2019_08_14.tar.gz
$ tar -zxvf ssd_mobilenet_v3_large_coco_2019_08_14.tar.gz
$ rm ssd_mobilenet_v3_large_coco_2019_08_14.tar.gz
$ cd ..

2-5-2. Create a conversion script from checkpoint format to saved_model format

import tensorflow as tf
import os
import shutil
from tensorflow.python.saved_model import tag_constants
from tensorflow.python.tools import freeze_graph
from tensorflow.python import ops
from tensorflow.tools.graph_transforms import TransformGraph

def freeze_model(saved_model_dir, output_node_names, output_filename):
  output_graph_filename = os.path.join(saved_model_dir, output_filename)
  initializer_nodes = ''
  freeze_graph.freeze_graph(
      input_saved_model_dir=saved_model_dir,
      output_graph=output_graph_filename,
      saved_model_tags = tag_constants.SERVING,
      output_node_names=output_node_names,
      initializer_nodes=initializer_nodes,
      input_graph=None,
      input_saver=False,
      input_binary=False,
      input_checkpoint=None,
      restore_op_name=None,
      filename_tensor_name=None,
      clear_devices=True,
      input_meta_graph=False,
  )

def get_graph_def_from_file(graph_filepath):
  tf.reset_default_graph()
  with ops.Graph().as_default():
    with tf.gfile.GFile(graph_filepath, 'rb') as f:
      graph_def = tf.GraphDef()
      graph_def.ParseFromString(f.read())
      return graph_def

def optimize_graph(model_dir, graph_filename, transforms, input_name, output_names, outname='optimized_model.pb'):
  input_names = [input_name] # change this as per how you have saved the model
  graph_def = get_graph_def_from_file(os.path.join(model_dir, graph_filename))
  optimized_graph_def = TransformGraph(
      graph_def,
      input_names,
      output_names,
      transforms)
  tf.train.write_graph(optimized_graph_def,
                      logdir=model_dir,
                      as_text=False,
                      name=outname)
  print('Graph optimized!')

def convert_graph_def_to_saved_model(export_dir, graph_filepath, input_name, outputs):
  graph_def = get_graph_def_from_file(graph_filepath)
  with tf.Session(graph=tf.Graph()) as session:
    tf.import_graph_def(graph_def, name='')
    tf.compat.v1.saved_model.simple_save(
        session,
        export_dir,# change input_image to node.name if you know the name
        inputs={input_name: session.graph.get_tensor_by_name('{}:0'.format(node.name))
            for node in graph_def.node if node.op=='Placeholder'},
        outputs={t.rstrip(":0"):session.graph.get_tensor_by_name(t) for t in outputs}
    )
    print('Optimized graph converted to SavedModel!')

tf.compat.v1.enable_eager_execution()

# Look up the name of the placeholder for the input node
graph_def=get_graph_def_from_file('./ssd_mobilenet_v3_small_coco_2019_08_14/frozen_inference_graph.pb')
input_name_small=""
for node in graph_def.node:
    if node.op=='Placeholder':
        print("##### ssd_mobilenet_v3_small_coco_2019_08_14 - Input Node Name #####", node.name) # this will be the input node
        input_name_small=node.name

# Look up the name of the placeholder for the input node
graph_def=get_graph_def_from_file('./ssd_mobilenet_v3_large_coco_2019_08_14/frozen_inference_graph.pb')
input_name_large=""
for node in graph_def.node:
    if node.op=='Placeholder':
        print("##### ssd_mobilenet_v3_large_coco_2019_08_14 - Input Node Name #####", node.name) # this will be the input node
        input_name_large=node.name

# ssd_mobilenet_v3 output names
output_node_names = ['raw_outputs/class_predictions','raw_outputs/box_encodings']
outputs = ['raw_outputs/class_predictions:0','raw_outputs/box_encodings:0']

# Optimizing the graph via TensorFlow library
transforms = []
optimize_graph('./ssd_mobilenet_v3_small_coco_2019_08_14', 'frozen_inference_graph.pb', transforms, input_name_small, output_node_names, outname='optimized_model_small.pb')
optimize_graph('./ssd_mobilenet_v3_large_coco_2019_08_14', 'frozen_inference_graph.pb', transforms, input_name_large, output_node_names, outname='optimized_model_large.pb')

# convert this to a s TF Serving compatible mode - ssd_mobilenet_v3_small_coco_2019_08_14
shutil.rmtree('./ssd_mobilenet_v3_small_coco_2019_08_14/0', ignore_errors=True)
convert_graph_def_to_saved_model('./ssd_mobilenet_v3_small_coco_2019_08_14/0',
                                 './ssd_mobilenet_v3_small_coco_2019_08_14/optimized_model_small.pb', input_name_small, outputs)

# convert this to a s TF Serving compatible mode - ssd_mobilenet_v3_large_coco_2019_08_14
shutil.rmtree('./ssd_mobilenet_v3_large_coco_2019_08_14/0', ignore_errors=True)
convert_graph_def_to_saved_model('./ssd_mobilenet_v3_large_coco_2019_08_14/0',
                                 './ssd_mobilenet_v3_large_coco_2019_08_14/optimized_model_large.pb', input_name_large, outputs)

2-5-3. Confirm the structure of saved_model 【ssd_mobilenet_v3_small_coco_2019_08_14】

$ saved_model_cli show --dir ./ssd_mobilenet_v3_small_coco_2019_08_14/0 --all

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['normalized_input_image_tensor'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 320, 320, 3)
        name: normalized_input_image_tensor:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['raw_outputs/box_encodings'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 2034, 4)
        name: raw_outputs/box_encodings:0
    outputs['raw_outputs/class_predictions'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 2034, 91)
        name: raw_outputs/class_predictions:0
  Method name is: tensorflow/serving/predict

2-5-4. Confirm the structure of saved_model 【ssd_mobilenet_v3_large_coco_2019_08_14】

$ saved_model_cli show --dir ./ssd_mobilenet_v3_large_coco_2019_08_14/0 --all

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['normalized_input_image_tensor'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 320, 320, 3)
        name: normalized_input_image_tensor:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['raw_outputs/box_encodings'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 2034, 4)
        name: raw_outputs/box_encodings:0
    outputs['raw_outputs/class_predictions'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 2034, 91)
        name: raw_outputs/class_predictions:0
  Method name is: tensorflow/serving/predict

2-5-5. Creating the destination path for the calibration test dataset 6GB

$ curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1Uk9F4Tc-9UgnvARIVkloSoePUynyST6E" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1Uk9F4Tc-9UgnvARIVkloSoePUynyST6E" -o TFDS.tar.gz
$ tar -zxvf TFDS.tar.gz
$ rm TFDS.tar.gz

2-5-6. Quantization

2-5-6-1. ssd_mobilenet_v3_small_coco_2019_08_14

import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np

def representative_dataset_gen():
  for data in raw_test_data.take(100):
    image = data['image'].numpy()
    image = tf.image.resize(image, (320, 320))
    image = image[np.newaxis,:,:,:]
    yield [image]

tf.compat.v1.enable_eager_execution()

# Generating a calibration data set
#raw_test_data, info = tfds.load(name="coco/2017", with_info=True, split="test", data_dir="./TFDS")
raw_test_data, info = tfds.load(name="coco/2017", with_info=True, split="test", data_dir="./TFDS", download=False)
print(info)

# Weight Quantization - Input/Output=float32
converter = tf.lite.TFLiteConverter.from_saved_model('./ssd_mobilenet_v3_small_coco_2019_08_14/0')
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
tflite_quant_model = converter.convert()
with open('./ssd_mobilenet_v3_small_coco_2019_08_14/mobilenet_v3_small_weight_quant.tflite', 'wb') as w:
    w.write(tflite_quant_model)
print("Weight Quantization complete! - mobilenet_v3_small_weight_quant.tflite")

# Integer Quantization - Input/Output=float32
converter = tf.lite.TFLiteConverter.from_saved_model('./ssd_mobilenet_v3_small_coco_2019_08_14/0')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
tflite_quant_model = converter.convert()
with open('./ssd_mobilenet_v3_small_coco_2019_08_14/mobilenet_v3_small_integer_quant.tflite', 'wb') as w:
    w.write(tflite_quant_model)
print("Integer Quantization complete! - mobilenet_v3_small_integer_quant.tflite")

# Full Integer Quantization - Input/Output=int8
converter = tf.lite.TFLiteConverter.from_saved_model('./ssd_mobilenet_v3_small_coco_2019_08_14/0')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_quant_model = converter.convert()
with open('./ssd_mobilenet_v3_small_coco_2019_08_14/mobilenet_v3_small_full_integer_quant.tflite', 'wb') as w:
    w.write(tflite_quant_model)
print("Full Integer Quantization complete! - mobilenet_v3_small_full_integer_quant.tflite")

2-5-6-2. ssd_mobilenet_v3_large_coco_2019_08_14

import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np

def representative_dataset_gen():
  for data in raw_test_data.take(100):
    image = data['image'].numpy()
    image = tf.image.resize(image, (320, 320))
    image = image[np.newaxis,:,:,:]
    yield [image]

tf.compat.v1.enable_eager_execution()

# Generating a calibration data set
#raw_test_data, info = tfds.load(name="coco/2017", with_info=True, split="test", data_dir="./TFDS")
raw_test_data, info = tfds.load(name="coco/2017", with_info=True, split="test", data_dir="./TFDS", download=False)

# Weight Quantization - Input/Output=float32
converter = tf.lite.TFLiteConverter.from_saved_model('./ssd_mobilenet_v3_large_coco_2019_08_14/0')
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
tflite_quant_model = converter.convert()
with open('./ssd_mobilenet_v3_large_coco_2019_08_14/mobilenet_v3_large_weight_quant.tflite', 'wb') as w:
    w.write(tflite_quant_model)
print("Weight Quantization complete! - mobilenet_v3_large_weight_quant.tflite")

# Integer Quantization - Input/Output=float32
converter = tf.lite.TFLiteConverter.from_saved_model('./ssd_mobilenet_v3_large_coco_2019_08_14/0')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
tflite_quant_model = converter.convert()
with open('./ssd_mobilenet_v3_large_coco_2019_08_14/mobilenet_v3_large_integer_quant.tflite', 'wb') as w:
    w.write(tflite_quant_model)
print("Integer Quantization complete! - mobilenet_v3_large_integer_quant.tflite")

# Full Integer Quantization - Input/Output=int8
converter = tf.lite.TFLiteConverter.from_saved_model('./ssd_mobilenet_v3_large_coco_2019_08_14/0')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_quant_model = converter.convert()
with open('./ssd_mobilenet_v3_large_coco_2019_08_14/mobilenet_v3_large_full_integer_quant.tflite', 'wb') as w:
    w.write(tflite_quant_model)
print("Full Integer Quantization complete! - mobilenet_v3_large_full_integer_quant.tflite")

2-6. MobileNetV2-SSDLite+VOC - Training -> Integer Quantization

3. TFLite Model Benchmark

$ sudo apt-get install python-future

## Bazel for Ubuntu18.04 x86_64 install
$ wget https://github.com/bazelbuild/bazel/releases/download/2.0.0/bazel-2.0.0-installer-linux-x86_64.sh
$ sudo chmod +x bazel-2.0.0-installer-linux-x86_64.sh
$ ./bazel-2.0.0-installer-linux-x86_64.sh
$ sudo apt-get install -y openjdk-8-jdk

## Bazel for RaspberryPi3/4 Raspbian/Debian Buster armhf install
$ wget https://github.com/PINTO0309/Bazel_bin/raw/main/3.1.0/Raspbian_Debian_Buster_armhf/openjdk-8-jdk/install.sh
$ ./install.sh
$ curl -sc /tmp/cookie \
  "https://drive.google.com/uc?export=download&id=1LQUSal55R6fmawZS9zZuk6-5ZFOdUqRK" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie \
  "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1LQUSal55R6fmawZS9zZuk6-5ZFOdUqRK" \
  -o adoptopenjdk-8-hotspot_8u222-b10-2_armhf.deb
$ sudo apt-get install -y ./adoptopenjdk-8-hotspot_8u222-b10-2_armhf.deb

## Bazel for RaspberryPi3/4 Raspbian/Debian Buster aarch64 install
$ wget https://github.com/PINTO0309/Bazel_bin/raw/main/3.1.0/Raspbian_Debian_Buster_aarch64/openjdk-8-jdk/install.sh
$ ./install.sh
$ curl -sc /tmp/cookie \
  "https://drive.google.com/uc?export=download&id=1VwLxzT3EOTbhSzwvRF2H4ChTQyTQBt3x" > /dev/null
$ CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
$ curl -Lb /tmp/cookie \
  "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1VwLxzT3EOTbhSzwvRF2H4ChTQyTQBt3x" \
  -o adoptopenjdk-8-hotspot_8u222-b10-2_arm64.deb
$ sudo apt-get install -y ./adoptopenjdk-8-hotspot_8u222-b10-2_arm64.deb

## Clone Tensorflow v2.1.0+
$ git clone --depth 1 https://github.com/tensorflow/tensorflow.git
$ cd tensorflow

## Build and run TFLite Model Benchmark Tool
$ bazel run -c opt tensorflow/lite/tools/benchmark:benchmark_model -- \
  --graph=${HOME}/Downloads/deeplabv3_257_mv_gpu.tflite \
  --num_threads=4 \
  --warmup_runs=1 \
  --enable_op_profiling=true

$ bazel run -c opt tensorflow/lite/tools/benchmark:benchmark_model -- \
  --graph=${HOME}/Downloads/deeplabv3_257_mv_gpu.tflite \
  --num_threads=4 \
  --warmup_runs=1 \
  --use_xnnpack=true \
  --enable_op_profiling=true

$ bazel run \
  -c opt \
  --config=noaws \
  --config=nohdfs \
  --config=nonccl \
  tensorflow/lite/tools/benchmark:benchmark_model_plus_flex -- \
  --graph=${HOME}/git/tf-monodepth2/monodepth2_flexdelegate_weight_quant.tflite \
  --num_threads=4 \
  --warmup_runs=1 \
  --enable_op_profiling=true

$ bazel run \
  -c opt \
  --config=noaws \
  --config=nohdfs \
  --config=nonccl \
  tensorflow/lite/tools/benchmark:benchmark_model_plus_flex -- \
  --graph=${HOME}/git/tf-monodepth2/monodepth2_flexdelegate_weight_quant.tflite \
  --num_threads=4 \
  --warmup_runs=1 \
  --use_xnnpack=true \
  --enable_op_profiling=true

x86_64 deeplab_mnv3_small_weight_quant_769.tflite Benchmark

Number of nodes executed: 171
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       45	  1251.486	    67.589%	    67.589%	     0.000	        0
	       DEPTHWISE_CONV_2D	       11	   438.764	    23.696%	    91.286%	     0.000	        0
	              HARD_SWISH	       16	    54.855	     2.963%	    94.248%	     0.000	        0
	                 ARG_MAX	        1	    24.850	     1.342%	    95.591%	     0.000	        0
	         RESIZE_BILINEAR	        5	    23.805	     1.286%	    96.876%	     0.000	        0
	                     MUL	       30	    14.914	     0.805%	    97.682%	     0.000	        0
	                     ADD	       18	    10.646	     0.575%	    98.257%	     0.000	        0
	       SPACE_TO_BATCH_ND	        7	     9.567	     0.517%	    98.773%	     0.000	        0
	       BATCH_TO_SPACE_ND	        7	     7.431	     0.401%	    99.175%	     0.000	        0
	                     SUB	        2	     6.131	     0.331%	    99.506%	     0.000	        0
	         AVERAGE_POOL_2D	       10	     5.435	     0.294%	    99.799%	     0.000	        0
	                 RESHAPE	        6	     2.171	     0.117%	    99.916%	     0.000	        0
	                     PAD	        1	     0.660	     0.036%	    99.952%	     0.000	        0
	                    CAST	        2	     0.601	     0.032%	    99.985%	     0.000	        0
	           STRIDED_SLICE	        1	     0.277	     0.015%	   100.000%	     0.000	        0
	        Misc Runtime Ops	        1	     0.008	     0.000%	   100.000%	    33.552	        0
	              DEQUANTIZE	        8	     0.000	     0.000%	   100.000%	     0.000	        0

Timings (microseconds): count=52 first=224 curr=1869070 min=224 max=2089397 avg=1.85169e+06 std=373988
Memory (bytes): count=0
171 nodes observed

x86_64 deeplab_mnv3_large_weight_quant_769.tflite Benchmark

Number of nodes executed: 194
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       51	  4123.348	    82.616%	    82.616%	     0.000	        0
	       DEPTHWISE_CONV_2D	       15	   628.139	    12.586%	    95.202%	     0.000	        0
	              HARD_SWISH	       15	    90.448	     1.812%	    97.014%	     0.000	        0
	                     MUL	       32	    29.393	     0.589%	    97.603%	     0.000	        0
	                 ARG_MAX	        1	    22.866	     0.458%	    98.061%	     0.000	        0
	                     ADD	       25	    22.860	     0.458%	    98.519%	     0.000	        0
	         RESIZE_BILINEAR	        5	    22.494	     0.451%	    98.970%	     0.000	        0
	       SPACE_TO_BATCH_ND	        8	    18.518	     0.371%	    99.341%	     0.000	        0
	       BATCH_TO_SPACE_ND	        8	    15.522	     0.311%	    99.652%	     0.000	        0
	         AVERAGE_POOL_2D	        9	     7.855	     0.157%	    99.809%	     0.000	        0
	                     SUB	        2	     5.896	     0.118%	    99.928%	     0.000	        0
	                 RESHAPE	        6	     2.133	     0.043%	    99.970%	     0.000	        0
	                     PAD	        1	     0.631	     0.013%	    99.983%	     0.000	        0
	                    CAST	        2	     0.575	     0.012%	    99.994%	     0.000	        0
	           STRIDED_SLICE	        1	     0.260	     0.005%	   100.000%	     0.000	        0
	        Misc Runtime Ops	        1	     0.012	     0.000%	   100.000%	    38.304	        0
	              DEQUANTIZE	       12	     0.003	     0.000%	   100.000%	     0.000	        0

Timings (microseconds): count=31 first=193 curr=5276579 min=193 max=5454605 avg=4.99104e+06 std=1311782
Memory (bytes): count=0
194 nodes observed

Ubuntu 19.10 aarch64 + RaspberryPi4 deeplab_v3_plus_mnv3_decoder_256_integer_quant.tflite Benchmark

Number of nodes executed: 180
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       38	    37.595	    45.330%	    45.330%	     0.000	       38
	                     ADD	       37	    12.319	    14.854%	    60.184%	     0.000	       37
	       DEPTHWISE_CONV_2D	       17	    11.424	    13.774%	    73.958%	     0.000	       17
	         RESIZE_BILINEAR	        4	     7.336	     8.845%	    82.804%	     0.000	        4
	                     MUL	        9	     4.204	     5.069%	    87.873%	     0.000	        9
	                QUANTIZE	       13	     3.976	     4.794%	    92.667%	     0.000	       13
	         AVERAGE_POOL_2D	        9	     1.809	     2.181%	    94.848%	     0.000	        9
	                     DIV	        9	     1.167	     1.407%	    96.255%	     0.000	        9
	                 ARG_MAX	        1	     1.137	     1.371%	    97.626%	     0.000	        1
	           CONCATENATION	        2	     0.780	     0.940%	    98.566%	     0.000	        2
	         FULLY_CONNECTED	       16	     0.715	     0.862%	    99.428%	     0.000	       16
	              DEQUANTIZE	        9	     0.473	     0.570%	    99.999%	     0.000	        9
	                 RESHAPE	       16	     0.001	     0.001%	   100.000%	     0.000	       16

Timings (microseconds): count=50 first=83065 curr=82874 min=82675 max=85743 avg=83036 std=499
Memory (bytes): count=0
180 nodes observed

Ubuntu 19.10 aarch64 + RaspberryPi4 deeplab_v3_plus_mnv2_decoder_256_integer_quant.tflite Benchmark

Number of nodes executed: 81
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       41	    47.427	    65.530%	    65.530%	     0.000	       41
	       DEPTHWISE_CONV_2D	       19	    11.114	    15.356%	    80.887%	     0.000	       19
	         RESIZE_BILINEAR	        4	     7.342	    10.145%	    91.031%	     0.000	        4
	                QUANTIZE	        3	     2.953	     4.080%	    95.112%	     0.000	        3
	                     ADD	       10	     1.633	     2.256%	    97.368%	     0.000	       10
	                 ARG_MAX	        1	     1.137	     1.571%	    98.939%	     0.000	        1
	           CONCATENATION	        2	     0.736	     1.017%	    99.956%	     0.000	        2
	         AVERAGE_POOL_2D	        1	     0.032	     0.044%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=72544 curr=72425 min=72157 max=72745 avg=72412.9 std=137
Memory (bytes): count=0
81 nodes observed

Ubuntu 19.10 aarch64 + RaspberryPi4 mobilenet_v3_small_full_integer_quant.tflite Benchmark

Number of nodes executed: 176
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       61	    10.255	    36.582%	    36.582%	     0.000	       61
	       DEPTHWISE_CONV_2D	       27	     5.058	    18.043%	    54.625%	     0.000	       27
	                     MUL	       26	     5.056	    18.036%	    72.661%	     0.000	       26
	                     ADD	       14	     4.424	    15.781%	    88.442%	     0.000	       14
	                QUANTIZE	       13	     1.633	     5.825%	    94.267%	     0.000	       13
	              HARD_SWISH	       10	     0.918	     3.275%	    97.542%	     0.000	       10
	                LOGISTIC	        1	     0.376	     1.341%	    98.883%	     0.000	        1
	         AVERAGE_POOL_2D	        9	     0.199	     0.710%	    99.593%	     0.000	        9
	           CONCATENATION	        2	     0.084	     0.300%	    99.893%	     0.000	        2
	                 RESHAPE	       13	     0.030	     0.107%	   100.000%	     0.000	       13

Timings (microseconds): count=50 first=28827 curr=28176 min=27916 max=28827 avg=28121.2 std=165
Memory (bytes): count=0
176 nodes observed

Ubuntu 19.10 aarch64 + RaspberryPi4 mobilenet_v3_small_weight_quant.tflite Benchmark

Number of nodes executed: 186
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       61	    82.600	    79.265%	    79.265%	     0.000	       61
	       DEPTHWISE_CONV_2D	       27	     8.198	     7.867%	    87.132%	     0.000	       27
	                     MUL	       26	     4.866	     4.670%	    91.802%	     0.000	       26
	                     ADD	       14	     4.863	     4.667%	    96.469%	     0.000	       14
	                LOGISTIC	        1	     1.645	     1.579%	    98.047%	     0.000	        1
	         AVERAGE_POOL_2D	        9	     0.761	     0.730%	    98.777%	     0.000	        9
	              HARD_SWISH	       10	     0.683	     0.655%	    99.433%	     0.000	       10
	           CONCATENATION	        2	     0.415	     0.398%	    99.831%	     0.000	        2
	                 RESHAPE	       13	     0.171	     0.164%	    99.995%	     0.000	       13
	              DEQUANTIZE	       23	     0.005	     0.005%	   100.000%	     0.000	       23

Timings (microseconds): count=50 first=103867 curr=103937 min=103708 max=118926 avg=104299 std=2254
Memory (bytes): count=0
186 nodes observed

Ubuntu 19.10 aarch64 + RaspberryPi4 Posenet model-mobilenet_v1_101_257_integer_quant.tflite Benchmark

Number of nodes executed: 38
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       18	    31.906	    83.360%	    83.360%	     0.000	        0
	       DEPTHWISE_CONV_2D	       13	     5.959	    15.569%	    98.929%	     0.000	        0
	                QUANTIZE	        1	     0.223	     0.583%	    99.511%	     0.000	        0
	        Misc Runtime Ops	        1	     0.148	     0.387%	    99.898%	    96.368	        0
	              DEQUANTIZE	        4	     0.030	     0.078%	    99.976%	     0.000	        0
	                LOGISTIC	        1	     0.009	     0.024%	   100.000%	     0.000	        0

Timings (microseconds): count=70 first=519 curr=53370 min=519 max=53909 avg=38296 std=23892
Memory (bytes): count=0
38 nodes observed

Ubuntu 19.10 aarch64 + RaspberryPi4 MobileNetV2-SSDLite ssdlite_mobilenet_v2_coco_300_integer_quant.tflite Benchmark

Number of nodes executed: 128
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       55	    27.253	    71.185%	    71.185%	     0.000	        0
	       DEPTHWISE_CONV_2D	       33	     8.024	    20.959%	    92.143%	     0.000	        0
	                     ADD	       10	     1.565	     4.088%	    96.231%	     0.000	        0
	                QUANTIZE	       11	     0.546	     1.426%	    97.657%	     0.000	        0
	        Misc Runtime Ops	        1	     0.368	     0.961%	    98.618%	   250.288	        0
	                LOGISTIC	        1	     0.253	     0.661%	    99.279%	     0.000	        0
	              DEQUANTIZE	        2	     0.168	     0.439%	    99.718%	     0.000	        0
	           CONCATENATION	        2	     0.077	     0.201%	    99.919%	     0.000	        0
	                 RESHAPE	       13	     0.031	     0.081%	   100.000%	     0.000	        0

Timings (microseconds): count=70 first=1289 curr=53049 min=1289 max=53590 avg=38345.2 std=23436
Memory (bytes): count=0
128 nodes observed

Ubuntu 19.10 aarch64 + RaspberryPi4 ml-sound-classifier mobilenetv2_fsd2018_41cls_weight_quant.tflite Benchmark

Number of nodes executed: 111
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 MINIMUM	       35	    10.020	    45.282%	    45.282%	     0.000	       35
	                 CONV_2D	       34	     8.376	    37.852%	    83.134%	     0.000	       34
	       DEPTHWISE_CONV_2D	       18	     1.685	     7.615%	    90.749%	     0.000	       18
	                    MEAN	        1	     1.422	     6.426%	    97.176%	     0.000	        1
	         FULLY_CONNECTED	        2	     0.589	     2.662%	    99.837%	     0.000	        2
	                     ADD	       10	     0.031	     0.140%	    99.977%	     0.000	       10
	                 SOFTMAX	        1	     0.005	     0.023%	   100.000%	     0.000	        1
	              DEQUANTIZE	       10	     0.000	     0.000%	   100.000%	     0.000	       10

Timings (microseconds): count=50 first=22417 curr=22188 min=22041 max=22417 avg=22182 std=70
Memory (bytes): count=0
111 nodes observed

Ubuntu 19.10 aarch64 + RaspberryPi4 ml-sound-classifier mobilenetv2_fsd2018_41cls_integer_quant.tflite Benchmark

Number of nodes executed: 173
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                QUANTIZE	       70	     1.117	    23.281%	    23.281%	     0.000	        0
	                 MINIMUM	       35	     1.104	    23.010%	    46.290%	     0.000	        0
	                 CONV_2D	       34	     0.866	    18.049%	    64.339%	     0.000	        0
	                    MEAN	        1	     0.662	    13.797%	    78.137%	     0.000	        0
	       DEPTHWISE_CONV_2D	       18	     0.476	     9.921%	    88.058%	     0.000	        0
	         FULLY_CONNECTED	        2	     0.251	     5.231%	    93.289%	     0.000	        0
	        Misc Runtime Ops	        1	     0.250	     5.211%	    98.499%	    71.600	        0
	                     ADD	       10	     0.071	     1.480%	    99.979%	     0.000	        0
	                 SOFTMAX	        1	     0.001	     0.021%	   100.000%	     0.000	        0
	              DEQUANTIZE	        1	     0.000	     0.000%	   100.000%	     0.000	        0

Timings (microseconds): count=198 first=477 curr=9759 min=477 max=10847 avg=4876.6 std=4629
Memory (bytes): count=0
173 nodes observed

Raspbian Buster aarch64 + RaspberryPi4 deeplabv3_mnv2_pascal_trainval_257_integer_quant.tflite Benchmark

Number of nodes executed: 82
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       38	   103.576	    56.077%	    56.077%	     0.000	       38
	       DEPTHWISE_CONV_2D	       17	    33.151	    17.948%	    74.026%	     0.000	       17
	         RESIZE_BILINEAR	        3	    15.143	     8.199%	    82.224%	     0.000	        3
	                     SUB	        2	    10.908	     5.906%	    88.130%	     0.000	        2
	                     ADD	       11	     9.821	     5.317%	    93.447%	     0.000	       11
	                 ARG_MAX	        1	     8.824	     4.777%	    98.225%	     0.000	        1
	                     PAD	        1	     1.024	     0.554%	    98.779%	     0.000	        1
	                QUANTIZE	        2	     0.941	     0.509%	    99.289%	     0.000	        2
	                     MUL	        1	     0.542	     0.293%	    99.582%	     0.000	        1
	           CONCATENATION	        1	     0.365	     0.198%	    99.780%	     0.000	        1
	         AVERAGE_POOL_2D	        1	     0.150	     0.081%	    99.861%	     0.000	        1
	                 RESHAPE	        2	     0.129	     0.070%	    99.931%	     0.000	        2
	             EXPAND_DIMS	        2	     0.128	     0.069%	   100.000%	     0.000	        2

Timings (microseconds): count=50 first=201226 curr=176476 min=176476 max=201226 avg=184741 std=4791
Memory (bytes): count=0
82 nodes observed

Ubuntu 18.04 x86_64 + XNNPACK enabled + 10 Threads deeplabv3_257_mv_gpu.tflite Benchmark

Number of nodes executed: 8
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                DELEGATE	        3	     6.716	    61.328%	    61.328%	     0.000	        3
	         RESIZE_BILINEAR	        3	     3.965	    36.207%	    97.534%	     0.000	        3
	           CONCATENATION	        1	     0.184	     1.680%	    99.215%	     0.000	        1
	         AVERAGE_POOL_2D	        1	     0.086	     0.785%	   100.000%	     0.000	        1

Timings (microseconds): count=91 first=11051 curr=10745 min=10521 max=12552 avg=10955.4 std=352
Memory (bytes): count=0
8 nodes observed

Note: as the benchmark tool itself affects memory footprint, the following is only APPROXIMATE to the actual memory footprint of the model at runtime. Take the information at your discretion.
Peak memory footprint (MB): init=3.58203 overall=56.0703

Ubuntu 18.04 x86_64 + XNNPACK disabled + 10 Threads deeplabv3_257_mv_gpu.tflite Benchmark

Number of nodes executed: 70
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	       DEPTHWISE_CONV_2D	       17	    41.704	    68.372%	    68.372%	     0.000	       17
	                 CONV_2D	       38	    15.932	    26.120%	    94.491%	     0.000	       38
	         RESIZE_BILINEAR	        3	     3.060	     5.017%	    99.508%	     0.000	        3
	                     ADD	       10	     0.149	     0.244%	    99.752%	     0.000	       10
	           CONCATENATION	        1	     0.109	     0.179%	    99.931%	     0.000	        1
	         AVERAGE_POOL_2D	        1	     0.042	     0.069%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=59929 curr=60534 min=59374 max=63695 avg=61031.6 std=1182
Memory (bytes): count=0
70 nodes observed

Note: as the benchmark tool itself affects memory footprint, the following is only APPROXIMATE to the actual memory footprint of the model at runtime. Take the information at your discretion.
Peak memory footprint (MB): init=0 overall=13.7109

Ubuntu 18.04 x86_64 + XNNPACK enabled + 4 Threads Faster-Grad-CAM weights_weight_quant.tflite Benchmark

umber of nodes executed: 74
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       31	     4.947	    77.588%	    77.588%	     0.000	       31
	                DELEGATE	       17	     0.689	    10.806%	    88.394%	     0.000	       17
	       DEPTHWISE_CONV_2D	       10	     0.591	     9.269%	    97.663%	     0.000	       10
	                    MEAN	        1	     0.110	     1.725%	    99.388%	     0.000	        1
	                     PAD	        5	     0.039	     0.612%	   100.000%	     0.000	        5
	              DEQUANTIZE	       10	     0.000	     0.000%	   100.000%	     0.000	       10

Timings (microseconds): count=155 first=6415 curr=6443 min=6105 max=6863 avg=6409.22 std=69
Memory (bytes): count=0
74 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads Faster-Grad-CAM weights_integer_quant.tflite Benchmark

Number of nodes executed: 72
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       35	     0.753	    34.958%	    34.958%	     0.000	        0
	                     PAD	        5	     0.395	    18.338%	    53.296%	     0.000	        0
	                    MEAN	        1	     0.392	    18.199%	    71.495%	     0.000	        0
	        Misc Runtime Ops	        1	     0.282	    13.092%	    84.587%	    89.232	        0
	       DEPTHWISE_CONV_2D	       17	     0.251	    11.653%	    96.240%	     0.000	        0
	                     ADD	       10	     0.054	     2.507%	    98.747%	     0.000	        0
	                QUANTIZE	        1	     0.024	     1.114%	    99.861%	     0.000	        0
	              DEQUANTIZE	        2	     0.003	     0.139%	   100.000%	     0.000	        0

Timings (microseconds): count=472 first=564 curr=3809 min=564 max=3950 avg=2188.51 std=1625
Memory (bytes): count=0
72 nodes observed

Ubuntu 18.04 x86_64 + XNNPACK enabled + 4 Threads EfficientNet-lite efficientnet-lite0-fp32.tflite Benchmark

Number of nodes executed: 5
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                DELEGATE	        2	     5.639	    95.706%	    95.706%	     0.000	        2
	         FULLY_CONNECTED	        1	     0.239	     4.056%	    99.762%	     0.000	        1
	         AVERAGE_POOL_2D	        1	     0.014	     0.238%	   100.000%	     0.000	        1
	                 RESHAPE	        1	     0.000	     0.000%	   100.000%	     0.000	        1

Timings (microseconds): count=168 first=5842 curr=5910 min=5749 max=6317 avg=5894.55 std=100
Memory (bytes): count=0
5 nodes observed

Ubuntu 18.04 x86_64 + XNNPACK enabled + 4 Threads EfficientNet-lite efficientnet-lite4-fp32.tflite Benchmark

Number of nodes executed: 5
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                DELEGATE	        2	    33.720	    99.235%	    99.235%	     0.000	        2
	         FULLY_CONNECTED	        1	     0.231	     0.680%	    99.915%	     0.000	        1
	         AVERAGE_POOL_2D	        1	     0.029	     0.085%	   100.000%	     0.000	        1
	                 RESHAPE	        1	     0.000	     0.000%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=32459 curr=34867 min=31328 max=35730 avg=33983.5 std=1426
Memory (bytes): count=0
5 nodes observed

Ubuntu 18.04 x86_64 + XNNPACK enabled + 4 Threads White-box-Cartoonization white_box_cartoonization_weight_quant.tflite Benchmark

Number of nodes executed: 47
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       18	 10731.842	    97.293%	    97.293%	     0.000	       18
	              LEAKY_RELU	       13	   236.792	     2.147%	    99.440%	     0.000	       13
	   TfLiteXNNPackDelegate	       10	    45.534	     0.413%	    99.853%	     0.000	       10
	         RESIZE_BILINEAR	        2	    11.237	     0.102%	    99.954%	     0.000	        2
	                     SUB	        3	     4.053	     0.037%	    99.991%	     0.000	        3
	                     DIV	        1	     0.977	     0.009%	   100.000%	     0.000	        1

Timings (microseconds): count=14 first=10866837 curr=11292015 min=10697744 max=12289882 avg=1.10305e+07 std=406791
Memory (bytes): count=0
47 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads edgetpu_deeplab_257_os16_integer_quant.tflite Benchmark

Number of nodes executed: 91
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       49	    54.679	    58.810%	    58.810%	     0.000	       49
	                     SUB	        2	    11.043	    11.877%	    70.687%	     0.000	        2
	                     ADD	       16	     8.909	     9.582%	    80.269%	     0.000	       16
	                 ARG_MAX	        1	     7.184	     7.727%	    87.996%	     0.000	        1
	         RESIZE_BILINEAR	        3	     6.654	     7.157%	    95.153%	     0.000	        3
	       DEPTHWISE_CONV_2D	       13	     3.409	     3.667%	    98.819%	     0.000	       13
	                     MUL	        1	     0.548	     0.589%	    99.408%	     0.000	        1
	                QUANTIZE	        2	     0.328	     0.353%	    99.761%	     0.000	        2
	                 RESHAPE	        2	     0.162	     0.174%	    99.935%	     0.000	        2
	         AVERAGE_POOL_2D	        1	     0.043	     0.046%	    99.982%	     0.000	        1
	           CONCATENATION	        1	     0.017	     0.018%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=92752 curr=93058 min=92533 max=94478 avg=93021.2 std=274
Memory (bytes): count=0
91 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads edgetpu_deeplab_257_os32_integer_quant.tflite Benchmark

Number of nodes executed: 91
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       49	    39.890	    52.335%	    52.335%	     0.000	       49
	                     SUB	        2	    11.043	    14.488%	    66.823%	     0.000	        2
	                     ADD	       16	     8.064	    10.580%	    77.403%	     0.000	       16
	                 ARG_MAX	        1	     7.011	     9.198%	    86.601%	     0.000	        1
	         RESIZE_BILINEAR	        3	     6.623	     8.689%	    95.290%	     0.000	        3
	       DEPTHWISE_CONV_2D	       13	     2.503	     3.284%	    98.574%	     0.000	       13
	                     MUL	        1	     0.544	     0.714%	    99.288%	     0.000	        1
	                QUANTIZE	        2	     0.313	     0.411%	    99.698%	     0.000	        2
	                 RESHAPE	        2	     0.178	     0.234%	    99.932%	     0.000	        2
	         AVERAGE_POOL_2D	        1	     0.041	     0.054%	    99.986%	     0.000	        1
	           CONCATENATION	        1	     0.011	     0.014%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=75517 curr=75558 min=75517 max=97776 avg=76262.5 std=3087
Memory (bytes): count=0
91 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads human_pose_estimation_3d_0001_256x448_integer_quant.tflite Benchmark

Number of nodes executed: 165
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       69	   343.433	    78.638%	    78.638%	     0.000	       69
	                     PAD	       38	    51.637	    11.824%	    90.462%	     0.000	       38
	       DEPTHWISE_CONV_2D	       14	    15.306	     3.505%	    93.967%	     0.000	       14
	                     ADD	       15	    14.535	     3.328%	    97.295%	     0.000	       15
	                     ELU	        6	     5.071	     1.161%	    98.456%	     0.000	        6
	                QUANTIZE	       11	     4.481	     1.026%	    99.482%	     0.000	       11
	              DEQUANTIZE	        9	     1.851	     0.424%	    99.906%	     0.000	        9
	           CONCATENATION	        3	     0.410	     0.094%	   100.000%	     0.000	        3

Timings (microseconds): count=50 first=425038 curr=423469 min=421348 max=969226 avg=436808 std=77255
Memory (bytes): count=0
165 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads + BlazeFace face_detection_front_128_integer_quant.tflite Benchmark

Number of nodes executed: 79
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                     ADD	       16	     2.155	    34.120%	    34.120%	     0.000	       16
	                 CONV_2D	       21	     2.017	    31.935%	    66.054%	     0.000	       21
	                     PAD	       11	     1.014	    16.054%	    82.109%	     0.000	       11
	       DEPTHWISE_CONV_2D	       16	     0.765	    12.112%	    94.221%	     0.000	       16
	                QUANTIZE	        4	     0.186	     2.945%	    97.166%	     0.000	        4
	             MAX_POOL_2D	        3	     0.153	     2.422%	    99.588%	     0.000	        3
	              DEQUANTIZE	        2	     0.017	     0.269%	    99.857%	     0.000	        2
	           CONCATENATION	        2	     0.006	     0.095%	    99.952%	     0.000	        2
	                 RESHAPE	        4	     0.003	     0.047%	   100.000%	     0.000	        4

Timings (microseconds): count=144 first=6415 curr=6319 min=6245 max=6826 avg=6359.12 std=69
Memory (bytes): count=0
79 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads + ssd_mobilenet_v2_mnasfpn_shared_box_predictor_320_coco_integer_quant.tflite Benchmark

Number of nodes executed: 588
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	      119	   109.253	    52.671%	    52.671%	     0.000	      119
	       DEPTHWISE_CONV_2D	       61	    33.838	    16.313%	    68.984%	     0.000	       61
	TFLite_Detection_PostProcess	        1	    22.711	    10.949%	    79.933%	     0.000	        1
	                LOGISTIC	        1	    17.696	     8.531%	    88.465%	     0.000	        1
	                     ADD	       59	    12.300	     5.930%	    94.395%	     0.000	       59
	                 RESHAPE	        8	     4.175	     2.013%	    96.407%	     0.000	        8
	           CONCATENATION	        2	     3.416	     1.647%	    98.054%	     0.000	        2
	 RESIZE_NEAREST_NEIGHBOR	       12	     1.873	     0.903%	    98.957%	     0.000	       12
	             MAX_POOL_2D	       13	     1.363	     0.657%	    99.614%	     0.000	       13
	                     MUL	       16	     0.737	     0.355%	    99.970%	     0.000	       16
	              DEQUANTIZE	      296	     0.063	     0.030%	   100.000%	     0.000	      296

Timings (microseconds): count=50 first=346007 curr=196005 min=192539 max=715157 avg=207709 std=75605
Memory (bytes): count=0
588 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads + object_detection_3d_chair_640x480_integer_quant.tflite Benchmark

Number of nodes executed: 126
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       60	   146.537	    63.805%	    63.805%	     0.000	       60
	       DEPTHWISE_CONV_2D	       26	    45.022	    19.604%	    83.409%	     0.000	       26
	                     ADD	       23	    23.393	    10.186%	    93.595%	     0.000	       23
	          TRANSPOSE_CONV	        3	     9.930	     4.324%	    97.918%	     0.000	        3
	                QUANTIZE	        5	     3.103	     1.351%	    99.269%	     0.000	        5
	           CONCATENATION	        4	     1.541	     0.671%	    99.940%	     0.000	        4
	              DEQUANTIZE	        3	     0.117	     0.051%	    99.991%	     0.000	        3
	                     EXP	        1	     0.018	     0.008%	    99.999%	     0.000	        1
	                     NEG	        1	     0.002	     0.001%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=218224 curr=217773 min=217174 max=649357 avg=229732 std=62952
Memory (bytes): count=0
126 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads + ssdlite_mobiledet_cpu_320x320_coco_integer_quant.tflite Benchmark

Number of nodes executed: 288
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       96	    22.996	    33.342%	    33.342%	     0.000	       96
	              HARD_SWISH	       57	    11.452	    16.604%	    49.946%	     0.000	       57
	                     MUL	       19	     9.423	    13.662%	    63.608%	     0.000	       19
	         AVERAGE_POOL_2D	       19	     8.439	    12.236%	    75.843%	     0.000	       19
	       DEPTHWISE_CONV_2D	       35	     7.810	    11.324%	    87.167%	     0.000	       35
	TFLite_Detection_PostProcess	        1	     5.650	     8.192%	    95.359%	     0.000	        1
	                     ADD	       12	     1.690	     2.450%	    97.809%	     0.000	       12
	                QUANTIZE	       12	     0.879	     1.274%	    99.084%	     0.000	       12
	                LOGISTIC	       20	     0.277	     0.402%	    99.485%	     0.000	       20
	              DEQUANTIZE	        2	     0.234	     0.339%	    99.825%	     0.000	        2
	           CONCATENATION	        2	     0.079	     0.115%	    99.939%	     0.000	        2
	                 RESHAPE	       13	     0.042	     0.061%	   100.000%	     0.000	       13

Timings (microseconds): count=50 first=69091 curr=68590 min=68478 max=83971 avg=69105.3 std=2147
Memory (bytes): count=0
288 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads + mobilenet_v2_pose_256_256_dm100_integer_quant.tflite Benchmark

Number of nodes executed: 189
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       86	    51.819	    70.575%	    70.575%	     0.000	       86
	       DEPTHWISE_CONV_2D	       73	    18.207	    24.797%	    95.372%	     0.000	       73
	                     ADD	        8	     1.243	     1.693%	    97.065%	     0.000	        8
	                QUANTIZE	       13	     1.132	     1.542%	    98.607%	     0.000	       13
	           CONCATENATION	        7	     0.607	     0.827%	    99.433%	     0.000	        7
	         RESIZE_BILINEAR	        1	     0.354	     0.482%	    99.916%	     0.000	        1
	              DEQUANTIZE	        1	     0.062	     0.084%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=73752 curr=73430 min=73191 max=75764 avg=73524.8 std=485
Memory (bytes): count=0
189 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads + mobilenet_v2_pose_368_432_dm100_integer_quant.tflite Benchmark

Number of nodes executed: 189
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       86	   141.296	    69.289%	    69.289%	     0.000	       86
	       DEPTHWISE_CONV_2D	       73	    53.244	    26.110%	    95.399%	     0.000	       73
	                QUANTIZE	       13	     3.059	     1.500%	    96.899%	     0.000	       13
	                     ADD	        8	     3.014	     1.478%	    98.377%	     0.000	        8
	           CONCATENATION	        7	     2.302	     1.129%	    99.506%	     0.000	        7
	         RESIZE_BILINEAR	        1	     0.852	     0.418%	    99.924%	     0.000	        1
	              DEQUANTIZE	        1	     0.155	     0.076%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=189613 curr=579873 min=189125 max=579873 avg=204021 std=70304
Memory (bytes): count=0
189 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads + mobilenet_v2_pose_256_256_dm050_integer_quant.tflite Benchmark

Number of nodes executed: 189
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       86	    40.952	    71.786%	    71.786%	     0.000	       86
	       DEPTHWISE_CONV_2D	       73	    13.508	    23.679%	    95.465%	     0.000	       73
	                QUANTIZE	       13	     1.123	     1.969%	    97.434%	     0.000	       13
	                     ADD	        8	     0.710	     1.245%	    98.678%	     0.000	        8
	           CONCATENATION	        7	     0.498	     0.873%	    99.551%	     0.000	        7
	         RESIZE_BILINEAR	        1	     0.193	     0.338%	    99.890%	     0.000	        1
	              DEQUANTIZE	        1	     0.063	     0.110%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=57027 curr=57048 min=56773 max=58042 avg=57135 std=229
Memory (bytes): count=0
189 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads + mobilenet_v2_pose_368_432_dm050_integer_quant.tflite Benchmark

Number of nodes executed: 189
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       86	   104.618	    71.523%	    71.523%	     0.000	       86
	       DEPTHWISE_CONV_2D	       73	    34.527	    23.605%	    95.128%	     0.000	       73
	                QUANTIZE	       13	     2.572	     1.758%	    96.886%	     0.000	       13
	           CONCATENATION	        7	     2.257	     1.543%	    98.429%	     0.000	        7
	                     ADD	        8	     1.683	     1.151%	    99.580%	     0.000	        8
	         RESIZE_BILINEAR	        1	     0.460	     0.314%	    99.894%	     0.000	        1
	              DEQUANTIZE	        1	     0.155	     0.106%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=172545 curr=146065 min=145260 max=172545 avg=146362 std=3756
Memory (bytes): count=0
189 nodes observed

RaspberryPi4 + Ubuntu 19.10 aarch64 + 4 Threads + yolov4_tiny_voc_416x416_integer_quant.tflite Benchmark

Number of nodes executed: 71
============================== Summary by node type ==============================
	             [Node type]	  [count]	  [avg ms]	    [avg %]	    [cdf %]	  [mem KB]	[times called]
	                 CONV_2D	       21	   149.092	    61.232%	    61.232%	     0.000	       21
	              LEAKY_RELU	       19	    77.644	    31.888%	    93.121%	     0.000	       19
	                     PAD	        2	     8.036	     3.300%	    96.421%	     0.000	        2
	                QUANTIZE	       10	     4.580	     1.881%	    98.302%	     0.000	       10
	           CONCATENATION	        7	     2.415	     0.992%	    99.294%	     0.000	        7
	             MAX_POOL_2D	        3	     0.982	     0.403%	    99.697%	     0.000	        3
	                   SPLIT	        3	     0.615	     0.253%	    99.950%	     0.000	        3
	              DEQUANTIZE	        2	     0.082	     0.034%	    99.984%	     0.000	        2
	 RESIZE_NEAREST_NEIGHBOR	        1	     0.032	     0.013%	    99.997%	     0.000	        1
	           STRIDED_SLICE	        1	     0.004	     0.002%	    99.998%	     0.000	        1
	                     MUL	        1	     0.004	     0.002%	   100.000%	     0.000	        1
	                   SHAPE	        1	     0.000	     0.000%	   100.000%	     0.000	        1

Timings (microseconds): count=50 first=233307 curr=233318 min=232446 max=364068 avg=243522 std=33354
Memory (bytes): count=0
71 nodes observed

4. Reference articles

pinto_model_zoo's People

Contributors

Stargazers

Watchers

Forkers

satoshirobatofujimoto intflow beosro chomolungma ragnariock hongqirun tonystark-001 zumbalamambo dun933 kyuhyoung xinsuinizhuan khanfarhan10 marvis lilujunai tuyaliang changerzz vehery muslihhakim jireh-father lovehuanhuan tricoffee 5l1v3r1 starstylesky romstriker umeshsati54 caneat karaage0703 paranoidrabbid eyebies qifei123 doitdodo feolcn mstaryi tyoungroy sushanthcheerla99 qodatecnologia masoudhassani tarsyer-hemant tuskaw zeta1999 komori556 chenyanyin gmonstriky zlheos sinianyutian sysau observantlabs hongchow saikiran321 devin-coder logdogvip rishiredd789 qaz734913414 xiaooquanwu saccadic sunilsurineni krisurnok davorjordacevic ghrahul moses5407 neixlo vidovalianto usama3059 ducbx yoline777 freehawkzk dlyshare light-reflection alex89607 mah110mat kylinyee black-yj lachose1 cjtang cvhadessun pvsemk benbro danielmao2015 gipsyblues thiwankajayasiri sunwayforever elton-lgb mejimaru chenfang0927 deepvegchicken miaorain linkapp-github jinwook-shim tonydamage ssikkiss quhb2455 templeblock swipswaps jjkennedy3 pandinosaurus kang-hana hidetoshi-furukawa azuredsky insad dev233

pinto_model_zoo's Issues

Is the blazeface_tflite_to_pb.py script supporting for converting quant TFLite model to pb

I found that the blazeface_tflite_to_pb.py script works for some float TFLite model. How about the quant TFLite model to pb?
thanks

yolov4-tiny with RaspberryPi

Hello,
It seems you have great performance with yolo tiny with an RPI :
RaspberryPi4 + CPU only + INT8 + Tensorflow Lite (4 threads) + 416x416 with 243ms/inference Performance.

I assume you used the following command :

bazel run -c opt tensorflow/lite/tools/benchmark:benchmark_model -- \
  --graph=${HOME}/Downloads/yolov4-tiny.tflite \
  --num_threads=4 \
  --warmup_runs=1 \
  --enable_op_profiling=true

However, how can I get similar results for but with my own images and classes? Do you have a repo for performing object detection optimised for RPI (aarch64/armhf)?

Also, I saw you have special wheels for TensorFlow in aarch64 (https://github.com/PINTO0309/Tensorflow-bin), but do you have the same for other packages like opencv in aarch64?

For https://github.com/PINTO0309/PINTO_model_zoo/tree/master/036_Objectron/08_coreml when I use it in ios project I get this error

For https://github.com/PINTO0309/PINTO_model_zoo/tree/master/036_Objectron/08_coreml/sneakers.ml model an error came up when I tried to use it in ios project using this code https://github.com/makeml-app/Live-Object-Recognition-CoreML saying "The model does not have a valid input feature of type image" UserInfo={NSLocalizedDescription=The model does not have a valid input feature of type image}

In this model can you rename "input" in inputs column to "image__0" and give me updated model. It will be a great help. Thanks in advance."

018_EfficientDet Model outputs?

Hi, I am trying to use your efficientDet model, but I am clueless as to what the output is supposed to be? afaik the output ia (1,100,7) I assume its 100 detections, and 7 something?
Here is what I am seeing from output:

output_data = self.model.get_tensor(self.output_details[0]['index'])
output_data.shape: (1, 100, 7)
[[[ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]
  [ 0.        0.        0.       20.931305 20.931305  0.       62.793915]]]

Can you explain what I am looking at?

repository structure

Hi again!

I've kept playing with all the amazing models you're collecting in the zoo repository, but lately, as the repository keeps growing, it's becoming more difficult to find models.

I would like to suggest you to reorganize the repository so it's arranged as themes

So far I would suggest these root repositories:

Artistic
Face Detection
Hand Detection
Body Pose Detection
Object Detection
Depth from monocular images
Misc

That way it would be easier to find and compare different models...

I got the latest facemesh files from Google ARCore, you can try to put them in your project

facemesh_from_google_arcore.zip

[EdgeTPU] RuntimeError: Found too many dimensions in the input array of operation 'reshape'.

I have experimented with a few models in the repo and run into this error several times. Namely, YOLOv3-lite and the mobilenet-MNSFPN compiled using the edge-tpu compiler result in this error when run using the DetectionEngine. Is this some error on my part, or have you also found the same errors?

P.S. the compilation works successfully

BlazeFace confidence score

Excellent translation of models! But how can I get the correct confidence score?
My code

def load_graph(frozen_graph_filename):
    detection_graph = tf.Graph()
    with detection_graph.as_default():
        od_graph_def = tf.GraphDef()
        with tf.gfile.GFile(frozen_graph_filename, 'rb') as f:
            serialized_graph = f.read()
            od_graph_def.ParseFromString(serialized_graph)
            tf.import_graph_def(od_graph_def, name='')

    return detection_graph

detection_face = load_graph('face_detection_front.pb')

img_inputs_face = detection_face.get_tensor_by_name('input:0')


classificators = detection_face.get_tensor_by_name('classificators:0')
regressors = detection_face.get_tensor_by_name('regressors:0')

img = cv2.imread('2-1.jpg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img, (128, 128))
image_np_expanded = np.expand_dims(img, axis=0)

(pred_classificators , pred_regressors) = persistent_sess.run([classificators , regressors],
        feed_dict={img_inputs_face: image_np_expanded})

I get very strange pred_classificators

[[-1.16985550e+01],
       [-9.47585297e+00],
       [-9.12463665e+00],
       [-5.20070457e+00],
       [-1.55940714e+01],
       [-1.01529875e+01],
       [-1.26312046e+01],
       [-4.64342594e+00],
       [-1.42407932e+01],
       [-7.08438396e+00],
       [-1.52337151e+01],
       .....]

Running inference using

Hi, I tried to run inference using a few edge TPU models from your model zoo, and faced the following errors. I was wondering if you could share the code script you used to run inference on mobile devices. Otherwise, It would be appreciated if you could help me solve these errors. Thank you so much.

Running on Odroid with coral usb acceleartor, ubuntu 18.01
Trying to detect person

Error1: `ssd_mobilenet_v2_mnasfpn_shared_box_predictor_320_coco_full_integer_quant_edgetpu.tflite``

Traceback (most recent call last):
  File "one_ch_pd_app.py", line 62, in <module>
    init_inference()
  File "/root/Detection/person_detection.py", line 115, in init_inference
    interpreter = common.make_interpreter(input_model)
  File "/root/Detection/common.py", line 28, in make_interpreter
    {'device': device[0]} if device else {})
  File "/usr/local/lib/python3.5/dist-packages/tflite_runtime/interpreter.py", line 204, in __init__
    model_path, self._custom_op_registerers))
ValueError: Found too many dimensions in the input array of operation 'reshape'.

Error2: `yolov4_416_full_integer_quant_edgetpu.tflite`

Traceback (most recent call last):
  File "one_ch_pd_app.py", line 77, in <module>
    objs = run_inference(THRESHOLD, 10, frame) # threshold, top_k, frame
  File "/root/Detection/person_detection.py", line 126, in run_inference
    interpreter.invoke()
  File "/usr/local/lib/python3.5/dist-packages/tflite_runtime/interpreter.py", line 506, in invoke
    self._interpreter.Invoke()
  File "/usr/local/lib/python3.5/dist-packages/tflite_runtime/interpreter_wrapper.py", line 113, in Invoke
    return _interpreter_wrapper.InterpreterWrapper_Invoke(self)
RuntimeError: Only float32 and uint8 is supported currently, got INT8.Node number 404 (LEAKY_RELU) failed to invoke.

Error3: `ssd_mobilenet_v2_oid_v4_300x300_full_integer_quant_edgetpu.tflite`

I could run the inference, but the bounding box was not shown on the screen. I think it could be because the id (class label) for person is different in "open image dataset" but not sure what is the class label for person here

Error4: `ssdlite_mobilenet_v2_coco_300_full_integer_quant_edgetpu.tflite`

Traceback (most recent call last):
  File "one_ch_pd_app.py", line 77, in <module>
    objs = run_inference(THRESHOLD, 10, frame) # threshold, top_k, frame
  File "/root/Detection/person_detection.py", line 127, in run_inference
    objs = get_output(score_threshold=threshold, top_k=top_k)
  File "/root/Detection/person_detection.py", line 83, in get_output
    scores = common.output_tensor(interpreter, 2)
  File "/root/Detection/common.py", line 48, in output_tensor
    output_details = interpreter.get_output_details()[i]
IndexError: list index out of range

Unhandled "Dequantize" operation in new MediaPipe BlazeFace model

I tried to run your BlazeFace conversion script on the most recent BlazeFace model in MediaPipe. I noticed that the dequantize operation (builtin_code = 6) was not handled in your script. I tried to edit the script to handle this, but I am not at all familiar with TensorFlow, so I am not sure what this operation maps to. Would you mind provide some advice? Thank you in advance!

(I may have used the wrong FlatBuffer definitions so the following operator_codes json might not look right. )

Model: https://github.com/google/mediapipe/blob/f15da632dec186f2c1d3c780f47086477e2286a9/mediapipe/models/face_detection_front.tflite

...
  "operator_codes": [
    {
      "deprecated_builtin_code": 3,
      "version": 1,
      "builtin_code": "ADD"
    },
    {
      "deprecated_builtin_code": 19,
      "version": 1,
      "builtin_code": "ADD"
    },
    {
      "deprecated_builtin_code": 4,
      "version": 1,
      "builtin_code": "ADD"
    },
    {
      "deprecated_builtin_code": 0,
      "version": 1,
      "builtin_code": "ADD"
    },
    {
      "deprecated_builtin_code": 34,
      "version": 1,
      "builtin_code": "ADD"
    },
    {
      "deprecated_builtin_code": 17,
      "version": 1,
      "builtin_code": "ADD"
    },
    {
      "deprecated_builtin_code": 22,
      "version": 1,
      "builtin_code": "ADD"
    },
    {
      "deprecated_builtin_code": 2,
      "version": 1,
      "builtin_code": "ADD"
    },
    {
      "deprecated_builtin_code": 6,
      "version": 2,
      "builtin_code": "ADD"
    }
  ],
...

BlazePose frozen model

Sorry to bother you again with this... I guess you're already working on converting the BlazePose TFLite models to Frozen model as long as saved model?

I've successfully ran the TfLite models, and even in desktop and using only CPU the models are very fast and responsive, it's definitely a much better solution than the PoseNet models.

My understanding is that PoseNet models are better suited for one shot detection, whereas BlazePose is better for continuous tracking, which makes it very good for realtime applications.

Mobilenet V3 + SSD

Hey there,
First of all, thank you for sharing your work.

I'm trying to train MobilenetV3_SSD on my own dataset and then get the tflite.
However, I cannot use TF Object Detection API for quantization aware training.
Seems like there are some issues with their published checkpoints.
I found people having the same problem on Github. here and here.

However, it looks like you were able to fix it. May I ask what method or modification did you use?

Thank you in advance.

BlazePose Output

Hi! Thank you for your great work!
I'm confused about BlazePose Output where (156,) numbers like
[ 2.52126999e+02, 6.02675476e+01, 0.00000000e+00, 7.85891479e+02, 1.81462738e+02, 1.63871140e+02, 0.00000000e+00, 7.20563904e+02, 1.45734039e+02, 1.29187439e+02, 0.00000000e+00, 6.98395630e+02, 2.13491409e+02, 1.06495354e+02, 0.00000000e+00, 6.22234802e+02, 2.00276703e+02, -8.05518913e+00, 0.00000000e+00, 8.24156921e+02, 2.26737579e+02, -6.08142509e+01, 0.00000000e+00, 8.02759888e+02, 3.44873230e+02, 1.73129330e+01, 0.00000000e+00, 7.70141296e+02, 1.80556412e+02, 2.63250000e+02, 0.00000000e+00, 5.80166565e+02, 3.85322113e+02, -1.70522858e+02, 0.00000000e+00, 7.14230835e+02, 6.90424728e+01, 1.44556747e+02, 0.00000000e+00, 6.49112549e+02, 1.49639694e+02, 1.06226685e+02, 0.00000000e+00, 7.19676697e+02, 5.28476562e+02, 3.08210297e+02, -1.69180872e+03, 5.81274231e+02, 1.55742020e+02, 3.07312897e+02, -1.85214661e+03, 5.41536011e+02, 9.28919907e+01, 8.45592224e+02, 1.27794141e+03, 2.13438950e+02, 1.48164871e+02, 5.72311401e+02, 1.61038025e+03, 3.95598450e+02, 3.03308502e+02, 3.77423492e+02, -7.68191895e+02, 2.65322083e+02, 4.57906586e+02, 2.28914776e+01, -1.81853455e+03, 4.70178528e+02, 6.31962402e+02, 2.88600311e+02, 0.00000000e+00, 2.67264465e+02, 5.21761108e+02, -1.34646225e+02, 0.00000000e+00, 4.36134796e+02, 5.85655136e+01, -6.74091949e+01, 0.00000000e+00, 2.82592468e+02, 6.13506226e+02, -1.69087173e+02, 0.00000000e+00, 5.03378204e+02, -7.62357254e+01, -9.83965836e+01, 0.00000000e+00, 3.04275391e+02, 6.24550049e+02, -3.54933510e+01, 0.00000000e+00, 4.93076843e+02, 5.50927551e+02, 3.69620392e+02, 7.18008652e+01, 4.75950956e+00, 2.44607971e+02, 4.69858917e+02, -8.90234451e+01, -1.11216354e+02, 2.67619415e+02, 5.19302246e+02, 2.35148270e+02, 2.89389679e+02, 3.65628471e+01, 3.67190094e+02, 4.64473145e+02, 8.65795364e+01, -1.73825119e+02, 6.05172424e+02, 1.30989319e+03, 2.43793793e+02, 1.52740601e+02, 8.13551758e+02, 1.99037500e+03, 1.52300385e+02, -1.58977951e+02, 6.85409485e+02, 0.00000000e+00, 2.29356506e+02, 5.41917419e+02, 8.15468506e+02, 0.00000000e+00, 1.49431351e+02, 1.45869461e+02, 8.23752747e+02, 0.00000000e+00, 2.91660767e+02, 3.70319244e+02, 1.30325195e+03, 0.00000000e+00, 2.07150436e+02, 4.17732330e+02, 4.39149231e+02, 1.05031455e+00, 1.83442783e+01, 8.89772263e+01, -2.31506516e+02, -7.93924236e+00, 2.01322327e+01, 3.84161530e+02, -4.48814926e+01, 9.11639392e-01, 2.05853485e+02, 2.75295074e+02, 2.07653046e+02, -2.28759232e+01, 2.15975464e+02, 5.05869934e+02, -4.85139046e+01, -1.24414492e+01, 5.11680725e+02, 3.77195312e+02, 9.83366470e+01, -2.68878841e+01, 5.35371887e+02]
I understand that in some way they describe each of 39kp, but they don't have normalization and I don't understant their meaning. Would you please help with it? How convert they in usual coordinates? Or what do they mean? Thanks!

Model resulted in Nan value during calibration.

Code (just a demo how I do quantize and it can't reproduce error)

def representative_dataset_gen():
    for x in validation_fingerprints:
      x = x[np.newaxis,:]
      yield [x]

converter = tf.lite.TFLiteConverter.from_saved_model(flags.train_dir + '/last_model')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
# converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS]
converter.allow_custom_ops = True
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
converter.representative_dataset = representative_dataset_gen
last_quant_model = converter.convert()
with open(flags.train_dir + '/quant_last_model.tflite', 'wb') as w:
  w.write(last_quant_model)

Some config
type(validation_fingerprints): <class 'numpy.ndarray'>
shape(validation_fingerprints): (3093, 16384)
type(x): <class 'numpy.ndarray'>
shape(x): (1,16384)
The model_summary
model_summary.txt

Validation_fingerprints is np.float32. I don't know if it would cause problem in full integer quantization. (I found that 4-2-6-7. Full Integer Quantization from saved_model (All 8-bit integer quantization) also use np.float32 tho.)
I've also found this issue but setting fused=False in batch norm doesn't help. Is there any advice? Stuck this for several days 😢

Trying to understand output of 25 head pose estimator

Hi again!

I am trying to use 25 head pose estimator; I'be been able to load and run the frozen_inference_graph.pb model, but I don't know how to decode the tensor output.

The model has an image input of 128x128 pixels.... and outputs an array of 136 float values.

The python code seems to imply there's several output tensors: scores, boxes, etc.... but I only see this 136 float output tensor.

So, how do I decode it?

As a side note: I am looking for a fast way of detecting faces on an image, at distances of up to 5 metres, so BlazeFace is not suitable for me. I don't need landmarks or any other kind of information, just face rectangles on an image.... given you have a collection of face detection models, which one do you think is the best for that task?

Thanks in advance!

Yolov4-Tiny EdgeTPU Missing Leaky Relu

Hello,
How were you able to execute a Yolo on the Coral with Leaky Relu? We are using tfLite interpreter with "libedgetpu.so.1.0" and we are getting errors.
self.interpreter = tf.lite.Interpreter( hw_model_path, experimental_delegates=[load_delegate('libedgetpu.so.1.0')] )
main.ERROR - Only float32 and uint8 is supported currently, got INT8.Node number 404 (LEAKY_RELU) failed to invoke.

DeepLabv3+ and OpenVINO

Hi @PINTO0309 ! Thanks for this amazing repo. Congrats!.

I'm looking for a way to run DeepLabv3+ with OpenVINO. I see that in your repository you have several folders related to this model. It seems that the one that best fits my needs is Sample.4 - Semantic Segmentation, DeeplabV3-plus 256x256. But I would like to do the development simply by calling the executable files and the XML, as in the typical OpenVINO examples. This example requires TensorFlow installation. What about OpenVINO-bin? I did not find a lot of documentation about this latter one.
I am using RPi4 and a OAK-D camera.

Thanks for your help.

MobileBERT model for Openvino?

Any plans for MobileBERT model for Openvino?

EfficientDet INT8 Quantization

Hi @PINTO0309,

Did you manage to fully quantize the EfficientDet models? I saw there are some quantized models in your zoo: https://github.com/PINTO0309/PINTO_model_zoo/tree/master/18_EfficientDet/03_integer_quantization, but they look more like only weight quantized, right?
I think you didn't add:
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
to limit the operators to be INT8?

I tried myself to fully quantize EfficientDet, and I failed with TF1.15.0, TF2.2.0 and TF2.3.0.

If you've managed to quantize the model, could you give me some enlightenment on this?

Thank you!

Is there an easy way to convert ONNX or PB from (NCHW) to (NHWC)?

@PINTO0309 Hi,
Nice work with YOLOv4 / tiny!

As I see you use:

NCHW for: OpenVINO (xml / bin), Darknet (cfg / weights)
NHWC for: TFLite, Keras (yolov4_tiny_voc.json / yolov4_tiny_voc.h5), TF1 (pb), TF2 (saved_models.json / saved_models.pb)

I have several questions:

Is there an easy way to convert ONNX or PB from (NCHW) to (NHWC)?
I've seen converters that add transpose before and after each layer, but this seems to slow things down a lot. Is it possible to do this transformation without slowing down the inference?
Is there an easy way to convert TF1-pb to TF2-saved_models.pb ?
Is NHWC slowing down execution on the GPU?
How many FPS do you get on Google Coral TPU-Edge and RaspberryPi4 for yolov4-tiny (int8)?
What script did you use to get yolov4_tiny_voc.json ?

Can not load saved BlazePose model.

model = keras.models.load_model('saved_model_pose_detection')

TypeError Traceback (most recent call last)
in
----> 1 model = keras.models.load_model('saved_model_pose_detection')

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\saving\save.py in load_model(filepath, custom_objects, compile)
188 if isinstance(filepath, six.string_types):
189 loader_impl.parse_saved_model(filepath)
--> 190 return saved_model_load.load(filepath, compile)
191
192 raise IOError(

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\saving\saved_model\load.py in load(path, compile)
114 # TODO(kathywu): Add saving/loading of optimizer, compiled losses and metrics.
115 # TODO(kathywu): Add code to load from objects that contain all endpoints
--> 116 model = tf_load.load_internal(path, loader_cls=KerasObjectLoader)
117
118 # pylint: disable=protected-access

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\saved_model\load.py in load_internal(export_dir, tags, loader_cls)
602 loader = loader_cls(object_graph_proto,
603 saved_model_proto,
--> 604 export_dir)
605 root = loader.get(0)
606 if isinstance(loader, Loader):

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\saving\saved_model\load.py in init(self, *args, **kwargs)
186 self._models_to_reconstruct = []
187
--> 188 super(KerasObjectLoader, self).init(*args, **kwargs)
189
190 # Now that the node object has been fully loaded, and the checkpoint has

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\saved_model\load.py in init(self, object_graph_proto, saved_model_proto, export_dir)
121 self._concrete_functions[name] = _WrapperFunction(concrete_function)
122
--> 123 self._load_all()
124 self._restore_checkpoint()
125

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\saving\saved_model\load.py in _load_all(self)
207 # loaded from config may create variables / other objects during
208 # initialization. These are recorded in _nodes_recreated_from_config.
--> 209 self._layer_nodes = self._load_layers()
210
211 # Load all other nodes and functions.

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\saving\saved_model\load.py in _load_layers(self)
307 continue
308
--> 309 layers[node_id] = self._load_layer(proto.user_object, node_id)
310
311 for node_id, proto in metric_list:

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\saving\saved_model\load.py in _load_layer(self, proto, node_id)
333 # Detect whether this object can be revived from the config. If not, then
334 # revive from the SavedModel instead.
--> 335 obj, setter = self._revive_from_config(proto.identifier, metadata, node_id)
336 if obj is None:
337 obj, setter = revive_custom_object(proto.identifier, metadata)

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\saving\saved_model\load.py in _revive_from_config(self, identifier, metadata, node_id)
351 obj = (
352 self._revive_graph_network(metadata, node_id) or
--> 353 self._revive_layer_from_config(metadata, node_id))
354
355 if obj is None:

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\saving\saved_model\load.py in _revive_layer_from_config(self, metadata, node_id)
406 try:
407 obj = layers_module.deserialize(
--> 408 generic_utils.serialize_keras_class_and_config(class_name, config))
409 except ValueError:
410 return None

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\layers\serialization.py in deserialize(config, custom_objects)
107 module_objects=globs,
108 custom_objects=custom_objects,
--> 109 printable_module_name='layer')

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\utils\generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
373 list(custom_objects.items())))
374 with CustomObjectScope(custom_objects):
--> 375 return cls.from_config(cls_config)
376 else:
377 # Then cls may be a function returning a class.

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in from_config(cls, config)
653 A layer instance.
654 """
--> 655 return cls(**config)
656
657 def compute_output_shape(self, input_shape):

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\layers\convolutional.py in init(self, filters, kernel_size, strides, padding, data_format, dilation_rate, activation, use_bias, kernel_initializer, bias_initializer, kernel_regularizer, bias_regularizer, activity_regularizer, kernel_constraint, bias_constraint, **kwargs)
597 kernel_constraint=constraints.get(kernel_constraint),
598 bias_constraint=constraints.get(bias_constraint),
--> 599 **kwargs)
600
601

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\layers\convolutional.py in init(self, rank, filters, kernel_size, strides, padding, data_format, dilation_rate, activation, use_bias, kernel_initializer, bias_initializer, kernel_regularizer, bias_regularizer, activity_regularizer, kernel_constraint, bias_constraint, trainable, name, **kwargs)
123 name=name,
124 activity_regularizer=regularizers.get(activity_regularizer),
--> 125 **kwargs)
126 self.rank = rank
127 if filters is not None and not isinstance(filters, int):

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\training\tracking\base.py in _method_wrapper(self, *args, **kwargs)
454 self._self_setattr_tracking = False # pylint: disable=protected-access
455 try:
--> 456 result = method(self, *args, **kwargs)
457 finally:
458 self._self_setattr_tracking = previous_value # pylint: disable=protected-access

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in init(self, trainable, name, dtype, dynamic, **kwargs)
292 }
293 # Validate optional keyword arguments.
--> 294 generic_utils.validate_kwargs(kwargs, allowed_kwargs)
295
296 # Mutable properties

~\miniconda3\envs\py36\lib\site-packages\tensorflow\python\keras\utils\generic_utils.py in validate_kwargs(kwargs, allowed_kwargs, error_message)
790 for kwarg in kwargs:
791 if kwarg not in allowed_kwargs:
--> 792 raise TypeError(error_message, kwarg)
793
794

TypeError: ('Keyword argument not understood:', 'groups')

033_Hand_Detection_and_Tracking : handedness ?

Congratulations and thank you for the great job you are doing !

I have downloaded the Openvino models from your repository (https://github.com/PINTO0309/PINTO_model_zoo/blob/master/033_Hand_Detection_and_Tracking/07_openvino/download.sh), and I am writing some python code to use them with Openvino.
Palm detection model is working fine. But for the hand landmarks model, I don't find the handedness output described in the google paper MediaPipe Hands: On-device Real-time Hand Tracking (https://arxiv.org/pdf/2006.10214.pdf).
When I use Netron on the current model from mediapipe repo (https://github.com/google/mediapipe/blob/master/mediapipe/models/hand_landmark.tflite), I can see the output named 'output_handedness', which does not exist in the model from your repo.
Is it because google published several versions of this model and you are using an older version ?
If yes, do you know if the improvements in the last version are worth using it and if you plan to convert it ?

Thanks !

Mediapipe facemesh tfjs

Any plans for optimization for mobile browsers. Or tfjs model provided in the repo will work.

BlazeFace - How to use?

Anyhow knows how to use BlazeFace ?

Models published here return an array of 4 tensors as output:

  tf_op_layer_classificators_1: dtype: float32, shape: [1, 512, 1]
  tf_op_layer_classificators_2: dtype: float32, shape: [1, 384, 1]
  tf_op_layer_regressors_1: dtype: float32, shape: [1, 512, 16]
  tf_op_layer_regressors_2: dtype: float32, shape: [1, 384, 16]

While a model published on TFHub
and used in their example app at https://www.npmjs.com/package/@tensorflow-models/blazeface returns a single tensor:

  Identity: dtype: float32, shape: [1, 896, 17]

Why? I already have a working app using model from TFHub, but I'd like to use BlazeFace back-model as it's 4x resolution and much better with smaller faces and only model published on TFHub is front-model.

schema.fbs error: illegal character: <

I am trying to convert blazeface_front.tflite to .pb and getting stack at there

output json command = flatc -t --strict-json --defaults-json -o . schema.fbs -- face_detection_front.tflite error: /home/duc/Desktop/FServing/flatbuffers/schema.fbs:6: 1: error: illegal character: < Traceback (most recent call last): File "/home/duc/Desktop/FServing/flatbuffers/blazeface_tflite_to_pb.py", line 210, in <module> main() File "/home/duc/Desktop/FServing/flatbuffers/blazeface_tflite_to_pb.py", line 168, in main ops, op_types = parse_json() File "/home/duc/Desktop/FServing/flatbuffers/blazeface_tflite_to_pb.py", line 30, in parse_json j = json.load(open(model_json_path)) FileNotFoundError: [Errno 2] No such file or directory: 'face_detection_front.json'

Updates for 33_Hand_Detection_and_Tracking

It seems google has improved the hand landmark model, the announcement is here , so maybe you might want to revisit it and update the models.

Also, I've noticed there's also updates for BlazeFace, where it seems there's two models, one for front facing cameras (selfie camera) and another for back facing (main) cameras.

Models seem to have been updated 2 months ago here.

deeplab cityscape edgetpu

Hi! Thank you for your work, it helps me a lot!
Now I am looking for deeplab model with cityscape pretrained, optimized for edgetpu (edgetpu.tflite)
Trying to build it by myself, but still no luck. I see you have a cityscape quant 257 and 769 - it would be ideally fit my use case. I'm trying to convert in in edgetpu.tflite file, but "Model not quantized".
If you can help me it would be awesome!

I want to use 3D object detection(Objectron) in ios is there someway I can do that?

Convert tflite with mask rcnn

How can you convert to mask rcnn to tflite model. Thanks

Object Detection Model Based on Open Image Dataset For Coral Tpu?

Would you be able to convert any of the object detection models based on open image dataset below for Coral Edge Tpu?

faster_rcnn_inception_resnet_v2_atrous_oidv4
ssd_mobilenetv2_oidv4
ssd_resnet_101_fpn_oidv4

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md

I tried to convert the models, but couldn't convert them.

from tensorflow.compat.v1.lite import TFLiteConverter
import tensorflow as tf
import cv2
from glob import glob

def represent():
	for img in glob('oid/*'):
		x = cv2.imread(img)
		x = cv2.resize(x, (300,300))
		x = cv2.cvtColor(x, cv2.COLOR_BGR2RGB)
		x = x/255
		yield [x]

dir = 'models/ssd_mobilenet_v2_oid_v4_2018_12_12/saved_model'
converter = TFLiteConverter.from_saved_model(dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
converter.representative_dataset = represent
converter.experimental_new_converter = True
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)

Converted palm detection model

Hi, thanks for sharing all these models with us. In the folder "033_Hand_Detection_and_Tracking" there are scripts to convert the palm_detection.tflite model to a float 32 .pb model but it is not possible to download it with the download script provided, since it just downloads the hand_landmark model. Could you please make the converted palm detection model available?

What dataset was used in training the 026_mobile-deeplabv3-plus OpenVINO models?

I am assuming that these are the same as the ones provided by TensorFlow at tf/models/research/deeplab.

If so, the models should output segmentations for 21 classes. However, the provided optimized models predict seg_maps containing only zeros and ones.

Can you please explain the difference between your output and the original model on tf? Is this because you retrained the model on a person class only? If so, what was the dataset used in training?

Thank you.

posenet versions and resnet.

Hi, first of all, thanks for this great repository of tensor flow models!, I am learning Tensor Flow and it's very useful!.

I am trying to use the models from Posenet, and the results I am getting don't look very good compared to what we can see in the online posenet demo.

Also, in the TensorFlow.JS repository, they say they're using the new PoseNet 2.0, which is only available for TensorFlow.JS.... and it comes in two modes: MobileNet and ResNet.

My questions are:

The Posenet models available in your repository, are based on the old Posenet models? or are they based on the new Posenet 2.0 advertised by tensorflow.js?

Could it be possible for you to include the new Posenet ResNet models in your repository?

Thanks in advance!

Segmenetation fault while training deeplabV3 on cityscapes

Hi
I am following your tutorial for training deeplab, can you help me with this segmentation fault, please.
here is the command I'm using for training:

python3 train.py \
    --logtostderr \
    --training_number_of_steps=5000 \
    --train_split="train_fine" \
    --model_variant="mobilenet_v3_small_seg" \
    --decoder_output_stride=16 \
    --train_crop_size="513,513" \
    --train_batch_size=2 \
    --dataset="cityscapes" \
    --save_interval_secs=300 \
    --save_summaries_secs=300 \
    --save_summaries_images=False \
    --log_steps=100 \
    --train_logdir=/home/rahbinsanat/amir/tensorflow-github/research/deeplab/datasets/cityscapes/exp/train_on_train_set/train \
    --dataset_dir=/home/rahbinsanat/amir/tensorflow-github/research/deeplab/datasets/cityscapes/tfrecord

Here you can see the error while starting queues:

I0905 11:47:48.102874 140652054267712 learning.py:768] Starting Queues.
Fatal Python error: Segmentation fault

Thread 0x00007fea357fa700 (most recent call first):
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443 in _call_tf_sessionrun
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350 in _run_fn
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365 in _do_call
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1359 in _do_run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1180 in _run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 956 in run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/training/training_util.py", line 68 in global_step
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/training/saver.py", line 1149 in save
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/training/supervisor.py", line 1119 in run_loop
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/training/coordinator.py", line 495 in run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/home/rahbinsanat/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007fea35ffb700 (most recent call first):
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443 in _call_tf_sessionrun
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350 in _run_fn
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365 in _do_call
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1359 in _do_run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1180 in _run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 956 in run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/training/training_util.py", line 68 in global_step
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/training/supervisor.py", line 1081 in run_loop
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/training/coordinator.py", line 495 in run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/home/rahbinsanat/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007fea367fc700 (most recent call first):
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443 in _call_tf_sessionrun
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350 in _run_fn
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365 in _do_call
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1359 in _do_run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1180 in _run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 956 in run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/training/supervisor.py", line 1045 in run_loop
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/training/coordinator.py", line 495 in run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/home/rahbinsanat/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007febde7fc700 (most recent call first):
  File "/home/rahbinsanat/anaconda3/lib/python3.7/threading.py", line 296 in wait
  File "/home/rahbinsanat/anaconda3/lib/python3.7/queue.py", line 170 in get
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/summary/writer/event_file_writer.py", line 159 in run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/home/rahbinsanat/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007fec1bbb4740 (most recent call first):
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443 in _call_tf_sessionrun
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350 in _run_fn
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365 in _do_call
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1359 in _do_run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1180 in _run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 956 in run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/contrib/slim/python/slim/learning.py", line 490 in train_step
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/contrib/slim/python/slim/learning.py", line 775 in train
  File "train.py", line 456 in main
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/absl/app.py", line 250 in _run_main
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/absl/app.py", line 299 in run
  File "/home/rahbinsanat/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40 in run
  File "train.py", line 462 in <module>
Segmentation fault (core dumped)```

How to fold batch norm in Conv2D

So I would like to fuse batch norm in Conv2D before doing full integer quantization. Is that doable? I've read the article on https://qiita.com/PINTO/items/865250ee23a15339d556 but didn't find the information. Maybe someone here could help me, thanks!

Mask-RCNN unresolved custom op

Hey, great project!

I tried to run the Mask-RCNN tflite model and I get the following error:

RuntimeError: Encountered unresolved custom op: NonMaxSuppressionV5.Node number 109 (NonMaxSuppressionV5) failed to prepare.

Do you have any plans to create this operation for tflite?

BlazePose

why not translate 'pose_landmark_upper_body.tflite' to pb,'pose_landmark_upper_body_256x256_float32' is not the standard model in google project.Can you translate'pose_landmark_upper_body.tflite' to pb? thanks a lot

mobilenetv3-ssd quantization

can you provide the float32 version of tflite file?
I found tflite_graph.pb and tflite_graph.pbtxt files, but I failed to convert it into tflite float32 or int8.
Or you can give me some advice or code how to do that?
Thanks !

model conversion failed

I tried to convert .pb file to .tflite file.
I run "06_mobilenetv2-ssdlite/01_coco/01_float32/03_integer_quantization_with_postprocess.py" files and get this error

ValueError: Invalid tensors 'TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3' were found.

I've run the script to download tflite_graph.pb.
could you please check out the problem for me?
Also can you tell me where you get tflite_graph.pb?
can I convert model zoo ssdlite_mobilenet_v2_coco by your code?
Thanks!

model's output tensor size is not 4

engine = DetectionEngine("yolov4_416_full_integer_quant_edgetpu.tflite")

ValueError: Dectection model should have 4 output tensors!This model has 3.

How to use U^2-net in Tensorflow?

I am trying to use the u2netp_480x640_float32.pb model in Tensorflow 2.3.1 (through the Java bindings but that should not make much of a difference). However I am not sure if my use of that model is incorrect or maybe something else is wrong.

What I do: load a 640x480 RGB image into a [1, 480, 640, 3] tensor and I scale the values between -1 and 1. I feed the tensor to the inputs layer and I fetch from the Identity layer, which is [1, 480, 640, 1]

What I get:

I also tried to scale values between 0-1, 0-255, -127-127. The latter gives me:

But that is not what I expect U^2-net to do either.

Am I missing something, or am I reading from the wrong layers?

Thanks

Model miss

Google has released a Hair Segmentation model, but you don't seem to have this project at present.

Model conversion error

After running download.sh, I am trying to convert face_detection_front.pb to quantized tflite model. I have tried all scripts inside 30_BlazeFace/01_float32 directory but all gets failed with the following error:

ValueError: This converter can only convert a single ConcreteFunction. Converting multiple functions is under development.

I am using TensorFlow 2.2.0 on MacOS. Also, tried with a Linux machine with 2.1.0 and 2.2.0.

NOTE: I am trying to rebuild quantized model to run on microcontroller. The quantized model for the blazeface provided in your repo download.sh has error while running on the microcontroller: "Didn't find op for builtin opcode 'CONV_2D' version '3'" or "Didn't find op for builtin opcode 'QUANTIZE' version '2'".

About full integer quantization to the edge tpu one

Thanks for the great work. I have a question about how to convert the model to edge tpu one.
I can see the steps in 03posenet, but how to convert the model to edge tpu one? Thanks.

keras-retinanet Tensorflow 2 SavedModel

I could not find keras-retinanet (https://github.com/fizyr/keras-retinanet) *.pb (TF2 SavedModel) in your repository.
Here is a trained model if you would like to try:
https://github.com/fizyr/keras-retinanet/releases/download/0.5.1/resnet50_coco_best_v2.1.0.h5

ImportError: No Module named tensorflow.lite.python.interpreter

I've tried to replicate one of the examples (specifically example 1) with a different OS in a raspberry pi 4B model.

Knowing that this is using Ubuntu instead of a Raspbian Buster, am I to presume this is an OS problem? or does the problem lie in the tensorflow/tensorflow-lite version i am using?

Many thanks in advance.

To the owner of the repo, you are amazing for creating this work

Here are the details:

$ sudo python mobilenetv2ssdlite_movie_sync.py
Traceback (most recent call last):
  File "mobilenetv2ssdlite_movie_sync.py", line 10, in <module>
    from tensorflow.lite.python.interpreter import Interpreter
ImportError: No module named tensorflow.lite.python.interpreter

--------------------------- PYTHON IMPORT SUCCESSFULLY BUT NOT INSCRIPT --------------------------- 

(2.2.0_Tensor) ubuntu@ubuntu:~/Repositories/PINTO_model_zoo/006_mobilenetv2-ssdlite/02_voc$ python
Python 3.7.5 (default, Apr 19 2020, 20:18:17)
[GCC 9.2.1 20191008] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
>>> from tensorflow.lite.python.interpreter import Interpreter
>>> from tflite_runtime.interpreter import Interpreter

--------------------------- SYSTEM ENVIRONMENT INFO --------------------------- 
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="19.10 (Eoan Ermine)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 19.10"
VERSION_ID="19.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=eoan
UBUNTU_CODENAME=eoan```


$ pip list
Package                Version
---------------------- ---------
absl-py                0.9.0
appdirs                1.4.4
astunparse             1.6.3
cachetools             4.1.1
certifi                2020.6.20
chardet                3.0.4
Cython                 0.29.21
distlib                0.3.1
easydict               1.9
filelock               3.0.12
gast                   0.3.3
google-auth            1.20.0
google-auth-oauthlib   0.4.1
google-pasta           0.2.0
grpcio                 1.30.0
h5py                   2.10.0
idna                   2.10
importlib-metadata     1.7.0
Keras-Preprocessing    1.1.2
Markdown               3.2.2
numpy                  1.19.1
oauthlib               3.1.0
opt-einsum             3.3.0
pbr                    5.4.5
Pillow                 7.2.0
pip                    20.2
protobuf               3.12.4
pyasn1                 0.4.8
pyasn1-modules         0.2.8
requests               2.24.0
requests-oauthlib      1.3.0
rsa                    4.6
scipy                  1.4.1
setuptools             49.2.0
six                    1.15.0
stevedore              3.2.0
tensorboard            2.2.2
tensorboard-plugin-wit 1.7.0
tensorflow             2.2.0
tensorflow-estimator   2.2.0
termcolor              1.1.0
tflite-runtime         2.2.0
urllib3                1.25.10
virtualenv             20.0.28
virtualenv-clone       0.5.4
virtualenvwrapper      4.8.4
Werkzeug               1.0.1
wheel                  0.34.2
wrapt                  1.12.1
zipp                   3.1.0```

Posenet Edge TPU

Hi,

I'm a little bit beginner and I'm looking for official posenet model (257) based on mobilenet for edgetpu (google dev board). Do you have such model please because I didn't find or I missed it in your repositories ? :)

Thanks for help

Is the blazeface_tflite_to_pb.py script supporting for converting UINT8 quant TFLite model to pb

Hi,

sorry to renew a ticket. Is the blazeface_tflite_to_pb.py script supporting for converting UINT8 quant TFLite model to pb?

thanks

How to use representative_dataset_gen()

I used tf 2.3 and met segmentation fault (core dumped) error when using representative_dataset_gen().
Code

    def representative_dataset_gen():
      for audio in validation_fingerprints:
        yield [audio]
    converter = tf.lite.TFLiteConverter.from_saved_model(flags.train_dir + '/last_model')
    converter.optimizations = [tf.lite.Optimize.DEFAULT]
    converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS]
    converter.allow_custom_ops = True
    converter.inference_input_type = tf.uint8
    converter.inference_output_type = tf.uint8
    # converter.representative_dataset = representative_dataset_gen
    quant_model = converter.convert()
    with open(flags.train_dir + '/quant_last_model.tflite', 'wb') as w:
      w.write(quant_model)

Above code can run without error but if I use converter.representative_dataset = representative_dataset_gen, it fail.

The type and shape of data are at below. The input layer size is [Batch, 16384]

type(validation_fingerprints): <class 'numpy.ndarray'>
shape(validation_fingerprints): (3093, 16384)

Any suggestion?

Blazeface back camera model (quantized)

I could not find any quantized version for face_detection_back.tflite model.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

pinto0309 / pinto_model_zoo Goto Github PK

pinto_model_zoo's Introduction

PINTO_model_zoo

Contributors

My article

List of pre-quantized models

1. Image Classification

2. 2D Object Detection

3. 3D Object Detection

4. 2D/3D Face Detection

5. 2D/3D Hand Detection

6. 2D/3D Human/Animal Pose Estimation

7. Depth Estimation from Monocular/Stereo Images

8. Semantic Segmentation

9. Anomaly Detection

10. Artistic

11. Super Resolution

12. Sound Classifier

13. Natural Language Processing

14. Text Recognition

15. Action Recognition

16. Inpainting

17. GAN

18. Transformer

19. Others

Sample.1 - Object detection by video file

Sample.2 - Object detection by USB Camera

Sample.3 - Head Pose Estimation, Multi-stage inference with multi-model

Sample.4 - Semantic Segmentation, DeeplabV3-plus 256x256

Sample.5 - MediaPipe/FaceMesh, face_detection_front_128_weight_quant, face_landmark_192_weight_quant

Sample.6 - MediaPipe/Objectron, object_detection_3d_chair_640x480_weight_quant

Sample.7 - MediaPipe/Objectron, object_detection_3d_chair_640x480_openvino_FP32

Sample.8 - MediaPipe/BlazeFace, face_detection_front_128_integer_quant

Sample.9 - MediaPipe/Hand_Detection_and_Tracking(3D Hand Pose), hand_landmark_3d_256_integer_quant.tflite + palm_detection_builtin_256_integer_quant.tflite

Sample.10 - DBFace, 640x480_openvino_FP32

Sample.11 - Human_Pose_Estimation_3D, 640x480, Tensorflow.js + WebGL + Browser

Sample.12 - BlazePose Full Body, 640x480, Tensorflow.js + WebGL + Browser

Sample.13 - Facial Cartoonization, 640x480, OpenVINO Corei7 CPU only

1. Environment

2. Procedure

2-1. MobileNetV3+DeeplabV3+PascalVOC

2-1-1. Preparation

2-1-2. "mobilenet_v3_small_seg" Float32 regular training

2-1-3. "mobilenet_v3_large_seg" Float32 regular training

2-1-4. Visualize training status

2-2. MobileNetV3+DeeplabV3+Cityscaps - Post-training quantization

2-2-1. Preparation

2-2-2. Parameter sheet

2-2-3. "mobilenet_v3_small_seg" Export Model

2-2-4. "mobilenet_v3_large_seg" Export Model

2-2-5. Google Colaboratory - Post-training quantization - post_training_integer_quant.ipynb

2-3. MobileNetV3+DeeplabV3+Cityscaps - Quantization-aware training

2-3-1. "mobilenet_v3_small_seg" Quantization-aware training

2-3-2. "mobilenet_v3_large_seg" Quantization-aware training

2-4. MobileNetV2+DeeplabV3+coco/voc - Post-training quantization

2-4-1. Preparation

2-5. MobileNetV3-SSD+coco - Post-training quantization

2-5-1. Preparation

2-5-2. Create a conversion script from checkpoint format to saved_model format

2-5-3. Confirm the structure of saved_model 【ssd_mobilenet_v3_small_coco_2019_08_14】

2-5-4. Confirm the structure of saved_model 【ssd_mobilenet_v3_large_coco_2019_08_14】

2-5-5. Creating the destination path for the calibration test dataset 6GB

2-5-6. Quantization

2-5-6-1. ssd_mobilenet_v3_small_coco_2019_08_14

2-5-6-2. ssd_mobilenet_v3_large_coco_2019_08_14

2-6. MobileNetV2-SSDLite+VOC - Training -> Integer Quantization

2-6-1. Training

2-6-2. Export model (--add_postprocessing_op=True)

2-6-3. Integer Quantization

3. TFLite Model Benchmark

4. Reference articles

pinto_model_zoo's People

Contributors

Stargazers

Watchers

Forkers

pinto_model_zoo's Issues

Error1: `ssd_mobilenet_v2_mnasfpn_shared_box_predictor_320_coco_full_integer_quant_edgetpu.tflite``

Error2: yolov4_416_full_integer_quant_edgetpu.tflite

Error3: ssd_mobilenet_v2_oid_v4_300x300_full_integer_quant_edgetpu.tflite

Error2: `yolov4_416_full_integer_quant_edgetpu.tflite`

Error3: `ssd_mobilenet_v2_oid_v4_300x300_full_integer_quant_edgetpu.tflite`

Error4: `ssdlite_mobilenet_v2_coco_300_full_integer_quant_edgetpu.tflite`