Giter Club home page Giter Club logo

Comments (13)

jsl303 avatar jsl303 commented on May 12, 2024 1

Thanks so much! It works!
Here's the minimal testing code, and the result seems pretty decent.

import detect_common as common
from PIL import Image
from glob import glob
import pandas as pd
import numpy as np

label_url = 'https://storage.googleapis.com/openimages/2018_04/class-descriptions-boxable.csv'
df = pd.read_csv(label_url, header=None)
labels = list(df[1])
model = 'ssd_mobilenet_v2_oid_v4_300x300_full_integer_quant_edgetpu.tflite'
interpreter = common.make_interpreter(model)
interpreter.allocate_tensors()
width, height, channels = common.input_image_size(interpreter)
files = glob('imgs/**/*.png', recursive=True)
for file in files:
	img = Image.open(file)
	img = img.resize((width, height), Image.NEAREST)
	if img.mode != 'RGB': img = img.convert('RGB')
	img = np.asarray(img)
	img = (img-255/2)*(2/255)
	common.input_tensor(interpreter)[:, :] = img
	interpreter.invoke()
	classes = common.output_tensor(interpreter, 1)
	scores = common.output_tensor(interpreter, 2)
	score = int(100 * scores[0])
	label = labels[int(classes[0])]
	if score > 0: print(file, label, score)

from pinto_model_zoo.

PINTO0309 avatar PINTO0309 commented on May 12, 2024

Looks like I did the normal quantization procedure and was able to convert correctly.

ssd_mobilenetv2_oidv4
https://drive.google.com/file/d/19vTomVGGb8T0s0_WV2shxdEhaYMuqKY1/view?usp=sharing
Screenshot 2020-07-08 23:08:21
Screenshot 2020-07-08 23:09:35

$ python3 object_detection/export_tflite_ssd_graph.py \
    --pipeline_config_path=${HOME}/Downloads/ssd_mobilenet_v2_oid_v4_2018_12_12/pipeline.config \
    --trained_checkpoint_prefix=${HOME}/Downloads/ssd_mobilenet_v2_oid_v4_2018_12_12/model.ckpt \
    --output_directory=${HOME}/Downloads/ssd_mobilenet_v2_oid_v4_2018_12_12/export \
    --add_postprocessing_op=True
### tensorflow-gpu==1.15.2

import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np

def representative_dataset_gen():
  for data in raw_test_data.take(100):
    image = data['image'].numpy()
    image = tf.image.resize(image, (300, 300))
    image = image[np.newaxis,:,:,:]
    image = image - 127.5
    image = image * 0.007843
    yield [image]

tf.compat.v1.enable_eager_execution()

raw_test_data, info = tfds.load(name="voc/2007", with_info=True, split="validation", data_dir="~/TFDS", download=False)

# Integer Quantization - Input/Output=float32
graph_def_file="export/tflite_graph.pb"
input_arrays=["normalized_input_image_tensor"]
output_arrays=['TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1', 
               'TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3']
input_tensor={"normalized_input_image_tensor":[1,300,300,3]}

converter = tf.lite.TFLiteConverter.from_frozen_graph(graph_def_file, input_arrays, output_arrays,input_tensor)
converter.allow_custom_ops = True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
tflite_quant_model = converter.convert()
with open('ssd_mobilenet_v2_oid_v4_300x300_full_integer_quant.tflite', 'wb') as w:
    w.write(tflite_quant_model)
print("Full Integer Quantization complete! - ssd_mobilenet_v2_oid_v4_300x300_full_integer_quant.tflite")

from pinto_model_zoo.

jsl303 avatar jsl303 commented on May 12, 2024

from pinto_model_zoo.

PINTO0309 avatar PINTO0309 commented on May 12, 2024

I just ran the following command at the end:

$ edgetpu_compiler -s ssd_mobilenet_v2_oid_v4_300x300_full_integer_quant.tflite

from pinto_model_zoo.

jsl303 avatar jsl303 commented on May 12, 2024

from pinto_model_zoo.

Namburger avatar Namburger commented on May 12, 2024

Wow this is nice, thanks @PINTO0309 !

from pinto_model_zoo.

jsl303 avatar jsl303 commented on May 12, 2024

Hmm, got Prematurely excited...

I tested the model using the modified code from google-coral/examples-camera.

https://github.com/google-coral/examples-camera

However, both score and class always return -1.180104e-38 for some reason.

from pinto_model_zoo.

PINTO0309 avatar PINTO0309 commented on May 12, 2024

If the results are incorrect, the only option is to revise the normalization process in small steps.

For example,

def representative_dataset_gen():
  for data in raw_test_data.take(100):
    image = data['image'].numpy()
    image = tf.image.resize(image, (300, 300))
    image = image[np.newaxis,:,:,:]
    #image = image - 127.5
    #image = image * 0.007843
    yield [image]
def representative_dataset_gen():
  for data in raw_test_data.take(100):
    image = data['image'].numpy()
    image = tf.image.resize(image, (300, 300))
    image = image[np.newaxis,:,:,:]
    image = image - 127.5
    #image = image * 0.007843
    yield [image]

INT8 quantization is a process with a lot of precision loss, so it may be difficult to detect if there are too many classes to classify, but I haven't investigated it carefully.

from pinto_model_zoo.

jsl303 avatar jsl303 commented on May 12, 2024

Thanks for the suggestion. Whatever worth, I'm just including my testing code that tries to process files in imgs/ folder.

import detect_common as common
from PIL import Image
import tflite_runtime.interpreter as tflite
from glob import glob
import pandas as pd

model = 'ssd_mobilenet_v2_oid_v4_300x300_full_integer_quant_edgetpu.tflite'
interpreter = common.make_interpreter(model)
interpreter.allocate_tensors()
label_url = 'https://storage.googleapis.com/openimages/2018_04/class-descriptions-boxable.csv'
df = pd.read_csv(label_url)
labels = [row[1] for i, row in df.iterrows()]
for file in glob('imgs/*.png'):
	print(file)
	img = Image.open(file)
	common.set_input(interpreter, img)
	interpreter.invoke()
	classes = common.output_tensor(interpreter, 1)
	scores = common.output_tensor(interpreter, 2)
	for i in range(len(classes)):
		print(classes[i], scores[i])

Here is detect_common.py from google-coral/examples-camera.

# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Common utilities."""
import numpy as np
from PIL import Image
import tflite_runtime.interpreter as tflite

EDGETPU_SHARED_LIB = 'libedgetpu.so.1'

def make_interpreter(model_file):
    model_file, *device = model_file.split('@')
    return tflite.Interpreter(
      model_path=model_file,
      experimental_delegates=[
          tflite.load_delegate(EDGETPU_SHARED_LIB,
                               {'device': device[0]} if device else {})
      ])

def set_input(interpreter, image, resample=Image.NEAREST):
    """Copies data to input tensor."""
    image = image.resize((input_image_size(interpreter)[0:2]), resample)
    input_tensor(interpreter)[:, :] = image

def input_image_size(interpreter):
    """Returns input image size as (width, height, channels) tuple."""
    _, height, width, channels = interpreter.get_input_details()[0]['shape']
    return width, height, channels

def input_tensor(interpreter):
    """Returns input tensor view as numpy array of shape (height, width, 3)."""
    tensor_index = interpreter.get_input_details()[0]['index']
    return interpreter.tensor(tensor_index)()[0]

def output_tensor(interpreter, i):
    """Returns dequantized output tensor if quantized before."""
    output_details = interpreter.get_output_details()[i]
    output_data = np.squeeze(interpreter.tensor(output_details['index'])())
    if 'quantization' not in output_details:
        return output_data
    scale, zero_point = output_details['quantization']
    if scale == 0:
        return output_data - zero_point
    return scale * (output_data - zero_point)

from pinto_model_zoo.

jsl303 avatar jsl303 commented on May 12, 2024

I tried to revise the normalization process following the instruction, but ran into a roadblock with python3 object_detection/export_tflite_ssd_graph.py.
It cannot import name 'anchor_generator_pb2'.
I found a related issues and tried suggestions there with no luck.
tensorflow/models#1962
When you get a chance, it would be amazing if you could investigate the converted model and see if you could get it to produce the correct output.
These object detections models would be very useful for people to be able to run object detection with coral!
Thank you so much for your time and expertise!

from pinto_model_zoo.

PINTO0309 avatar PINTO0309 commented on May 12, 2024

Just normalize it on the business logic side. The certainty of the results has not been verified.

detect.py

import detect_common as common
from PIL import Image
import tflite_runtime.interpreter as tflite
from glob import glob
import pandas as pd

model = 'ssd_mobilenet_v2_oid_v4_300x300_full_integer_quant_edgetpu.tflite'
interpreter = common.make_interpreter(model)
interpreter.allocate_tensors()
label_url = 'https://storage.googleapis.com/openimages/2018_04/class-descriptions-boxable.csv'
df = pd.read_csv(label_url)
labels = [row[1] for i, row in df.iterrows()]
img = Image.open('grace_hopper.bmp')
common.set_input(interpreter, img)
interpreter.invoke()
classes = common.output_tensor(interpreter, 1)
scores = common.output_tensor(interpreter, 2)
for i in range(len(classes)):
    print(classes[i], scores[i])

detect_common.py

"""Common utilities."""
import numpy as np
from PIL import Image
import tflite_runtime.interpreter as tflite

EDGETPU_SHARED_LIB = 'libedgetpu.so.1'

def make_interpreter(model_file):
    model_file, *device = model_file.split('@')
    return tflite.Interpreter(
      model_path=model_file,
      experimental_delegates=[
          tflite.load_delegate(EDGETPU_SHARED_LIB,
                               {'device': device[0]} if device else {})
      ])

def set_input(interpreter, image, resample=Image.NEAREST):
    """Copies data to input tensor."""
    print(input_image_size(interpreter)[0:2])
    image = image.resize((input_image_size(interpreter)[0:2]), resample)
    image = np.asarray(image)
    image = image - 127.5
    image = image * 0.007843
    input_tensor(interpreter)[:, :] = image

def input_image_size(interpreter):
    """Returns input image size as (width, height, channels) tuple."""
    _, height, width, channels = interpreter.get_input_details()[0]['shape']
    return width, height, channels

def input_tensor(interpreter):
    """Returns input tensor view as numpy array of shape (height, width, 3)."""
    tensor_index = interpreter.get_input_details()[0]['index']
    return interpreter.tensor(tensor_index)()[0]

def output_tensor(interpreter, i):
    """Returns dequantized output tensor if quantized before."""
    output_details = interpreter.get_output_details()[i]
    output_data = np.squeeze(interpreter.tensor(output_details['index'])())
    if 'quantization' not in output_details:
        return output_data
    scale, zero_point = output_details['quantization']
    if scale == 0:
        return output_data - zero_point
    return scale * (output_data - zero_point)
$ sudo python3 detect.py 
(300, 300)
307.0 0.61328125
501.0 0.55859375
160.0 0.5
119.0 0.44140625
-1.180104e-38 -1.180104e-38
-1.180104e-38 -1.180104e-38
-1.180104e-38 -1.180104e-38
-1.180104e-38 -1.180104e-38
-1.180104e-38 -1.180104e-38
-1.180104e-38 -1.180104e-38

from pinto_model_zoo.

PINTO0309 avatar PINTO0309 commented on May 12, 2024

I have imported the quantization model into the repository.

from pinto_model_zoo.

jsl303 avatar jsl303 commented on May 12, 2024

Very cool!
I finally got my environment working, so I could run export_tflite_ssd_graph.py, convert the model using the script you provided earlier, and compile for Coral.
Even though I'm just copying pasting your command and script, the model I get has much poorer result than the one you produced.
Mine correctly labels about 15% of the time where yours correctly labels 45% of the time.
I just checked the accuracy of labeling, not boxes.
Very puzzled...

from pinto_model_zoo.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.