Comments (8)
So I understood everything and found the root cause of my problems here and in the neighbor issues.
I had to patch flatbuffers to avoid tensorflow/tensorflow#34523 and ensure the lossless tflite<>JSON roundabout, but that's minor.
There is an unfortunate limitation that was actually documented in https://coral.withgoogle.com/docs/edgetpu/models-intro/#model-requirements as "Output tensor is one-dimensional.". Both my inputs were 2D. As soon as I changed one input to be 1D, it successfully compiled. I will still be able to implement matrix multiplication by running N 1D MatMul ops in parallel and concatenating the result. Hacky, but that's the whole point.
I would leave all my recent issues open nevertheless because the misleading compiler status messages could definitely be improved.
from edgetpu.
Ufff, that was some time ago, I don't remember... This blog post could probably shed the light.
from edgetpu.
@vmarkovtsev it looks like you're creating a tflite model, but it's not fully quantized, so you take advantage of flatc
to turn the model into it's json representation, rewrite over some values and then use flatc
to convert it back to a tflite model.
We never documented any type of support similar to this. As for the issue, your model is not fully quantize which is probably why it's not mapped. Please review this again, you'll need representative data gen and TFLITE_BUILTINS_INT8
and please refrain from tf2.0 for now if you want it to be compatible with our compiler.
P.S. My apologies on the compiler not being open source
from edgetpu.
I have a working hypothesis. "Filter" is the second matrix in the matrix multiplication op. It does not like that it is taken as an input, hence "not constant". I'll check it.
from edgetpu.
@vmarkovtsev
Are you still using the from_concrete_function
api from tensorflow2.0? Can you possibly attach some snippets for creating this model?
from edgetpu.
I am, and you will definitely not like the code: I am patching flatbuffers JSON. However, I want to understand how everything works inside and hack it as much as I can. It should not really matter how I generate the model after all. Not having the source code of the compiler does not help, and I am feeling like a reverse engineer.
Code
import json
import os
from subprocess import run, PIPE, STDOUT
import tensorflow as tf
# !wget https://github.com/tensorflow/tensorflow/raw/master/tensorflow/lite/schema/schema.fbs
def echo_run(*cmd):
p = run(list(cmd), stdout=PIPE, stderr=STDOUT)
output = p.stdout.decode()
if output:
print(output)
p.check_returncode()
def test(size):
@tf.function(input_signature=[tf.TensorSpec([size] * 2, tf.int32)] * 2)
def bench_func(a, b):
x = tf.linalg.matmul(a, b, transpose_b=True) # b is not transposed, but this is a benchmark, whatever
return x # tf.reduce_sum(x) fails some internal assertion, see later
converter = tf.lite.TFLiteConverter.from_concrete_functions([bench_func.get_concrete_function()])
tflite_model = converter.convert()
fn = "bench_model_%d.tflite" % size
with open(fn, "wb") as fout:
fout.write(tflite_model)
echo_run("flatc", "-t", "--strict-json", "--defaults-json", "schema.fbs", "--", fn)
fn_json = fn.split(".")[0] + ".json"
with open(fn_json) as fin:
model_json = json.load(fin)
for t in model_json["subgraphs"][0]["tensors"]:
if t["name"] == "MatMul_bias":
t["type"] = "INT32"
continue
t["type"] = "UINT8"
try:
data = model_json["buffers"][t["buffer"]]["data"]
except KeyError:
continue
model_json["buffers"][t["buffer"]]["data"] = data[::4]
with open(fn_json, "w") as fout:
json.dump(model_json, fout, indent=4, sort_keys=True)
echo_run("flatc", "-b", "schema.fbs", fn_json)
# os.remove(fn_json)
echo_run("edgetpu_compiler", "-s", fn)
os.remove(fn.split(".")[0] + "_edgetpu.log")
return
os.remove(fn)
fn = fn.split(".")[0] + "_edgetpu.tflite"
test(16)
from edgetpu.
I'll try a different strategy tomorrow: taking a sample tflite model which is known to be compiled, convert it to JSON and methodically strip away everything but the final fully connected layer. I will be able to find the bloody problem and share it here.
from edgetpu.
@vmarkovtsev May I ask how did you run N 1D MatMul ops in parallel on Edge TPU? My model has only one "matmul" operation layer with two 1D inputs. Before converting to the TensorFlow Lite, the model would output a correct matrix multiplication result. However, after converting to the TensorFlow Lite model, the lite model would output an incorrect matrix multiplication result.
matrix1_input = keras.Input(shape=(16,), name="matrix1", dtype=tf.float32)
matrix2_input = keras.Input(shape=(16,), name="matrix2", dtype=tf.float32)
matrix_output = tf.matmul(matrix1_input, matrix2_input, name="matmul")[0]
model = keras.Model(inputs=[matrix1_input, matrix2_input], outputs=matrix_output,
name="model")
from edgetpu.
Related Issues (20)
- trying to call an object detection model within the camera streaming example
- gasket-dkms installation fails on debian testing (Trixie) HOT 1
- Which board in the most powerful for ML inference ? DevBoard ? DevBoard Mini ? DevBoard Micro ?
- pcie driver is not loading
- coral error from running demo
- Could not work with custom tensorflow lite models
- M.2 TPU not showing in Device Manager
- Topton N6005 NAS board - M2 NVME port which coral work with it? HOT 1
- Coral Dev Board: Tx Power is zero when wifi is set to adhoc mode
- MobileNetV2 is slow on EdgeTPU
- please delete this issue
- Unable to run model on Edge TPU
- PREEMPT_RT Kernel Patch with Mendel OS?
- M.2 Coral TPU inside USB Enclosure HOT 1
- Can not reach coral mini
- Coral USB TPU misbehaving on Unraid
- LSTM edgeTPU compiling error
- apt-key deprecated
- Install gasket-dkms not supported on linux kernel 5.15 (Ubuntu22.04) HOT 1
- Hardware short? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from edgetpu.