Object detection - CPU resources about edgetpu HOT 7 CLOSED

Ok, you saved me... My bad!

I did not compile the model.
I only converted it to .tflite and I was assuming that this was the compile step.

Thank you for your help.
I'm closing the issue.

from edgetpu.

@aaguiar96 Can you give some code snippets, as well as models used? It's a little hard to tell with the given info, but yes, most inference works will be delegated to the TPU (depending on the model). Keep in mind that input processing will be executed with the CPU.
Edit: Also some data transferring should be using CPU

from edgetpu.

My main question is:

• If I use the detection engine for example as:

engine             = new coral::DetectionEngine((*params).model);
input_tensor_shape = (*engine).get_input_tensor_shape();
labels             = coral::ReadLabelFile((*params).labels);
...
std::vector<uint8_t> input_tensor = coral::GetInputFromImage(in_image, {input_tensor_shape[1], input_tensor_shape[2], input_tensor_shape[3]});
auto results = (*engine).DetectWithInputTensor(input_tensor);

will this code be delegated to the USB Accelerator device?

I'm using a retrained MobileNet model, compatible with the edgetpu.

Thanks in advance.

from edgetpu.

@aaguiar96 Hi, as far as I could tell, the code looks okay at glance. As for your retrained model, have you compiled it for the edgetpu? And could you also confirm that it is indeed using CPU with htop?

from edgetpu.

Yes, it is compiled for the edgetpu.
So, this code should been working? I profiled the code and the Invoke() routine in RunInference() function is the one who is responsible for the major CPU occupation.

I think this should not occur if the detection is running on the coral device.
Am I wrong?

from edgetpu.

Hi, you're not wrong, when running inference after creating a DetectionEngine should automatically delegate works to TPU. However, I'm wondering if all operation were mapped to the edgetpu during compilation. Any operation that was not mapped to the tpu will be executed on the CPU, can you provide the log file generated by the edgetpu_compiler by any chance (it should give very nice details on which operations are mapped to the edgetpu and which are not)?

from edgetpu.

No problems!

from edgetpu.