ERROR : Deadline exceeded: USB transfer error 2 [LibUsbDataOutCallback] about edgetpu HOT 16 CLOSED

@Namburger thank you very much, I'll try with a powered usb hub.

from edgetpu.

I am using HomeAssistant yellow, and I'm still experiencing this issue.

I have tried using a powered USB hub and I made sure it can supply the Coral's peak current usage of 900 mA (I've used https://sandberg.world/en-us/product/USB-3.0-Hub-7-ports)

I have tried several different USB cables, and even tried using an extender to a total of 2m, since some people suggested moving the TPU away from the HomeAssistant yellow to avoid USB3 interference, but to no avail.

After the reboot, the TPU unit is detected, and the first detection or two seem to be working fine, but after that it just errors out with the aforementioned error.

EDIT: I have tried it with RaspberryPI 400 and it works fine, so it is definitely something with Raspberry PI Compute module 4 or the HomeAssistant Yellow hardware itself.

from edgetpu.

Hi artBIT, have you found a solution for your issue?

No, but as a workaround, I've moved my Frigate + TPU setup to a RPi400.

from edgetpu.

@yoyomolinas I know you said you are running 2 threads on 2 different TPUs, but I just wanted to clarify.
Are you ever allowing 2 threads to run inference on a single TPU at the same time?
If so, I would suggest allowing one thread to access one TPU at a time to see if you can reproduce the issue.

from edgetpu.

@Namburger two different threads have access to two different TPUs. So the two threads never try to run inference on the same TPU but, might attempt to use the USB Bus for transferring data to the TPUs at the same time.

from edgetpu.

@yoyomolinas I see, that does sound strange. If possible, could you include a little snippet of the offending code? Also it would be helpful to see the full out put of the error.
At the same time, could you recreate the issue if it is running as 2 difference processes instead of a single multi-threaded process?

from edgetpu.

A snippet of my code:

class ImageProcessor(threading.Thread):
    def __init__(self, device_path):
        self.terminated = False
        self.stream = io.BytesIO()
        self.event = threading.Event()
        self.engine_1 = edgetpu.detection.engine.DetectionEngine(SSD_MODEL_PATH, device_path=device_path)
        self.engine_2 = edgetpu.basic.basic_engine.BasicEngine(FEATEX_MODEL_PATH, device_path=device_path)
        self.start()

    def run(self):
        """
        This method runs in a separate thread
        """
        while not self.terminated:
            # Wait for an image to be written to the stream
            if self.event.wait(1):
                self.stream.seek(0)
                # Read the image and do some processing on it
                img = Image.open(self.stream)
                objects = detect(self.engine_1, img, threshold=CONF_THRESHOLD, top_k=10)
                objects, features = extract_features(self.engine_2, img, objects, input_shape=(224, 224))

In the main thread I create two ImageProcessor threads and continually write into their stream, which they wait to process. The detect function calls engine.detect_with_image function, while the extract_features function first crops all objects in the image and calls engine.run_inference on all.

@Namburger I've also tried previously to run as two separate processes instead of a single multi-threaded process. The result was the same, I still got the error after some time.

It is really odd that the error is not raised immediately but takes some time to pop out.

I was able to only find 2 online posts reporting this error: first, second (both in Japanese)

from edgetpu.

Are you using externally powered USB hub to connect usb sticks with TPU to RPI?

from edgetpu.

@lleonid no, I connect straight into RPI.

from edgetpu.

@lleonid do you think it could be a power issue?

from edgetpu.

@yoyomolinas Yes, it could definitely be a power issue. As running inference after a while will draw more power; this could explain how you are able to run inference on both devices for a while befor crashing. Did you installed the libedgetpu1-std or the libedgetpu1-max (max will draw more power)?
Your code doesn't look bad on first glance; I'm guessing you're creating 2 different ImageProcessor object (or another object similar) with a difference device_path? Also, which RPI are you running this on, I don't believe data speed is the issue for this particular case but 3.0 is recommended.
Lastly, could you recreate this issue on a different host machine?
I don't have a second USB Accelerator with me currently, that's why I'm asking you all these questions but I got another one on the way so allow me some times to try to recreate this issue!

from edgetpu.

As for power, it is recommended that your host machine can supply at least 500 mA at 5V but at peak inference the accelerator can draw up to 900 mA each:
https://coral.withgoogle.com/docs/accelerator/datasheet/#power-specifications

from edgetpu.

@Namburger exactly, I'm creating two ImageProcessor objects with different device_paths. I'm running this on RPI 4, so data speed is not an issue. As for the edgetpu runtime, I installed the python3-edgetpu package which I believe installs the libedgetpu1-std by default.

I have reproduced the issue on 2 different RPIs with 2 different pair of Usb Accelerators.

from edgetpu.

@yoyomolinas I just confirmed with the team that this issue is caused by not supplying enough power to the accelerators. A single accelerators could take up to 900mA so for multiple accelerators, it is recommended that you try with an externally powered usb hub.

from edgetpu.

I am using HomeAssistant yellow, and I'm still experiencing this issue.

I have tried using a powered USB hub and I made sure it can supply the Coral's peak current usage of 900 mA (I've used https://sandberg.world/en-us/product/USB-3.0-Hub-7-ports)

I have tried several different USB cables, and even tried using an extender to a total of 2m, since some people suggested moving the TPU away from the HomeAssistant yellow to avoid USB3 interference, but to no avail.

After the reboot, the TPU unit is detected, and the first detection or two seem to be working fine, but after that it just errors out with the aforementioned error.

Same issue for me

from edgetpu.

I am using HomeAssistant yellow, and I'm still experiencing this issue.

I have tried using a powered USB hub and I made sure it can supply the Coral's peak current usage of 900 mA (I've used https://sandberg.world/en-us/product/USB-3.0-Hub-7-ports)

I have tried several different USB cables, and even tried using an extender to a total of 2m, since some people suggested moving the TPU away from the HomeAssistant yellow to avoid USB3 interference, but to no avail.

After the reboot, the TPU unit is detected, and the first detection or two seem to be working fine, but after that it just errors out with the aforementioned error.

EDIT: I have tried it with RaspberryPI 400 and it works fine, so it is definitely something with Raspberry PI Compute module 4 or the HomeAssistant Yellow hardware itself.

Hi artBIT, have you found a solution for your issue?

from edgetpu.