luxonis / depthai-model-zoo Goto Github PK

Steps to reproduce

1. Open https://github.com/luxonis/depthai-model-zoo
2. Click on the link

   A web interface for easier navigation is available at zoo.luxonis.com.

Expected result

Actual result

Start with the why:

There are many converted models that are available to the community. Currently, most of them are located in the luxonis/depthai-experiments repository. By moving them to the model zoo, we are collecting all the available models and additional information in one location. This will allow an easy search over the models with the help of the new web based search engine, as well as reduce the size of the luxonis/depthai-experiments repository.

Move to the what:

Moving the models from the experiments to the model zoo. Due to experiments usually containing .blob files, generation of .xml and .bin files would be required in most cases. Afterwards, experiments could be updated to use the model zoo.

Move to the how:

1. Prepare a list of models in luxonis/depthai-experiments that are not yet in the model zoo + model from other repositories.
2. Upload them to the model zoo.
3. Update experiments in luxonis/depthai-experiments to use the model zoo.

Start with the why:

We've accumulated a lot of different models that are available in luxonis/depthai-experiments and this Model ZOO. With the development of web based search engine for models available on OAK devices, it would be beneficial to evaluate the models on the data sets they were trained on, with execution on OAKs. During the conversion some layers might be replaced, and models based on FP32 are converted to FP16. Such evaluation would show whether the model was converted successfully, emphasize possible issues, as well as enable us to compare which model performs better on OAK devices.

Move to the what:

• Evaluation of models in model ZOO on popular task related data sets:

  • depth_estimation_mbnv2_240x320,
  • depth_estimation_mbnv2_480x640,
  • dmcount_540x960,
  • east_text_detection_256x256,
  • fast_depth_256x320,
  • fast_depth_480x640,
  • head-pose-estimation_160x160,
  • hrdepth_192x640,
  • inceptionv4_299x299,
  • megadepth,
  • mobile_object_localizer_192x192,
  • scdepth_256x832,
  • yolop_320x320.

• Provide information on which data sets the models in the zoo were trained on.

Move to the how:

This can be executed in two steps:

1. provide an API that exposes the OAK as a backbone for computation, so evaluation on device is as easy as few lines of code,
2. download required data sets and adapt models' evaluation scripts so that OAK is used as a backbone.

As this is a tedious process, help from the community is much appreciated!

Greetings,

@Erol444 Helped me submit a model I've developed called Coneslayer to the luxonis model zoo last December. It's a custom-trained variant of yolov7-tiny which can recognize many common orange traffic cones.

In the time since, Luxonis has discovered and patched a bug with tools.luxonis.com which may significantly improve how the bounding boxes appear when a converted model is run on an Oak-D. I've reconverted the model and updated the Coneslayer GitHub repo

It should be possible to use the newly-converted model with a normal IoU threshold (such as 0.5) instead of 0.1 as well

Here's a link to the update commit:
mkrupczak3/Coneslayer@9c40bb8