Currently, most of our tests are done manually. We need a CI/CD system that can build and test neuralet applications for various architecture, and deploy containers to the neuralet dockerhub.

@MrUPGrade

Describe the bug
The codes, which are written for Ros, use both Ros(Publisher and Subscriber), and Thread.
the get image data from ROS Subscriber and send it to thread and wait for receive result from thread and publish to a topic.
In this section, I think you added a useless step. One of the benefits of Ros is the convenience of multitasking.
it not better, feed data to the model in ros-server-example.py and publish result with a custom message in a topic?

Our current implementation of NMS needs to be profiled and optimized for speed.

We can improve detections using:

• Scene priors
• Background/Foreground subtraction and modeling

@alpha-carinae29 & @mrn-mln I suggest you take a look at this.

Have a class that builds different distance calculation method based on config file and create a different class for each distance method with similar interfaces.

An application that detects pedestrians and identifies if two or more people are in close proximity and violate the physical distancing orders.

The application can be based on current pedestrian detection models that run on edge TPUs and GPUs.
E.g. thatbrguy/Pedestrian-Detection

We should figure out the best encoder (alternative to vp8enc) for each platform. Maybe we can get some hints from how encodebin chooses the encoder.

Hey,

Thanks for the well-documented and useful repository. I was wondering if the docker images and instructions presented are runnable on Coral Dev Boards only? I followed the instructions but cannot build or run the docker image and demo.

Ubuntu 18 - Cuda 10.2.

Thanks.

• 1 *
  docker run -it --privileged --net=host -v $(pwd)/../../:/repo neuralet/coral-dev-board:$MODEL_NAME
  Unable to find image 'neuralet/coral-dev-board:mobilenet_ssd_v2_face' locally
  mobilenet_ssd_v2_face: Pulling from neuralet/coral-dev-board
  77b067f4ec14: Already exists
  f55f20663706: Already exists
  6cf10d87432d: Already exists
  5209673823c9: Already exists
  853b3604bd0a: Already exists
  48ecfc0f64d8: Already exists
  641c7ec0d6e7: Pull complete
  Digest: sha256:550c93a4aad00967c163df01ecaa0694f4621b4e06a4bcb1cd2cd066571bb5d6
  Status: Downloaded newer image for neuralet/coral-dev-board:mobilenet_ssd_v2_face
  standard_init_linux.go:211: exec user process caused "exec format error"

2
docker build -t "neuralet/coral-dev-board:$MODEL_NAME" .
Sending build context to Docker daemon 30.72kB
Step 1/9 : FROM arm64v8/debian:buster
---> 798e532a0a9f
Step 2/9 : VOLUME /repo
---> Using cache
---> ba56834da20d
Step 3/9 : RUN apt-get update && apt-get install -y wget gnupg && rm /etc/apt/sources.list && rm -rf /var/lib/apt/lists && wget -qO - https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
---> Running in 6addf734ef9a
standard_init_linux.go:211: exec user process caused "exec format error"
The command '/bin/sh -c apt-get update && apt-get install -y wget gnupg && rm /etc/apt/sources.list && rm -rf /var/lib/apt/lists && wget -qO - https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -' returned a non-zero code: 1

Is your feature request related to a problem? Please describe.
Please add GPU support. because in Iran we don't access to Tpu edge

For testing face-mask detection, I need a video benchmark for creating a demo and test the capability of the model in a real-world scenario. Currently, we don't have video files in which people wear a face mask.

https://azuremarketplace.microsoft.com/en-us/marketplace/apps/category/internet-of-things?page=1&subcategories=iot-edge-modules

Examples:
https://github.com/Azure-Samples/NVIDIA-Deepstream-Azure-IoT-Edge-on-a-NVIDIA-Jetson-Nano
https://github.com/ebertrams/iotedge-iva-nano
https://github.com/Azure-Samples/Custom-vision-service-iot-edge-raspberry-pi

Currently, we don't have any host-level or application-level logging or monitoring for the app.

This issue needs more specification from the community, but we probably want rudimentary logging of host and application using something like Prometheus and a simple Grafana dashboard.

Hi everyone.
I have tried to train a SSD Mobilenet V2 model with TLT V2.0 for pedestrian detection with Oxford Town Center dataset based on the instruction provided by Nvidia in here.
My model have been trained successfully and after pruning and retraining the inference result was acceptable:


Then I tried to create a TensorRT engine from my exported .etlt model on a x86 device with RTX 1070 GPU and after setting up TensorRT 7.0 and TensorRT OOS I successfully created the TensorRT engine.
Then I tried to integrate this engine with DeepStraem 5.0. I modified some sample config files to inference with my trained model and sink the output in a mp4 file. However I got very poor result as you can see:

Hi,
I am following the instructions from:
https://github.com/neuralet/neuralet/tree/master/applications/smart-distancing

My Machine Configuration -
OS- Ubuntu 18.04
GPU- Nvidia GeForce 940M

I was able to build the docker image . When I try to run the docker container, I get the following error:

2020-05-12 09:03:45.421680: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory 2020-05-12 09:03:45.431507: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory 2020-05-12 09:03:45.431613: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. Downloading data from http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz 187932672/187925923 [==============================] - 341s 2us/step 2020-05-12 09:11:36.017416: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory 2020-05-12 09:11:36.705277: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303) 2020-05-12 09:11:37.051137: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (9d18664c1959): /proc/driver/nvidia/version does not exist 2020-05-12 09:11:38.717353: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 2020-05-12 09:11:47.817744: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 1800000000 Hz 2020-05-12 09:11:49.181041: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x9e8f0e0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-05-12 09:11:49.181396: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version Device is:  x86 Detector is:  mobilenet_ssd_v2 image size:  [300, 300, 3]  * Serving Flask app "ui.web_gui" (lazy loading)  * Environment: production    WARNING: This is a development server. Do not use it in a production deployment.    Use a production WSGI server instead.  * Debug mode: on  * Running on http://0.0.0.0:8000/ (Press CTRL+C to quit) opened video  /repo/applications/smart-distancing/data/TownCentreXVID.avi 2020-05-12 09:16:07.029768: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 25159680 exceeds 10% of system memory.

Build unit tests and end-to-end tests for the smart social distancing application.

Looking for community help here @mrn-mln @emmaWdev

We need to provide more documentation and tutorials for deploying neuralet models and applications.

@mhsekhavat this will be great if you can help with.

I'm setting up the smart distancing application on the Jeson Nano. I am using the sd card image from:
https://developer.nvidia.com/jetson-nano-sd-card-image

I have docker 18.09.7 installed.

I am following the instructions from:
https://galliot.us/2020/04/16/smart-distancing-technical-document/

I was able to build the docker image ok. When I try to run the docker container, I get the following error:

Traceback (most recent call last):
  File "neuralet-distancing.py", line 22, in 
    DistanceApp(args)
  File "neuralet-distancing.py", line 13, in __init__
    self.engine = CvEngine(self.config)
  File "/repo/applications/smart-distancing/libs/core.py", line 23, in __init__
    self.detector = Detector(self.config)
  File "/repo/applications/smart-distancing/libs/detectors/jetson/detector.py", line 17, in __init__
    from . import mobilenet_ssd_v2
  File "/repo/applications/smart-distancing/libs/detectors/jetson/mobilenet_ssd_v2.py", line 3, in 
    import tensorrt as trt
  File "/usr/local/lib/python3.6/dist-packages/tensorrt/__init__.py", line 1, in 
    from .tensorrt import *
ImportError: libnvinfer.so.6: cannot open shared object file: No such file or directory

I tried to check nvinfer version with dpkg -l | grep nvinfer and it shows version 7.1.0-1+cuda10.2. Does this mean I have the wrong version of the library?

I am a Linux rookie.

The goal for Neuralet is to simplify applying state-of-the-art deep learning models to edge devices. We are seeking the community to provide feedback on our roadmap.

Neuralet Design Principles

• No abstraction is absolutely better than a bad abstraction. We need to make it easy to understand and adapt to the user's needs, even if it takes more code. Simplify user path to adopt a model for an edge device in the shortest possible time.
• Users won't feel overwhelmed by looking at Neuralet repo. It should look familiar to users who know docker container and deep learning frameworks.
• We don't lock-in our users to use our models or containers. Users should be empowered to take a Neuralet example and adapt to their needs with a low barrier.
• The separations of concerns between each part of the platform should be clear - well-abstracted and modular.

High-level objectives

Supported Devices

• Google's Coral Edge-TPU
• NVIDIA's Jetson Nano
• x86-64 CPU architecture
• ARM CPU architecture
• NVIDIA GPUs (via NVIDA-docker)

Inference Modes

• Standardize around efficient and lightweight docker containers
• ROS 1 support
• CSI and USB Camera

Supported Models

• Visual recognition algorithms including
  • Object detection
  • Image classification
  • Image segmentation
• Speech recognition
  • Wake-Word detection
  • Speech recognition

Training and Exporting Models

• Docker containers for each supported model that trains, validates and exports models for target edge devices
  • Support single and multi-GPU training
  • Support transfer learning based on popular pre-trained models
• Export models to TFLite for Edge-TPU
• Export models for Jetson Nano using TensorRT
• Export models to ONNX for CPU and GPU inference

After ~2 minutes of running below error is raised.

Traceback (most recent call last):
File "neuralet-distancing.py", line 22, in
DistanceApp(args)
File "neuralet-distancing.py", line 16, in init
self.ui.start()
File "/repo/applications/smart-distancing/ui/web_gui.py", line 149, in start
self.__ENGINE_INSTANCE.process_video(video_path)
File "/repo/applications/smart-distancing/libs/core.py", line 78, in process_video
cv_image, objects, distancings = self.__process(cv_image)
File "/repo/applications/smart-distancing/libs/core.py", line 49, in __process
cv_image = cv.resize(cv_image, tuple(resolution))
cv2.error: /build/opencv-ytxPbe/opencv-3.2.0+dfsg/modules/imgproc/src/imgwarp.cpp:3492: error: (-215) ssize.width > 0 && ssize.height > 0 in function resize

Adding models that run on CPU and GPU on x86-64 nodes.

Definition of done:
1- A new detectors in libs/detectors that calls a GPU or CPU model
2- A new dockerfile to build the application container for CPU
3- A new dockerfile to build the application container for NVIDIA GPU, using NVIDIA Docker
4- Documentation for running and building the application.

Is your feature request related to a problem? Please describe.
Yes, issue #74 happened.

Describe the solution you'd like
Add instructions for using with x86 (and desktop GPU)

Describe alternatives you've considered

Additional context

Describe the bug
The following error is raised when a frame without object is given:

Traceback (most recent call last):
  File "neuralet-distancing.py", line 22, in <module>
    DistanceApp(args)
  File "neuralet-distancing.py", line 16, in __init__
    self.ui.start()
  File "/repo/applications/smart-distancing/ui/web_gui.py", line 191, in start
    self.__ENGINE_INSTANCE.process_video(video_path)
  File "/repo/applications/smart-distancing/libs/core.py", line 94, in process_video
    self.logger.update(objects, distancings)
  File "/repo/applications/smart-distancing/libs/loggers/loggers.py", line 48, in update
    self.logger.update(objects_list, distances)
  File "/repo/applications/smart-distancing/libs/loggers/csv_processed_logger.py", line 43, in update
    self.log_objects(objects_list, distances, objects_log_file_path)
  File "/repo/applications/smart-distancing/libs/loggers/csv_processed_logger.py", line 57, in log_objects
    violating_objects = extract_violating_objects(distances, self.dist_threshold)
  File "/repo/applications/smart-distancing/tools/objects_post_process.py", line 18, in extract_violating_objects
    triu_distances = np.triu(distances) + np.tril((float(dist_threshold) + 1) * np.ones(distances.shape))
  File "<__array_function__ internals>", line 6, in triu
  File "/usr/local/lib/python3.6/dist-packages/numpy/lib/twodim_base.py", line 465, in triu
    mask = tri(*m.shape[-2:], k=k-1, dtype=bool)
TypeError: tri() missing 1 required positional argument: 'N'

To Reproduce
Use dummy detector with default config on x86

We need to add support for the current input sources:

• Offline Video Files (either stored locally or hosted online)
• Live Video Feeds (IP Camera, etc)
• CSI on Jetson and Coral

Deploy the smart social distancing application to the Azure Marketplace for the community.

https://azuremarketplace.microsoft.com/en-us/marketplace/

The final goal of the Social Distancing app is to deploy something can help business and people adhere to social distancing best practices.
Nuralet team, since this is your baby did you guys have any thought how this is going to be deployed to help the public.
Since I will be concentrating on the Jetson nano as my platform and as soon as the app is working smoothly on it. I was going to go to a few local business and ask the if I can deploy it at there locations.
So I can start getting real world feedback.

Curate a dataset and build a facemask detection model.

A sample dataset can be found here:
https://github.com/VirajDeshwal/mask-detection-dataset

• Configuring input sources for cameras

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Hi,

I have created docker image as mentioned here https://github.com/neuralet/neuralet/tree/master/applications/smart-distancing for x86, but after creating the image, the docker run is not working.

I'm getting error:

Unable to find image 'neuralnet/x86_64:applications-smart-distancing' locally
docker: Error response from daemon: pull access denied for neuralnet/x86_64, repository does not exist or may require 'docker login': denied: requested access to the resource is denied. 

I have cloned the neuralnet repo and tried docker commands for x86 on Ubuntu 18.04. Are there any other steps to make this work? Or any prerequisites for x86?

Here is my machine configuration:

OS: Ubuntu 18.04
GPU: Nvidia GeForce GTX 1050 Ti

Please help.

Thanks,
Archana

Is your feature request related to a problem? Please describe.
The back-end for WebGUI is not optimized, speed is reduced while several clients open the feed. Also it is hard to actually stream the data to the client side and visualize there if you are not using any external streaming protocol.

Describe the solution you'd like
I will use Gstreamer to make a video stream broadcasted in the WebGUI for now, each frame will be the current output frame in the code.

Describe alternatives you've considered
Nothing.

Additional context

I am trying run this on Nano and I am getting the following error

docker run -it --runtime nvidia --privileged -p 8000:8000 -v /home/nano/neuralet/:/repo neuralet/jetson-nano:applications-smart-distancing
[TensorRT] ERROR: Could not register plugin creator: FlattenConcat_TRT in namespace:
Traceback (most recent call last):
File "neuralet-distancing.py", line 22, in
DistanceApp(args)
File "neuralet-distancing.py", line 13, in init
self.engine = CvEngine(self.config)
File "/repo/applications/smart-distancing/libs/core.py", line 23, in init
self.detector = Detector(self.config)
File "/repo/applications/smart-distancing/libs/detectors/jetson/detector.py", line 18, in init
self.net = mobilenet_ssd_v2.Detector(self.config)
File "/repo/applications/smart-distancing/libs/detectors/jetson/mobilenet_ssd_v2.py", line 57, in init
self.engine = self._load_engine()
File "/repo/applications/smart-distancing/libs/detectors/jetson/mobilenet_ssd_v2.py", line 26, in _load_engine
with open(trt_bin_path, 'rb') as f, trt.Runtime(self.trt_logger) as runtime:
FileNotFoundError: [Errno 2] No such file or directory: 'libs/detectors/jetson/data/TRT_ssd_mobilenet_v2_coco.bin'
Exception ignored in: <bound method Detector.del of <libs.detectors.jetson.mobilenet_ssd_v2.Detector object at 0x7f9c64c080>>
Traceback (most recent call last):
File "/repo/applications/smart-distancing/libs/detectors/jetson/mobilenet_ssd_v2.py", line 70, in del
del self.stream
AttributeError: stream

I checked and I see no files in data folder
@jetson-nano:/neuralet/applications/smart-distancing/libs/detectors/jetson$ cd data
@jetson-nano:/neuralet/applications/smart-distancing/libs/detectors/jetson/data$ ls
@jetson-nano:~/neuralet/applications/smart-distancing/libs/detectors/jetson/data$

From where I can d/l TRT_ssd_mobilenet_v2_coco.bin.

Hello,

I am trying to run the social-distance like this:

docker run -it --runtime nvidia --privileged -p 8000:8000 -v /data/projects/neuralet:/repo neuralet/jetson-nano:applications-smart-distancing

But I am getting this error:

Traceback (most recent call last): File "neuralet-distancing.py", line 22, in <module> DistanceApp(args) File "neuralet-distancing.py", line 13, in __init__ self.engine = CvEngine(self.config) File "/repo/applications/smart-distancing/libs/core.py", line 19, in __init__ self.logger = Logger(self.config) File "/repo/applications/smart-distancing/libs/loggers/loggers.py", line 19, in __init__ self.name = self.config.get_section_dict("Logger")["Name"] File "/repo/applications/smart-distancing/libs/config_engine.py", line 59, in get_section_dict return self.section_options_dict[section] KeyError: 'Logger'

Same issue if I run inside the container the neuralet-distancing.py:

root@5ad09acb184a:/repo/applications/smart-distancing# python3 neuralet-distancing.py --config config-jetson.ini Traceback (most recent call last): File "neuralet-distancing.py", line 22, in <module> DistanceApp(args) File "neuralet-distancing.py", line 13, in __init__ self.engine = CvEngine(self.config) File "/repo/applications/smart-distancing/libs/core.py", line 19, in __init__ self.logger = Logger(self.config) File "/repo/applications/smart-distancing/libs/loggers/loggers.py", line 19, in __init__ self.name = self.config.get_section_dict("Logger")["Name"] File "/repo/applications/smart-distancing/libs/config_engine.py", line 59, in get_section_dict return self.section_options_dict[section] KeyError: 'Logger'

Any ideas how I can overcome this issue?

Thank you,
FM

Deploy the social distancing detection models on the AWS SageMaker marketplace for the community.

As the annotations only exist for the first 2500 frames of the Oxford Town Center Dataset it helps training accurate models for pedestrian detection if someone can annotate the ramaining frames and add it to the bottom of the annotation CSV file.