Components should be as independent as possible to allow for faster testing and demo development. For example, services should be able to be run without creating a RoutineContainer object.

Current model (Intel Midas/v2_1_small) runs at approximately 900 ms without optimizations on the Raspberry Pi 4 using TFLite with Python. Should find either a new model which has a lower latency, modify a model, or create a model based on existing architectures.

Create a TensorFlow Lite model and apply it to images to produce a depth map. Based on Zhang, Ziyu, et al. “A Simple Baseline for Fast and Accurate Depth Estimation on Mobile Devices.” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021.

Use hardware PWM to connect to and play audio through the speaker. Figure out what kind of audio can be played, how to store and play audio clips, and consider how routines will request audio to be played.

Setup DPHS, SystemContainer, Executable, SystemService, and Service components. Basically, setting up the framework for the core functionality of the project.

When canceling execution of Output.py, there is no function to delete the PWM instance (which enables sleep mode). Additionally, flickering occurs if chip is not disconnected from power between runs.

The essential components need to be fully integrated to produce a minimum, working, and error-free product. In particular, the following must be connected:

• API: Camera module, Forward LiDAR, Input, Output, Speaker, PWM Motor control
• Services and Routines: DepthPerception, EmergencyDetection
• Modes: General, Low Power, and (for demonstration) LiDAR/CV only
• Scheduler, System (mode/state), Hardware Interrupts, and mode indications for callback functions

Note: The intention is for this, and the representative value issue #31 , and potentially #14 be the last PRs for dev/v1-0-0. Tuning, optimizations, error checking/handling should be covered in a future dev/v1-1-0 or dev/v2-0-0

Methods and variables that should not be exposed to other components should be preceded by a '_'. For example, "self.input_" should become "self._input_".

Connect to LiDAR and make data available to services.

implement outdoor mode with curb detection

Place integer quantized model.

Implement curb detection using LiDAR.

Make image from the camera available to other software components via the input component.

Modules included in the API do not properly disable/re-enable/destroy. Must implement better resource management to prevent errors such as #33 . Additional optimizations may be included in this issue.

Find out how to connect software to PCA9685 (PWM LED/motor control chip). Additionally set output unit to handle changing duty cycles.

Special signal for front-facing LiDAR detecting threat.

Use GPIO interrupts to change the system state/mode on button push.

Tie general mode, low power mode, and outdoor mode together. This is likely to go in scheduler.py. Scheduler should poll the mode and run/toggle services based on that mode.

Errors and exceptions should be handled to prevent the system from crashing or stalling out across all components.

Must transform data from depth inference to 9 duty cycle values. Current approach is bilinear resize output to 32x32, apply Gaussian kernels to different points of image and find a maximum value, feed the maximums into a function for intensity of estimated depth, and output the results over PWM.

Introduction of PWM and LiDAR modules to the API folder has made things slightly disorganized. Adding an Input/ and Output/ directory to place these modules in (or a modules/ directory) is desirable. Additionally, separating the Camera code from the Input.py file is desirable for organization, clarity, ease-of-updating, and ensuring proper resource management.

Must set pins to their appropriate ALTs based on @ChadPauley 's GPIO assignment document. GPIO0/1 may not work for the lidar. In that case use any other unoccupied SDA/SCL.

Bug has to do with #27 and its associated PR. The bus is closed before the pwm script can write 0 to all outputs, which results in the LEDs remaining on after program termination. A quick fix is to write 0 to all outputs before closing the smbus object, but this may not be an ideal solution.

Enable a low power mode on detection of no movement for some amount of time.

Get accelerometer data and make it available to services.

Noticing redundant creation of SMBus objects. Will be more effective to create one in system.py and pass it into the input and output components. Additionally consider using with _ as bus and implementing __enter__ and __exit__. Former likely in DPHS.py, the latter within system.py. Frankly creating multiple of these objects may not even work, so this is important to address.

Set peripheral devices (e.g. LiDAR, camera, motor controller) to their low power modes. Mode may be revisited in the future for improved performance.