This will be necessary as our codebase will develop. Some requirement notes:

• .x, .y, .t, .p attributes
• numpy compatibility
• torch tensor conversion?
• raw event, batch events, iteratored events

We should consider if we drop the support for Python 3.7.
Need to check:

• PyTorch requirements
• Scipy requirements
• ROS requirements?

Hello!

I want to start a discussion on the I/O logic of this library. Do we want to a) accept only pre-recorded data files or b) also accept streaming devices (cameras recording live)?

I think it is important to start the discussion because this may change the way the events (and other possible data) is handled.

This is the parent issue that I intend to write down the roadmap and design decisions for opening up the development process to other people.

Goal

OpenCV for event-based vision.

Challenges

... and our strategies:

• No standardized fileformat of event data. > So this library provides an API layer which abstracts different file formats. (✅ 70% done)
• No state-of-the-art consensus for some of the typical computer vision tasks. > So, this library should be extensible to try and implement a new method.
• Different use-case for the data loading. > So, this library provides the interface for the data class both for iterator-based method (for online processing) and random-access--based method (for offline processing, e.g., DNN training).
• There are many different ways of handling synchronization and calibration for multiple sensors (event camera + other sensors). > So, this library provides a unified way to handle the sync, calib, by encapsulating the difference among existing datasets.
• Some people develop real-time applications (mostly in ROS) and others want to develop DNN (mostly in PyTorch). > So this library should be compatible with numpy and torch at the core concept. (to be discussed: jax and tensorflow). We prepare custom data types for event data which is natively compatible with these different use-cases.
• C++ vs. python implementation at the very bottom layer. > Not decided yet.

Features

• Data format: abstracts different file format and returns custom event data types.
• Dataset (data-loader): provides iter-based access vs. random access, handling syncronization.
• Preproc: Preprocessing, filters and so on.
• Representation: Convert events into different representations to bridge to some algorithms (e.g., voxel is necessary for many DNNs.)
• Visualization: Provides 2D- and 3D-view of event data and other modalities.
• Processing: provides multiple tasks and different solutions.

Wanted

• Developers
• Diagram of the modules (assigned: @shiba24 )

There seems some issue on the codec.fileformat.evk, needs to be investigated.

Pre-commit logic exists but is not used now. At some point I want to make it available.