neuromorphs / tonic Goto Github PK

at the moment this is the only option. would be nice to be able to choose

In the VPR dataset, we have GPS data. It would be great if we can incorporate that data as well. I use a super simple script at the moment to do that:

https://github.com/Tobias-Fischer/ensemble-event-vpr/blob/master/read_gps.py

use sphinx and host on readthedocs. Walk through can be found here: https://brendanhasz.github.io/2019/01/05/sphinx.html

skew timestamps using a fixed pattern/function for all events

Hook up testing suite to travis

If I add a transform:

subsample_transform = tonic.transforms.Subsample(1e-3)
dataset = tonic.datasets.SHD(save_to='./data', train=False, transform=subsample_transform)
events, label = dataset[0]

events is now a tuple with a superfluous None. This is really annoying so I'm more than happy to fix this in tonic rather than my code but, I'm not quite sure why transform classes are returning images at all (e.g. here) so not totally clear how to fix it. Would the correct fix be to duplicate this logic from transform.Compose across all transforms (the 'functional' transforms just seem to return events).

The TUMVIE visual inertial event dataset could be interesting for Tonic: https://vision.in.tum.de/data/datasets/visual-inertial-event-dataset

It's public under CC BY 4.0

need to subtract 240 from y coordinates

At the moment the datasets are always verified, even if download=False. Hashing huge files takes quite a while though, and being able to skip the verification would be great.

Thanks for a great library! We're starting to rely on it quite a bit at my group.

I can see some datasets, such as the NMNIST, relies on a Dropbox link. This may make complete sense, but it may also be beneficial to use decentralized peer-to-peer services using, for instance, IPFS. That should make the datasets more resilient to link expiration, service outage, and download bandwidth.

This could also be coupled with #111 to decrease the download size and increase unpacking/processing speed.

I tried downloading the poker-dvs dataset and it works fine.However ,trying other datasets (Navgesture for example) ,download starts and after a while at about 27% it gives me an error: File not found or corrupted (its raised in the function check_integrity from download_utils.py) but the files do exist in the desired directory nor do I make any changes while the files are downloading. I am running this in a linux system (18.04) , I installed tonic with conda.

First example under Readme has a dependency issue:

import tonic
import tonic.transforms as transforms

transform = transforms.Compose([transforms.Denoise(filter_time=10000),
                                transforms.TimeJitter(std=10),])

testset = tonic.datasets.NMNIST(save_to='./data',
                                train=False,
                                transform=transform)

testloader = tonic.datasets.DataLoader(testset, shuffle=True)

events, target = next(iter(testloader))

ModuleNotFoundError: No module named 'importRosbag.messageTypes'

tonic was installed via setup.py

maybe that's okay, but when building frame representations from the events afterwards, the frames are not cropped because they use dataset.sensor_size

Currently a lot of redundant extraction is happening when creating dataset objects to guarantee dataset consistency. Maybe this could be streamlined somehow. Mentioning @rob-luke @Jegp

For the moment it's only able to deal with event data from vision datasets

After a successful download and extraction of the NMNIST dataset, the check_integrity check keeps failing for consecutive calls. Resulting in the zip file being extracted every time the dataset is being applied.

The expected behaviour would be to reuse the existing directory structure in the ./data/nmnist/ folder.

apart from NCARS, which we already have, it would be nice to have access to some Gen4 camera datasets: https://docs.prophesee.ai/stable/datasets.html#datasets

Create an object interface such that it is easy to compose a pipeline out of objects that store meaningful state.

if we want to be able to chain torchvision transforms, we need to slightly change the way tonic transforms are instantiated. Currently we return events, images and sensor size from each one, but torchvision only works on a single piece of data (images). It would be good to bundle whatever a single sample of a tonic dataset returns into 1 tuple. That way, different transforms can be applied to each piece of data separately

test for many more variations including event arrays that are using dtypes, booleans for polarities, different orderings etc

setup.py etc.

I'm wondering why one would use a float type to return address-events, for example here:

In neuromorphic sensors or hardware, addresses generally results from AER and are thus better encoded as unsigned integers

maybe this can be evaluated lazily when combined with caching or written to disk

along the lines of https://github.com/pytorch/vision/tree/master/torchvision/datasets
makes it easy to get going quickly

some groups use seconds, some use microseconds

we currently include quite a large dependency for what we're using of it. Probably the most important functions we're relying on are pytorch visions download and verification utils https://github.com/pytorch/vision/blob/master/torchvision/datasets/utils.py
which could probably be copied and referenced, same goes for the VisionDataset class. Most transforms use numpy anyway, and for pytorch's sparse tensor transform we can just include it as an optional dependency.
This would help adoption by people who use other backends.

This seems like a "lower hanging fruit" than #116 or #111 but I feel it's likely to help significantly

randomly modify timestamps using e.g. Gaussian distribution

More a FYI than an actual issue - check this out: https://github.com/tub-rip/events_h52bag

some simulators such as Norse and SNNTorch use time x batch x channels dimensions. In order to save one permute call, we can introduce this additional parameter

i.e. SHD would have a sensor size of (700,) and a "tx" ordering to indicate it has no polarity and a DVS128 dataset would have a sensor size of (128,128,2) and a "xypt" ordering. Without this, the only way of knowing the actual number of input neurons is to check whether there are different polarities in the "p" row.

Sorry for making so many issues 😄