aa-samad / conv_snn Goto Github PK

View Code? Open in Web Editor NEW

119.0 2.0 26.0 77 KB

Code for "Convolutional spiking neural networks (SNN) for spatio-temporal feature extraction" paper

License: GNU General Public License v3.0

Python 94.92% MATLAB 4.56% Dockerfile 0.33% Shell 0.18%

snn neuromorphic cifar10 spatio-temporal-analysis cnn convolutional-neural-networks deep-learning pytorch

conv_snn's Introduction

Conv-SNN

Convolutional spiking neural networks (SNN) for spatio-temporal feature extraction

This paper highlights potentials of Convolutional spiking neural networks and introduces a new architecture to tackle training deep convolutional SNN problems.

Prerequisites

The Following Setup is tested and it is working:

Python>=3.5
Pytorch>=0.4.1
Cuda>=9.0
opencv>=3.4.2

Docker

Set up the environment where all the programs can run
- Run ./run.sh

Data preparation

Download CIFAR10-DVS dataset
- Extract the dataset under DVS-CIFAR10/dvs-cifar10 folder
- Use test_dvs.m in matlab to convert events into matrix of t, x, y, p (make sure to adjust the test_dvs.m folder addresses inside the code)
- Run python3 dvscifar_dataloader.py to prepare the dataset (make sure to have files like dvs-cifar10/airplane/0.mat inside main.py directory)

Training & Testing

CIFAR10-DVS model
- Run python3 main.py
Spatio-temporal feature extraction tests
- For each architecture simply run main file with python3
Note: There are problems with training SNNs, such as extreme importance of initialization; Therefore, you may not reach the highest accuracy as mentioned in the paper. The solution is to try other torch versions and parameters or contact me / make an issue if you truly need the highest accuracy.

Citing

Please adequately refer to the papers any time this Work is being used. If you do publish a paper where this Work helped your research, Please cite the following papers in your publications.

@misc{samadzadeh2020convolutional,
        title={Convolutional Spiking Neural Networks for Spatio-Temporal Feature Extraction},
        author={Ali Samadzadeh and Fatemeh Sadat Tabatabaei Far and Ali Javadi and Ahmad Nickabadi and Morteza Haghir Chehreghani},
        year={2020},
        eprint={2003.12346},
        archivePrefix={arXiv},
        primaryClass={cs.CV}
    }

conv_snn's People

Contributors

Stargazers

Watchers

conv_snn's Issues

It looks like the latest code hasn't been uploaded yet

Hello author, it looks like the latest code hasn't been uploaded yet. If possible, could you upload it? This will be very helpfu to me. Thanks!

about the hybrid training method proposed in the paper

Hello author, it seems that there is no hybrid training method proposed in the paper (STS-ResNet on the CIFAR10-DVS dataset) in the code implementation. Is this the final version of the code?

Thanks for sharing the code of this paper. I am still wondering if current code is the latest for the paper?

Thanks for sharing the code of this paper. I am still wondering if the current code is the latest for the paper? As I find that some people find some bugs in this code. Thanks.

Parameter or weight for high accuracy

Hi,
We are trying to make some further developments based on your methods. Is it possible to share the training parameters to achieve the high accuracy or even the trained weight for CIFAR10-DVS?
Thanks.

@meltingCat from torch.optim import lr_scheduler

          @meltingCat   from torch.optim import lr_scheduler

Originally posted by @wangxiao5791509 in #11 (comment)
after importing am getting the following error
'module' object is not callable

NameError: name 'lr_scheduler' is not defined

Running the code in folder Tests_1-3_and_5. It seens that all the code had the same error below.

8192it [00:02, 3912.96it/s]                                                                                               
9920512it [00:19, 1117448.86it/s]                                                                                         Epoch [1/10], Step [100/600], Loss: nan
Time elasped: 11.564528942108154
                                                                                                                         Epoch [1/10], Step [200/600], Loss: nan
Time elasped: 22.981971263885498]                                                                                         
Epoch [1/10], Step [300/600], Loss: nan
Time elasped: 34.35000038146973
Epoch [1/10], Step [400/600], Loss: nan
Time elasped: 45.67948770523071
Epoch [1/10], Step [500/600], Loss: nan
Time elasped: 56.9581983089447
Epoch [1/10], Step [600/600], Loss: nan
Time elasped: 68.23811769485474
Traceback (most recent call last):
  File "main.py", line 94, in <module>
    optimizer = lr_scheduler(optimizer, epoch, learning_rate, 40)
NameError: name 'lr_scheduler' is not defined
9920512it [01:22, 120381.43it/s] 
1654784it [01:12, 22939.77it/s]

NMNIST

Thanks for your code. And I guess how this code run on the NMNIST dataset. Could you provide an example?

implementation errors

hello. I am a student reproducing your paper.
I found the following problem in the code.

AveragePooling is removing binarization characteristics of spike input.
Spatial dropout is removing the binarization feature.

Actually, I can't understand that part of the paper.
"they(avgPool) do not interfere with the binary nature of the architecture (a fact never mentioned before in previous works"
Event-based spikies generally have sparse data, so of course the binary property is lost during the average pooling process (because of 0).

Also, I confirmed that a value greater than 1 occurs due to the x/(dropout weight) formula in the spatial dropout.

+) In addition, when constructing the resnet block, it was confirmed that it goes through Convolution immediately after Convolution. This is an implementation that doesn't make sense.

I want to know if there is an exact implementation code.

First convolution is on input instead of synapse output

The first convolution at here: https://github.com/aa-samad/conv_snn/blob/master/CIFAR10-DVS/network.py#L158

is on input spikes. Figure 3 in the paper shows that it should be on the synapse output: https://arxiv.org/pdf/2003.12346.pdf

about seq5 in synthetic dataset

In you paper, the seq5 example in Figure 4 is rotated with a random degree between 0 and 360 degrees. but it can't be distinguished between clockwise or counterclockwise. Because i can't find some evidence for rotation direction. In other words, clockwise 30 degree is equivalent to counterclockwise 270 degree, so how you can classify the rotation direction.

RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

Got an error in Training & Testing

(conv_snn) test@test-dual1060:~/Desktop/project/conv_snn/CIFAR10-DVS$ python3 main.py
device cuda
  0%|                                                                                                      | 0/1750 [00:00<?, ?it/s]
==> Build model and setup loss and optimizer
==> Epoch:[0/500][training stage]
  0%|                                                                                                      | 0/1750 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "main.py", line 231, in <module>
    main()
  File "main.py", line 65, in main
    model.run()
  File "main.py", line 124, in run
    self.train_1epoch()
  File "main.py", line 172, in train_1epoch
    prec1, prec5 = accuracy(output.data, label.cuda(), topk=(1, 5))
  File "/home/test/Desktop/project/conv_snn/CIFAR10-DVS/utils.py", line 30, in accuracy
    correct_k = correct[:k].view(-1).float().sum(0)
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

Typos in code

line: https://github.com/aa-samad/conv_snn/blob/master/CIFAR10-DVS/dvscifar_dataloader.py#L117
should be:
torch.save([torch.Tensor(frames), torch.Tensor([labels,])],

line: https://github.com/aa-samad/conv_snn/blob/master/CIFAR10-DVS/main.py#L211
should be:
output = self.model(data0.cuda())

line: https://github.com/aa-samad/conv_snn/blob/master/CIFAR10-DVS/main.py#L217
should be:
prec1, prec5 = accuracy(output.data, label.cuda(), topk=(1, 5))

line: https://github.com/aa-samad/conv_snn/blob/master/CIFAR10-DVS/main.py#L227
should be:
return top1.avg

line: https://github.com/aa-samad/conv_snn/blob/master/CIFAR10-DVS/main.py#L133
should be:
prec1 = self.validate_1epoch()

I would also get rid of this line: https://github.com/aa-samad/conv_snn/blob/master/CIFAR10-DVS/main.py#L132

line: https://github.com/aa-samad/conv_snn/blob/master/CIFAR10-DVS/network.py#L165
should be:
x = torch.cat(c2_spike[1::2], dim=1)

file path for record_info

It might be better to do a pull request for this, but it is only a couple of lines..

First call to record_info will break if correct folders are not created manually. it can be fixed by adding the following lines before https://github.com/aa-samad/conv_snn/blob/master/CIFAR10-DVS/utils.py#L85

if not os.path.exists(os.path.split(filename)[0]): os.makedirs(os.path.split(filename)[0])