nengo / keras-spiking Goto Github PK

From what I can tell this tool only supports "relu". Is this accurate?

Hello,

I have been recently using KerasSpiking, but I am still not sure which neuromorphic hardware does it simulate ?
For example,
I am running this code, but I am not sure which neuromorphic hardware does the simulator use to run the SNN ?

Hi community,
excellent work. I am curious if this activation layer can be applied in a CNN network. Is there any example similar to this https://www.nengo.ai/keras-spiking/examples/spiking-fashion-mnist.html but using CNNs?

The input to the network would be of (T,28,28,1) dimensionality if we follow the previous example. How would the network have to be modified?

Best regards

This was revealed to be broken in nengo/nengo-dl#187. See nengo_dl/tests/test_converter.py:test_spiking_activation.

Would be nice to support spiking activations within RNNs such as the keras-lmu or an LSTM. Ideally this would just be a matter of supplying a different activation argument to the RNN. Currently this can be done with a stochastic (i.e., stateless) spiking model (e.g., activation = StochasticSpiking("tanh")):

# The following code is made available under the KerasSpiking license:
# https://www.nengo.ai/keras-spiking/license.html

import tensorflow as tf


def pseudo_gradient(forward, backward):
    """Multiplexes between one of the forward or backward tensors."""
    # The following trick can be used to supply a pseudo gradient. It works as follows:
    # on the forwards pass, the backward terms cancel out; on the backwards pass, only
    # the backward gradient is let through. This is similar to using tf.custom_gradient,
    # but that is prone to silent errors since certain parts of the graph might not
    # exist in the context that any tf.gradients are evaluated, which can lead to None
    # gradients silently getting through.
    return backward + tf.stop_gradient(forward - backward)


class StochasticSpiking:
    """Turn a static activation into a spiking one using stochastic rounding.

    Similar to ``nengo.neurons.StochasticSpiking``.

    The effective spike rate is controlled by ``dt``. Specifically, if ``a = f(x)``
    is the static activity, then at most ``ceil(abs(a * dt))`` spikes are generated
    per time-step.

    Uses the gradient of the underlying activation function on the backward pass.
    """

    def __init__(self, wrapped_activation, dt=1):
        self._activation = tf.keras.activations.get(wrapped_activation)
        self._dt = dt

    def __call__(self, x):
        a = self._activation(x)
        y = a * self._dt

        # Apply stochastic rounding: y -> y_rounded.
        n = tf.math.floor(y)
        r = y - n
        y_rounded = n + tf.cast(tf.random.uniform(shape=tf.shape(x)) < r, dtype=x.dtype)

        a_rounded = y_rounded / self._dt
        return pseudo_gradient(forward=a_rounded, backward=a)

Otherwise I've been unable to get the SpikingActivation layer or SpikingActivationCell to work within RNNs.

There are two features I've found myself needing lately w.r.t. the lowpass layer:

1. (Non-Trainability) The ability to make the time-constant non-trainable. apply_during_training is close, but setting this to False skips over the lowpass entirely during training. I still want the lowpass in the forward training pass. I just don't want its time-constants to be modified from the initial value that I've provided.

2. (Homogeneity) The ability to learn only a single time-constant. Currently the initial tau is broadcasted like so:
   

   keras-spiking/keras_spiking/layers.py

   Lines 582 to 583 in 116fc6c

   	shape=[1] + self.state_size.as_list(),
   	initializer=tf.initializers.constant(np.ones(self.state_size) * self.tau),

   
   such that a different tau is learned for each dimension. Sometimes prior knowledge can tell us that the time-constant should be the same. This would also make trained lowpass filters compatible with NengoDL's converter (see nengo/nengo-dl#60 (comment)). But even independently of that, I've encountered a sitaution where I'd like to be able to learn just a single time-constant, and then change the shape of the data going into the layer at inference time (i.e., have the single lowpass that is broadcast across all of the dimensions, and also with the same initial values).

I am trying to get some energy values from a DenseNet architecture and when I run the code below I get a valueError (as shown below). I tried narrowing down where the issue occurs and found that it first happens at a layers.Concatenate call in the keras implementation. Has anyone seen a similar issue to this? When I use keras’ vgg16 implementation it works, which doesn’t have a Concatenate.

from tensorflow.keras.applications import DenseNet121
import keras_spiking
import numpy as np

denseNet_model = DenseNet121(
weights=None, include_top=True, input_shape=(224, 224, 3)
)

energy = keras_spiking.ModelEnergy(denseNet_model, example_data=np.ones((32, 224, 224, 3)) * 3)

ValueError: could not broadcast input array from shape (32,56,56,64) into shape (32,56,56)