okozelsk / net Goto Github PK

Currently predictors are collected only once, after all pattern data are processed in the reservoirs.
Newly will be possible to define multiple collection of predictors during pattern data processing in the reservoirs.

Allow to build the resulting interconnection of neurons by successive application of interconnection schemes. Start with random (existing) and chain / ring schemes.

Novel general input unit replacing existing input neuron. The aim is to enable both analog and spiking approaches of input coding together while keeping the same time resolution of coding. Unit will provide original analog and corresponding "spike train" in paralell. Spike train will be a horizontal bitwise representation of an analog value (in required precision - sensitivity).

In addition to the existing horizontal spike coding (neural population), enable also classical vertical coding (spike-train) and direct analog coding for spiking hidden neurons.
Horizontal coding converts an analog value to a simultaneous activity of spiking input neurons population at the same time T. Vertical coding converts the analog value to a spike train on a single input neuron within the given post time frame (so there will be more reservoir cycles and thus much slower reservoir processing).
Direct analog coding means no conversion of analog value to spikes (analog value is used directly as a stimulation of hidden spiking neuron).

Pseudoalg:
Set weights for "One winner" member clusters. Weight is cluster's total error from cross-validation / number of positive samples
Sort weighted clusters' predictions in descending order
Select the first (the winner)

Introduce common separate input encoder providing input neurons for reservoirs (analog neuron and population of spiking neurons per each input feature).

• Add spike code specification at the input fields level
• Create new component "InputEncoder" processing all inputs, providing all input neurons to be connected to reservoir instances
• Discard current InputUnit component from ReservoirInstance, keep here only input connections specification

Warning: This will have a major impact on setup xml.

• Introduce chainable input transformations (univariate, bivariate). Transformed input fields will have the same behavior as external input fields
• Introduce Constant, Linear and Nonlinear short-term plasticity dynamics for the synapses
• Introduce homogenous excitability of spiking hidden neurons
• Remove excitatory and inhibitory role of hidden neurons (neuron groups). Synapse takes over the excitatory and inhibitory role
• Simplification of interconnection configurations

Introduce definition of time-independent input fields within the patterned input feeding context and possibility to route them to the readout layer once.
It enables possibility to combine "static" and preprocessed "dynamic" information at the readout level.

The configuration of predictors will now only be at the level of a group of neurons. Instead of a set of bool attributes and subsequent specification of parameters in separate child xml elements, each predictor will now have its own xml element. Multiple use of a predictor of the same type but with different parameter values will be allowed. The set of available predictors will be revised, unused predictors will be removed, used predictors will be generalized, and new predictors will be added.

Aim is to make StateMachine able to solve both time dependent and time independent problems efficiently.

Helper component proposing State Machine configuration. It must be simple to use it.

Before cross-validation exclude data set of specified size for further calibration. After cross-validation use excluded data to train new network where inputs will be predictions of trained sub-networks (folds). Final cluster prediction will be the prediction of this new trained network.

Wrong way :-(

Introduce more sophisticated weighting and postprocessing of the particular results produced by cluster member netwoks.
For each cluster member network, weighting to be taking into account number of samples and achieved accuracy during training and testing. Finally softmax to be used to balance member networks weights.
Introduce optional 2nd level network for computation on cluster, where computed outputs from member networks is an input for that network.

Prepare a tutorial application with code examples of using RCNet components

Bring possibility to setup StateMachine and other components directly from the code.

Try to introduce anti-spike neurons. The goal is to improve descriptivity of reservoir's predictors by enabling "mirrored" activity to be projected in a neural activity and thus also in the predictors.
Density of anti-spike neurons should be a parameter at analog-group and spiking-group level.

Ensure that every component of the build chain has its own progress event and provides only information relevant for that component and its subordinate components.

SQNL

SQNL derivative

Remove the distinction between input and internal synapses. Remove the synapse's interface and base class and simply implement a synapse supporting both signal delay and short-term plasticity.

Ongoing investigation...

• Parallel processing where it is missing
• Exceptions refactoring
• Comments revision
• RCNetTypes.xsd (missing constraints, revision of comments and annotations)
• SMDemoSettings.xml (projection of recent changes, revision of comments)

Add more code examples showing an use of non-recurrent networks (Cluster, Cluster Chain and theirs builders) as standalone components with no relation to State Machine.
Add sub-menus to application root menu to make navigation better structured.