Write remaining unittests for the new (and old) components of this library.

Missing unittests are:

• utils.rs:
  • LearnRate
  • LearnMomentum
• nn.rs:
  • NeuralNet
• topology_v4.rs:
  • FullyConnectedLayer
  • ActivationLayer
  • AnyLayer
  • LayerSize
  • Topology
  • InitializingTopology
• layer:
  • ActivationLayer
  • FullyConnectedLayer
  • ContainerLayer
  • AnyLayer
    • utils module:
      • buffer_base module
      • matrix_base module
• trainer:
  • utils module:
    • MeanSquaredError
  • mentor module:
    • Context
    • Mentor
    • InitializingMentor

Implement parallel execution based on rayon via ndarray provided wrappers.

This is the initial step towards full parallel execution.

The entire neural network concrete layers were designed to be very memory efficient which resulted in a design with assymetric layer sizes (because of the way bias neurons are represented). This has led to several issues and implementation difficulties preventing the usage of more efficiently implemented lower-level algorithms provided by ndarray and other libraries.
This refactoring of layer sizes also helps with implementing the new layer and topology layer types to path the way for Convolutional Layers in the end.

The current topology builder is not flexible enough to use it for building up complex neural networks involving different layer types such as Convolutonal Layers.

The new topology build infrastructure should enforce a mirroring between abstract topology layers and concrete neural network layers.

Some thoughts on this matter has been made, results so far are:

• There could be a need of some kind of InputLayer in topology structure to represent the initial layer.
• Fully connected layers could be renamed to DenseLayers which is shorter.
• There is a need for a generic ActivationLayer that is only responsible for serving the activation function. For identity activation functions this has the effect of simply leaving the activation layer away.
• At first DenseLayer and ActivationLayer are fit to replace the current system. However, this structure makes it possible to later add layer types for convolutional layer computation to the set such as 2-dimensional PoolingLayer and ConvolutionLayer.
• For interoperability between convolutional layers (2D) and normal layers (1D) there needs to be a kind of conversion between the 2-dimensional layers and the 1-dimensional layers.

Right now the project looks dead on Github since there wasn't any activity on master for ~1 year, for most code parts even ~2 years.

That makes it hard to attract more developers (or even to explain to your colleagues that the project is not totally abandoned ...)

I think it would make sense to make next the new master for better visibility.

I don't see any recent activity on this repository, I was wondering if the library is still maintained or not.

Since many versions appveyor randomly and undeterministically fails to build and test Prophet. Especially the integration tests are causing trouble. Things are running fine on travisCI though.

For more information, visit: https://users.rust-lang.org/t/windows-undeterministic-test-timeout/12284

Currently Prophet handles too many (potential) errors with panics instead of proper Rust-style error handling via Result return type and proper error kinds etc.
This also leads to less testable code and prevents implementing better error reporting strategies and should be addressed.

Branch next already implements (decent) error kinds and a rust-style Error struct with typical interface impls. The next step is to adjust the rest of the code base to this format away from asserts.

Create utility abstractions that help to provide guarantees for invariants throughout the execution of the program. Also testability should be significantly improved by partitioning the program complexity into smaller tiles.

This includes but is not limited to following utility components:

• WeightsMatrix & DeltaWeightsMatrix: Abstraction built on top of Array2<f32> to more semantically provide an interface for weights matrices mainly used by FullyConnectedLayer.
• SignalBuffer: Abstraction to model signals instead of using raw Array1<f32>.
• ErrorSignalBuffer: Same as with SignalBuffer but specialized for gradients in gradient descent computation.

Certain layer kinds:

• FullyConnectedLayer: Abstracts mechanics of fully connected (dense) layers.
• ActivationLayer: A layer for activation application.
• ContainerLayer: A layer that contains other layers in a strictly sequential order.
• Layer, including variants for FullyConnectedLayer and ActivationLayer

Core functionality & utilities:

• NeuralNet: Implements the concrete interface for neural networks and contains layers that describe its computation.

Also model some traits that each stand uniquely for an important operation and behaviour shared by those abstractions:

• ProcessInputSignal: For internal (layer-based) feed forward operations.
• HasOutputSignal: For layers with output signals. In theory every layer requires this.
• CalculateOutputErrorSignal: For layers that work with gradient descent learning.
• HasErrorSignal: See above ...
• PropagateErrorSignal: See above ...
• ApplyErrorSignalCorrection: See above ...
• SizedLayer: Interface for working with input and output size of layers.