Light

fl03 / concision Goto Github PK

View Code? Open in Web Editor NEW

2.0 1.0 0.0 1.26 MB

Concision is a toolkit for building machine-learning models in Rust.

Home Page: https://fl03.github.io/concision/

License: Apache License 2.0

RenderScript 0.12% Rust 99.88%

artificial-intelligence crates-io neural-network rust-lang toolkit

concision's Introduction

Concision

The library is currently in the early stages of development and is not yet ready for production use.

Concision is designed to be a complete toolkit for building machine learning models in Rust.

Concision is a machine learning library for building powerful models in Rust prioritizing ease-of-use, efficiency, and flexability. The library is built to make use of the both the upcoming autodiff experimental feature and increased support for generics in the 2024 edition of Rust.

Getting Started

Building from the source

Start by cloning the repository

git clone https://github.com/FL03/concision.git
cd concision

cargo build --features full -r --workspace

Usage

Example: Linear Model (biased)

    extern crate concision as cnc;

    use cnc::prelude::{linarr, Linear, Result, Sigmoid};
    use ndarray::Ix2;

    fn main() -> Result<()> {
        tracing_subscriber::fmt::init();
        tracing::info!("Starting linear model example");

        let (samples, d_in, d_out) = (20, 5, 3);
        let data = linarr::<f64, Ix2>((samples, d_in)).unwrap();

        let model = Linear::<f64>::from_features(d_in, d_out).uniform();
        // let model = Linear::<f64, cnc::linear::Unbiased>::from_features(d_in, d_out).uniform();

        assert!(model.is_biased());

        let y = model.activate(&data, Sigmoid::sigmoid).unwrap();
        assert_eq!(y.dim(), (samples, d_out));
        println!("Predictions:\n{:?}", &y);

        Ok(())
    }

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

Please make sure to update tests as appropriate.

License

concision's People

Contributors

Stargazers

Watchers

concision's Issues

Reinforcement Learning

Reinforcement learning is a type of machine learning that involves training an agent to take actions that maximize a reward signal.

Description
Implement a complete framework for building reinforcement learning models in pure Rust.

Create a generalized structure suitable for handling information

Due to the lack of generalization present in Rust, it is important to consider exactly what any given model is operating on. Standardizing this would provide the system with the intended flexibility and may even have performance benefits.

Scaled Dot-Product Attention

Formula

$$\text{Attention}\left({Q, K, V}\right)=\text{softmax}\left(\frac{Q\cdot{K^T}}{\sqrt{d_k}}\right)\cdot{V}$$

Implement a LinearParam store

cnc::linear::Params

Fix the `blas` feature for the crate

Design a procedural macro (`nn`) which streamlines the setup of a neural networks

Development of this macro officially kicks off a long-standing desire to create a DSL of-sorts for building powerful, agile, and scalable neural networks.

CNC 0001 - Implement the core behaviors of a Neural Network

A neural network is a set of layers that work together in-order to make a prediction.

Traits

Backward
Compile
Forward
Train

Implement an Actor structure for representing the learnable activation function

Implement the behavior of an Optimizer

Implement additional tests for implemented distributions

Look into creating an `Initializer`

Feature Request: Begin implementing aggregative utilities

Create a Module trait that describes the behaviors of modules in a given neural network

Resources

PyTorch Module

Create a `TruncatedNormal` distribution

TruncatedNormal is similar to Normal distributions, however, any values exceeding two times the standard deviation are discarded and regenerated.

Optimize the `Merge` and `Split` ops

Create an efficient store for managing the learnable parameters of any given model or module

Implement the Scaled Dot-Product Attention Mechanism

Scaled Dot-Product Attention is a mechanism most extensively leveraged in the popular Transformer model.

Resources

Attention is All You Need

CI/CD: Migrate to the new GitHub Action's templates

Implement an `Encoder` layer for the `Transformer` model

Tracking issue for the `Transformer` model

The transformer model was introduced in 2017 with the release of the Attention is all you need paper. Over the last few years, the popularity of the Transformer model has grown exponentially due to its abilities in natural language processing (NLP) workloads.

Transformers are mathematically simply, primarily relying on dot-product / matrix multiplication algorithms.

Issues

Pull Requests

Resources

`LecunNormal`: a truncated normal distribution

LecunNormal initializes objects using a truncated normal distribution that is centered at 0 and has a standard deviation of 1/n where n is the number of inputs.

Create the `concision-s4` crate

Implement a struct and several traits for a layer in a neural network

Create a set of traits that adequately describe the behaviors of a layer in a neural network
Implement a Layer structure

Tracking issue for `pre-processing` algorithms

Implement the `Decoder` for the `Transformer` model

Bayesian Network

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Fix the initialization methods

Currently the rely on the wrong dimension, thus altering the shape.

Generally clean up the crate

Currently, concision is far too clunky and needs to be cleaned up before any additional work can take place.

Multi-Head Attention

General Formula

$$\text{MultiHead}({Q, K, V}) = \text{concat}({\textbf{z}_0, ..., \textbf{z}_n })\cdot{W^O}$$

Update the Artificial Neurons

Artificial neurons are given two free-parameters, bias and weights, that are to be learned through training. The bias describes an array of values that don't require any input to have an affect.

Tracking issue for the Structured State Space Sequence Model (`S4`)

Issues

#86

Pull Requests

[ ]

Resources

Decision Trees

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Implement `Xavier` distributions

`XavierNormal`

Each weight is drawn from a normal distribution where the mean $\left({\mu}\right)$ is set to 0 and the standard deviation $\left({\sigma}\right)$ is computed as follows:

$$\sigma = \sqrt{\frac{2}{d_{in} + d_{out}}}$$

`XavierUniform`

Each weight is drawn from a uniform distribution in $\left[{-x, x}\right]$

$$x = \sqrt{ \frac{ 6 }{ d_{in} + d_{out} } }$$

Resources

Understanding the difficulty of training deep feedforward neural networks

`InitializeExt`: extends the `Initialize` trait with methods corresponding to various distributions

Methods

lecun_normal
normal
stdnorm
uniform

Fix the Fast-Fourier Transform of Real-Valued Signals

Create functions that consider Real-Valued inputs.

Updates

Clustering

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Upgrade the id to `Uuid`

Use either uuid::Uuid or another Copyable type for ids.

Implement Layer Normalization (`LayerNorm`)

Layer normalization directly estimates the normalization statistics from the summed inputs to the neurons in a hidden layer.

Model

Configuration

Parameters

The implementation can leverage the ParamsBase created for linear models.

Bias (optional)
Weight

References

Update the casual convolution function

The casual convolution function currently considers two array's of equal dimensions, however, the generated kernel is defined to be a one-dimensional array by the 'kernel_dplr' function.

Neural Networks

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Implement the piece-wise feed-forward neural network for the `Transformer` model

Implement an `NdTensor` trait

This trait should unify all numeric operators implemented in the crate and streamline several additional processes.

CNC 0000 - Implement a Tensor structure

A tensor is defined to be an n-dimensional array.

Reduced complexity with relaxed dimensional typing

References

Tracking Issue for `Linear` modules

Describe the proposal or feature that this issue is tracking.

Issues

Pull Requests

Implement the State-Space Model Scanner

In the paper, the Annotated S4, they leverage the scan function from jax. While Rust has a well-developed scan feature present in iterators, it seems to differ in execution than the one from jax.

Tracking issue for the `KAN` model

Issues

#87

Pull Requests

[ ]

Resources

Kolmogorov-Arnold Networks (KAN)

Implement an `Initialize` trait

Tracking issue for `init`

This is a tracking issue for initialization primitives.

Issues

Custom Distributions

Implement plotting capabilities

Implement a `Dropout` feature

References

Improving neural networks by preventing co-adaptation of feature detectors

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.