shiffman / neural-network-p5 Goto Github PK

View Code? Open in Web Editor NEW

221.0 23.0 39.0 5.82 MB

Deprecated! See:

Home Page: https://github.com/CodingTrain/Toy-Neural-Network-JS/

JavaScript 100.00%

neural-network machine-learning neuroevolution p5js javascript genetic-algorithm

neural-network-p5's Introduction

Deprecated!

new repo: https://github.com/CodingTrain/Toy-Neural-Network-JS/

Simple Artificial Neural Network JavaScript library

This repository is a library for creating simple vanilla 3-layer ANNs in JavaScript. I'm using it for the second edition of the Nature of Code book, as well as examples for my ITP course: Intelligence and Learning.

At the moment this library is depends on p5.js. However, it's my intention to remove this dependency for the library itself (while still making examples using p5): #10. I also intend to port this library to Java for Processing: #11.

Finally, this library has a terribly inefficient matrix implementation and should likely include options for using math.js and/or gpu.js.

The code is based on the book Make Your Own Neural Network by Tariq Rashid (book source code).

Example Demos

The neuro-evolution examples are inspired by the chrome experiment Flappy Learning by xviniette.

Use

// Creating a Neural Network with # of inputs, hidden neurons, and outputs
var inputs = 4;
var hidden = 16;
var outputs = 2;
var nn = new NeuralNetwork(inputs, hidden, outputs);

// Training the Neural Network with inputs and known outputs
var inputs = [-0.3, 0.5, 0.3, 0.2];
var targets = [0.99, 0.01];
nn.train(inputs, targets);

// Querying the Neural Network with inputs
var inputs = [-0.3, 0.5, 0.3, 0.2];
var prediction = nn.query(inputs);

By default, the library will use a sigmoid activation function. However, you can select other activation functions as follows (tanh only at the moment)):

var nn = new NeuralNetwork(inputs, hidden, outputs, 'sigmoid');

var nn = new NeuralNetwork(inputs, hidden, outputs, 'tanh');

neural-network-p5's People

Contributors

Stargazers

Watchers

neural-network-p5's Issues

Videos for the new code?

I see you have improved the library quite much, can you at least make a video expaining what has been done?

Performance - MNIST demo

Right now it calculates a percentage completed since the sketch first begins. There's probably a better way to think about this.

Drawing - Damage and Distortion

One of the key nice things about neural networks is that they are resilient to some damage.

That means either damage to the network itself (e.g. broken links), or .. more interestingly .. damage to the input image.

It might be interesting for users to draw a digit, then click "damage" and "distort" buttons which would blow holes in the image, or stretch/wobble/twist the image a bit.

This would allow users to see for themselves that you can damage an input image quite a bit before overall performance degrades. I did some experiments myself at http://makeyourownneuralnetwork.blogspot.co.uk/2016/03/your-own-handwriting-real-test.html

retrain network if guesses user digit incorrectly - MNIST demo

This would require an interface to allow the user to specify the drawn digit.

Seems like examples aren't running

Yo Daniel! I tried to run the steering and flappy bird examples with provided URLs in README description, but they seem to be missing your library somehow.

https://shiffman.github.io/Neural-Network-p5/examples/neuro-evolution/flappy/

So yeah, not super important to fix, but just would be nice so I can show my friends!

Pausing - MNIST demo

You should probably still be able to draw digits while "paused". The pause should just be for the training/testing.

There seems to be a problem with the matrix library

 My code:

var nn;
function setup() {
nn=new NeuralNetwork(2,3,5);
}
function draw() {
}

 The error:

172: Uncaught SyntaxError: Block-scoped declarations (let, const, function, class) not yet supported outside strict mode
"%cDid you just try to use p5.js's str() function? If so, you may want to move it into your sketch's setup() function.\n\nFor more details, see: github.com/processing/p5.js/wiki/Frequently-Asked-Questions#why-cant-i-assign-variables-using-p5-functions-and-variables-before-setup"
73: Uncaught ReferenceError: Matrix is not defined

Port to Java / Processing

Sigmoid derivative in nn.js

As I read it, your formula for the derivative of the sigmoid function is wrong in nn.js.
You have

NeuralNetwork.dSigmoid = function(x) {
  return x * (1 - x);
}

but it should be

NeuralNetwork.dSigmoid = function(x) {
  var y = NeuralNetwork.sigmoid(x);
  return y* (1 - y);
}

Reference

The style and substance of your 'Coding Train' material is very enjoyable. 👍 Thank you.

[Improvement] Demo ⁉️

It would be great if you could have a main website with demos, I mean a thing that you can experiment with, without having to write a line of code.

Using a matrix library

I like that this example has no other dependencies but could be nice to have a branch that explores math.js or gpu.js.

Example - visualizing the network

The network itself could be visualized (using greyscale colors to show weights, hidden layer nodes, etc.)

Reload a pre-trained model

Right now there is a button to download a JSON file with all of the network weights. It would be nice to add a feature to reload weights.

Sometimes not working properly

I don't know why but sometimes (often) the outputs values seems weird, for this simple sketch:


function setup() {
  createCanvas(800, 600)

  // Creating a Neural Network with # of inputs, hidden neurons, and outputs
  nn = new NeuralNetwork(2, 4, 1);
  // Training the Neural Network with inputs and known outputs
  for (var i = 0; i < 20000; i++)
  {
  	nn.train([0,0], [0]);
  	nn.train([0,1], [1]);
  	nn.train([1,0], [1]);
  	nn.train([1,1], [0]);
  }

  console.log(nn.query([0,0]));
  console.log(nn.query([0,1]));
  console.log(nn.query([1,0]));
  console.log(nn.query([1,1]));
}

function draw() {
  background(51)
}

I obtain something like this (sometimes, often):

[0.014234893059839713] [0.9856006837596043] [0.4997097129483589] [0.5001547703319872]

Backquery - MNIST demo

I'd like to demonstrate a "back query" (feeding a desired output backwards through the network and getting pixels as a result) with this example. In @makeyourownneuralnetwork, the back query concept is explained and a sample image is provided (for the digit 0):

Perhaps I'm too impatient but after running through 10,000 training images, my backquery for zero looks like:

I think I'm missing something. Code in progress is in the backquery branch:

https://github.com/shiffman/Neural-Network-p5/blob/backquery/nn.js#L127

neuroevolution flappy bird .mutate() missing

hi! just wanted to know how you mutated the brain/neural net? what aspects of it did you mutate? the hyper parameters? the # of hidden layers? # of neurons per hidden layer? i just noticed that it calls a fcn .mutate() but i couldn't find any function related to that.

thanks!

Remove p5 dependancies

No reason to require p5 for use of nn.js and matrix.js