Comments (21)
And one example more. Just for history.
I have training data with 292 different columns, and by default hiddenLayers equal to [145]
. And come out very large nn (as json) and very slow training.
But if I set custom hiddenLayer like as [3, 3] that's make nn better and less (~5Mb vs 50Kb).
So strange =)
from brain.js.
The first citation you reference is from:
if (!hiddenSizes) {
sizes.push(Math.max(3, Math.floor(inputSize / 2)));
} else {
hiddenSizes.forEach(size => {
sizes.push(size);
});
}
This context would be when hiddenSizes
aren't defined, as you mentioned a default value, otherwise it uses whatever you give it for hiddenSizes
.
The question: "what size should my hidden layers be?!?!" is always one I see tossed around a lot, and it seems there is a lot of debate on what makes it a better size. I tend to usually use defaults, and try and tweak but have never really found an optimum set of sizes.
I do know that it is easier for the net to go deeper (ie deep learning which is 3 or more) hidden layers than wider based off of research, and that sounds like it would make sense, as there are more layers to abstract patterns into, rather than the other than flattening them immensely. If you were to go even deeper, say [3, 3, 3]
or [145, 50, 3]
the same rule applies. Deeper is better than wider. What does your output look like?
from brain.js.
For reference and credit: http://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw
from brain.js.
After when I wote this, i divided training data to train set and test set (90% - 170 samples and 10% - 20 samples). And what I got:
For hiddenLayers [200] I got accuracy 65% on test data set and 0.002 training error.
For hiddenLayers [50] I got acuracy 60% on test data set and 0.001 training error.
Deeper is better than wider.
Okey, I will testing with 2- and 3- count of hiddenLayers array.
In some sources I readed what neuro networks with more that 3 hidden layers not effective.
Some info about my data, if it interesting for you.
I have some commands and their effecienty as from 0 to 0.5 is bad effective and from 0.5 to 1.0 is good effective.
For example, my data look as:
{
input: {
"m_id_58374": 1, // leader id, he is main participiant, always 1
"e_id_254123": 0.5, // member of group, he involved in project 50%
"e_id_914260": 0.25,
"e_id_535412": 0.2,
"e_id_530421": 0.04,
"e_id_228592": 0.01
},
output: [0.29] // points of result for this command, mean bad result as 42% ((0.5 - 0.29)*2*100)
}
And nn config:
var net = new brain.NeuralNetwork({
hiddenLayers: [200] // here I past different values for experiments
});
var trainResult = net.train(trainingData, {
errorThresh: 0.001,
iterations: 50000,
learningRate: 0.2,
callbackPeriod: 1000,
callback: function(state) {
// run on testing data set
}
});
from brain.js.
And about errorThresh. In process of train():
err: 0.087; accuracy on test data: 79%;
err: 0.004; accuracy on test data: 67%;
err: 0.003; accuracy on test data: 65%;
err: 0.002; accuracy on test data: 62%;
err: 0.001; accuracy on test data: 59%;
@robertleeplummerjr wat? NN very strong adapted for training data?
Is it overfitting?
from brain.js.
Can you ask that question a different way? I apologize but I don't understand.
from brain.js.
Yep, my english very good :) Sorry!
In config of net I set errorThresh: 0.001
and see train log from callback:
state.error: 0.087; accuracy on test data: 79%
state.error: 0.004; accuracy on test data: 67%
state.error: 0.003; accuracy on test data: 65%
state.error: 0.002; accuracy on test data: 62%
state.error: 0.001; accuracy on test data: 59%
As you can see with a decrease in error is reduced and accuracy in the test sample.
And my question - is it overfitting? Do I need increase errorThreshold in this case?
P.S. At this moment I set 0.0025 and this look better.
from brain.js.
How long are you training for?
from brain.js.
How long are you training for?
For example,
for hiddenLayers [20,20,5] 73 sec. and 2800 iterations
for hiddenLayers [120,20,5] 840 sec. and 5400 iterations
And yes, I use cycle for get better hiddenLayers param =)
for (var iLayer_3 = 5; iLayer_3 <= 30; iLayer_3+=5) {
for (var iLayer_2 = 10; iLayer_2 <= 100; iLayer_2+=10) {
for (var iLayer_1 = 20; iLayer_1 <= 200; iLayer_1+=20) {
from brain.js.
On the issue
Option hiddenLayers. Ask about readme
I wanted to write "question" instead "ask".
And my question about readme. It is not written about dividing input data size and it's confusing. Is not it?
from brain.js.
How much data are you training with?
from brain.js.
167 samples same as I wrote
19 samples for test
from brain.js.
Any way you could increase the data into a few thousand samples?
from brain.js.
I don't think. It's real data from database.
Maximum what I can – to attempt get error about 70..75% on test data. It's norm for my issue.
from brain.js.
Results of my experiment with double layer.
Net trained to 0.0025 error on each param hiddenLayers.
I wrote just best results with accuracy on test data over 66%:
(columns - count neurons in 1 layer, row - count neurons in 2 layer)
20 | 40 | 60 | 80 | 100 | 120 | 140 | 160 | 180 | 200 | |
---|---|---|---|---|---|---|---|---|---|---|
10 | 68.23 | 67.10 | 67.89 | 66.27 | ||||||
20 | 67.34 | 67.70 | 66.67 | |||||||
30 | 67.12 | |||||||||
40 | 66.22 | |||||||||
50 | ||||||||||
60 | ||||||||||
70 | 66.17 | |||||||||
80 | 66.34 | |||||||||
90 | ||||||||||
100 | 66.90 | |||||||||
110 | 66.25 | |||||||||
120 | ||||||||||
130 | 66.25 | |||||||||
140 |
I think what you right about [145, 50, 5], I will continue experiments around these numbers. Thanks!
from brain.js.
This is really fascinating! It would be nice to arrive at some formula to understand the optimum size to go with. Too, keep in mind, it will take much longer to train, but you can go deeper than 3 layers.
from brain.js.
This is really fascinating! It would be nice to arrive at some formula to understand the optimum size to go with.
I think, these parameters depend havily on type train dataset.
Too, keep in mind, it will take much longer to train, but you can go deeper than 3 layers.
Could not get a good result as at 4-layers network. And it's really slow.
At this moment I have best result with hiddenLayers = [180,30,5]
and 70.13% of accuracy.
I'll keep you informed :)
from brain.js.
It would be nice to arrive at some formula to understand the optimum size to go with.
About optimization, what I found.
I use callback for check accuracy on test data and... training data, and what I got – on state.error = 0.01
accuracy on training data is around 90%, it's enough result for my task and it does not make sense to continue. It's saves a lot of time for train()
. Because usual I set errorThresh
in range 0.001..0.0025, now I set errorThresh
to 0.01.
Success story! :)
from brain.js.
what did your layers look like in the end?
from brain.js.
While in the process.
Now I experimenting with hidden Layer [X, Y, 5].
And I think what in a future set errorThresh even more... because in two direction - accuracy on test data and training data train() make these cross.
For example train() for some net:
iterations | accuracy on training data | accuracy on test data |
---|---|---|
0 | 75% | 80% |
100 | 88% | 74% |
500 | 90% | 68% |
1000 | 98% | 65% |
I think what exist some point when these two directions cross and it will be most better result for my dataset.
P.S. I found point of intersection these directions. For my dataset this point is 80% and require errorThreshold
is 0.05. It's mean that when error of training is 0.05 accuracy on training and test data become around 80%. And if I will continue training for less error than accuracy on training data will grow, but accuracy on test data will less.
from brain.js.
fantastic tracking of results!
from brain.js.
Related Issues (20)
- Deno Support?
- The Recurrent class will not function without a GPU HOT 1
- Unable to install Brain.js on PopOS HOT 22
- GPU powered AI demo is a 404 HOT 2
- "Forecasting" demo is a 404 HOT 2
- tree process
- Freezing when training NeuralNetworkGPU (async).
- Train with NeuralNetworkGPU more slower than CPU HOT 3
- Always the same output data on NeuralNetwork HOT 1
- Unable to install on MacOS Ventura HOT 1
- Did you try the built in distance calc?
- need to update URL for example
- MacOS node 20.10.0 build failing HOT 2
- Fatal error while installing via NPM package HOT 1
- Most example links are broken HOT 4
- Summary()
- Type 'InputType' is not assignable to type 'Partial<INumberHash>'... and many others
- stream-example.js page not found error HOT 1
- Npm i brain.js || node-gyp error
- Brain.js to detect bad words HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from brain.js.