Comments (8)
Ok, it seems really to be a V8 bug. But I've found awful and strange solution.
1) Float64Array needs to be changed to std array in src/network.js
From:
var hardcode = "";
hardcode += "var F = Float64Array ? new Float64Array(" + optimized.memory +
") : []; ";
to
var hardcode = "";
hardcode += "var F = []; ";
2) NodeJS process must be executed with --inspect flag (so remote debugger can be attached)
node --inspect my_script.js
It's strange but when --inspect
is provided issue is not happening anymore.
Update:
When running multiple node processes (separately) then second one suddenly stops again after cca 5000 iterations. But not consuming memory any more - maybe another issue?
from synaptic.
I have the same error! Using the parameters --max_old_space_size and --max_new_space and setting them to large values doesn't change anything. Making the network smaller (only 20 neurons in the hidden layer) seems to work -- but isn't exactly what I want. Did you manage to fix this somehow?
from synaptic.
Not yet, maybe reseting network via reset method should help, but it clears all traces so it won't work well, I think.
from synaptic.
I have the same issue after around 105 iterations. It suddenly jumps from 1G to 10G RAM usage.
I have a perceptron with around 700 inputs, 32 hidden and around 160 output layers.
from synaptic.
What's the status of this issue?
from synaptic.
I've made code debugging and I've traced propagation function calls. It all seems to be OK. I found that when issue occours 'propagate' functions is not even called. When I changed Float64Array to standard array '[]' then problem occours later. So I guess that this is some bug in V8 engine. Maybe with memory management or garbage collector.
But when network optimization is turned off using 'net.setOptimize(false)' this problem is not happening. In other project I've found, that V8 has problems with large arrays and objects. Nesting helps a lot.
Additionaly my code which is using synaptic works without issues on OS X (Node 5.0, 1M iterations) but when executed on Linux server (Node 6.9, CentOS, i7, 16GB RAM) this issue happens after 5000 iterations. So it can depend on OS / hardware.
from synaptic.
Can you please check whether this is also happening with Float32Array?
from synaptic.
With Float32Array is rapidly starts to eating memory.
With standard array it is just stucked with CPU at 100% without memory leak.
from synaptic.
Related Issues (20)
- Neuron is not defined HOT 2
- Can I manually set the connection weights? HOT 1
- Poor scalability HOT 1
- capture iteration and error
- use seeded randomness HOT 6
- TypeError: this.set is not a function HOT 1
- Save and Load Training Progress HOT 1
- saving training progress in a file HOT 1
- Can't save to JSON? HOT 4
- possible error in layer connection and ALL_TO_ELSE
- Interoperability HOT 1
- clicking on example http://caza.la/synaptic/#/wikipedia plays sounds HOT 1
- Image as an input HOT 3
- Feature Request: Add Mish activation HOT 2
- Running synaptic on a website (browser) that restricts 'unsafe-eval' with CSP
- 优化器赋值失败 HOT 3
- Can I get the H and C states of the LSTM?
- Link Leads to Porn Site HOT 4
- Multi Hidden-Layer Network not learning appropriately.
- how to reward the choice of the ai?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from synaptic.