Comments (2)
Hi, first of all instead of overloading Layer by modifying its source code, you could also use the convenience function Layer.fromFunction
. It will return a Layer generator from your function (i.e. equivalent to overloading by modifying the source). There's a detailed guide in the docs (help Layer.fromFunction
).
As for saving the output, there are a few work-arounds:
-
Recompute the forward pass in your backward function to get it (while it seems wasteful, if this is not the bottleneck of your network then it might be ok, and easier than any alternative).
-
Save it using the
persistent
keyword (check official Matlab docs). If you can guarantee only one instance of this layer, this will work without any complication. If there's many, you'll want separate state variables for each one; this can be implemented with the same persistent variable but with some additional bookkeeping to ensure it accesses a different state for each layer (I don't recommend it though). -
Many layers with state can actually be implemented in a state-less way if you decompose them into smaller layers. For example, DropOut would need state, but in autonn it's broken up in 2: one is the mask generator (samples random masks), the other is the dropout layer that just applies the mask. So what would normally have to be stored as state, is now just a simple variable passed from one layer to another. It might take some creativity and I realize it might not apply when you're stuck with someone else's monolithic function, but if possible this is the cleanest choice in my opinion.
from autonn.
Wow, I didn't expect to get an answer this quick.
What a neat solution. I tried Option 1, and it seems to work well so far!
Thanks!
from autonn.
Related Issues (20)
- Copy weights HOT 1
- Insert Layers HOT 1
- How does W and b randomly initialized? HOT 1
- Different results of AutoNN and DagNN? HOT 1
- weighted multi loss derivative HOT 2
- Multi Loss Backpropagation Error HOT 1
- Selective Weight Vector for Loss Func HOT 2
- cann't run the cnn example and debug the code
- issues in the nutshell example HOT 1
- KL HOT 3
- An error occurred when using the power operator in GPU mode. HOT 5
- Operator overloading of the function "gradient" HOT 1
- softmaxlog implementation in autonn
- autograd/tf.gradients - is there support? HOT 3
- Does it lost the function 'vl_argparse'? HOT 2
- Does it have fully connected layer? HOT 1
- An error with optimizeGraph function HOT 2
- Way to use a network object in parallel HOT 2
- adding lstm layer
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from autonn.