Comments (8)
The docs indicate constructors like Momentum(), in actual code you require Momentum(w). Should we go back to Momentum()?
from knet.jl.
I am not sure if the functional interface: (w,p)=update!(w,g,p) is worth supporting. The original motivation was scalar parameters. But you use axpy! so you don't support this anyway?
from knet.jl.
There should be a update!(w,g;lr=0.1) which defaults to Sgd. This will come in handy if w,g are iterators, and we just want to apply a simple Sgd update to all without having to construct separate optimization objects.
from knet.jl.
Since all current update! methods require a well typed prms argument, the untyped update!(w,g,p) can be used to implement the iterator method. In fact this will support iterators within iterators (embedded arrays). The only thing it won't support is w::Dict, which requires its own method. In each case we require w,g,p to have parallel structure.
from knet.jl.
Do we have the Nesterov version of Momentum or the regular? Maybe we can make that a flag...
from knet.jl.
I am working on this in the newupdate
branch.
from knet.jl.
We use the interface (w,p)=update!(w,g,p) to allow further developments. We don't want everyone change their own code when we change our inner code.
We don't have any nesterov...
from knet.jl.
The new calls have been integrated. The problem with the (w,p)=update!(w,g,p) syntax is (1) either we are modifying in-place, in which case it is not necessary, (2) or we are constructing new w,p in which case we need to match their types to the originals, i.e. dictionary, tuple, array, struct etc. I don't think it is worth the trouble.
from knet.jl.
Related Issues (20)
- dropout seeds the global RNG
- MethodError: no method matching LinearIndices HOT 2
- ERROR: LoadError: KeyError: key :nw not found
- missing docstring error for "File I/O" section
- Making a new CUDA 3 compatible release? HOT 7
- Create a DeepMap.jl package
- Knet 1.4.7: libknet8 library not found HOT 4
- LoadError: Failed to precompile Knet [1902f260-5fb4-5aff-8c31-6271790ab950] to my julia directory HOT 3
- TagBot trigger issue HOT 2
- load/save problem with CuArrays in tutorial/60.rnn.ipynb HOT 2
- Derivative of a Function That Includes @diff Macro HOT 4
- R1 Regularization HOT 9
- type DataType has no field mutable HOT 2
- Quick Start tutorial notebook is broken HOT 1
- Cannot locate artifact 'libknet8' for aarch64-linux-gnu-libgfortran5-cxx11-libstdcxx29-julia_version+1.7.1 in Docker Container on Apple Silicon HOT 9
- Fix deprecations from MLDatasets in tutorial notebooks
- Problem with conv4 on gpu HOT 4
- UndefVarError: accuracy not defined in tutorial 30.lin.ipynb HOT 1
- CuArray{Float32} in ytype raised error HOT 4
- how to use the resnet modules in example folder
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from knet.jl.