Reduce network size after training about nn HOT 10 OPEN

Check out https://github.com/Atcold/net-toolkit/tree/master . I agree that it would be nice to include this into the core nn repo.

from nn.

Great, thanks! This one should definitely get included into standart NN package - almost everybody who uses NN needs those more than half of the actual NN package functions anyway.

from nn.

There is an easier way.
Define model, clone out cleanModel, train model, getParameters from model and copy onto cleanModel's parameters.

from nn.

@Atcold so true! Hadn't thought about it that way. #200 would actually make that quite easy.

from nn.

Alright @nicholas-leonard, and this

clone = mlp:clone() -- this copies all the parameters and gradParameters
clone:share(mlp,"weight","gradWeight","bias","gradBias") -- this deletes and replaces them

should go into the manual (since I never used shared stuff before and it looks like it would make things easier).

from nn.

Noted.

from nn.

Great.

from nn.

I would like to revive this issue with the following suggestion: I propose each module have a special function sanitize() which would clear all temporary variables, including outputs and gradInputs. All other ways of "sanitizing" the model are just hacks which don't handle everything or clean too much; only modules itself can know what's appropriate (encapsulation).
One could even have this in write() method, i.e. serialization would do this automatically, although this might introduce side effects (when serializing to clone or if one wants to really save everything for some reason); nevertheless, in cudnn this is used.
@soumith Could it even get to the Roadmap:)?

from nn.

@mys007 Sergey will open a PR for this shortly, he wrote this feature and wrote unit tests when he was interning at Facebook.

from nn.

I call this programming by thought:).

from nn.