nagyistge / rlntm Goto Github PK

Author: ilyasu@,  [email protected]

This is not an official Google project.

First, install 'strict' (luarocks install strict)
Second, make sure that torch lives in /usr/bin/torch (or edit exprs/run_utils_new.py to point to the right torch path)

To run the gradient checker on the new layers, type

> luarocks install totem
> torch rlntm_pkgs/test_individual_layers.lua


To run a full gradient checker (which includes exact expectations for Reinforce), type:
> torch rlntm_pkgs/RL_check_grad.lua



To train a model that reverses a list, where the input head is forced to move forward, type:
> python exprs/train_reverse.py

It takes about 10,000 parameter updates to converge, which sadly takes 3 days on my workstation
(GPU not required).

If you are not willing to wait for 3 days, please see the output of the included printout log: 
> less rlntm_runs/train_repeatCopy-1/train_reverse-1.stdout

The printout shows the evolution of the execution traces of the RL-NTM, which 
are fun to inspect.



To train a model that solves the repeat copy task, type:

> python exprs/train_repeat_copy.py

It sometimes converges in as little as 1,000 parameter updates. If you
are not willing to wait for a day, please see the output of the
included printout log: 
> less rlntm_runs/train_repeatCopy-1/train_repeatCopy-1.stdout




WARNING:  These experiments are fragile.  It is possible that they will not succeed with 
the provided random seeds if the random number generation library is different. 
We observed that between 30-50% of the random seeds produce the desired result, so you may need
to tweak it a modest number of times.