The purpose of the logscale_factor=3. in the actnorm function about glow HOT 3 CLOSED

Closing this issue, since I have realized that logscale_factor actually reduces out and does nothing in this code. Sorry for bothering you with this stupid question.

from glow.

Why are you saying that logscale_factor reduces out? Do you mean that the neural network will learn to adjust to this factor i.e. instead of outputting `logs' it will output a scaled down version of it instead?

from glow.

The logscale_factor can accelerate the update of the parameter. For example, $\theta$ is the parameter before logscale_factor, $\beta$ is the parameter after logscale_factor and the logscale_factore equals 3.

$$ \begin{align*} \theta&=3\beta\\ \frac{\partial\mathcal{l}(3\beta)}{\partial\beta}&=3\frac{\partial\mathcal{l}(3\beta)}{\partial3\beta}=3\frac{\partial\mathcal{l}(\theta)}{\partial\theta}\\ \theta'&=\theta-\alpha\frac{\partial\mathcal{l}(\theta)}{\partial\theta}\\ 3\beta'&=3\beta-9\alpha\frac{\partial\mathcal{l}(\theta)}{\partial\theta} \end{align*} $$

from glow.