Negative KL divergence about normalizing-flows HOT 3 CLOSED

Hi @yarinbar,

the forward KL divergence is given by $\text{KL}(p||q)=\mathbf{E}_p[\log\frac{p(x)}{q(x)}]=\mathbf{E}_p[\log p(x)]-\mathbf{E}_p[\log q(x)]$, where $p$ is the target and $q$ is the model. Since the target distribution is often unknown, as seems to be the case for your problem, the expectations are estimated with samples from the target, i.e. data. $\mathbf{E}_p[\log p(x)]$ still cannot be estimated in this case, but since it does not contain any model parameters, it is just a constant and is left out when computing the forward KL divergence. Hence, your loss is not literally the forward KL divergence, but the forward KL divergence minus an unknown constant shift, and, therefore, can become negative.

Best regards,
Vincent

from normalizing-flows.

Hello,

Do I understand correctly that minimising such loss (KL divergence minus an unknown constant shift) will still be correct, despite it being negative?

from normalizing-flows.

Hi @ArtemKar123,

Yes, since the constant does not depend on the model's parameters, so it will disappear anyway when computing the gradient with respect to the parameters for the optimizer.
Moreover, in this case you are essentially minimizing $-\mathbf{E}_p[\log q(x)]$, so minimizing the forward KL divergence corresponds to maximizing the model's likelihood of the samples from the target, which itself is a common way to train machine learning models.

Best regards,
Vincent

from normalizing-flows.