Bits per dimension about glow-pytorch HOT 5 CLOSED

Can anyone please explain what is bit/dimension... I've just started working with GANs, almost every 2nd paper now days comes with some form of bit/dimension metric, I tried really hard to find any literature on it but no information is out there.

Or can anyone at least provide me with any source (repo) using which I can compute this metric on my own generated data?

from glow-pytorch.

You can find it at this line https://github.com/openai/glow/blob/master/model.py#L185. Basically it is same as reddit posts. (Subtract log(n_bits) and change log base to 2) I used same formula with tensorflow implementations, but I haven't directly compared a numbers.

from glow-pytorch.

Thanks, I'm sure your loss function is comparable to that in the Tensorflow implementation. My question is about how it is related to bits per dimension.

I don't think this bits_x number is comparable to bits per dimension as in the paper. In fact, when I ran the Tensorflow code on CIFAR10, I could get bits_x as low as 2.94 after about 60 epochs (and it definitely gets lower after more training), but the reported value for bits per dimension is 3.35.

from glow-pytorch.

I don't know why but it should match. You can also refer to this. openai/glow#43

from glow-pytorch.

Never mind. I think you're right. bits_x is what they meant by bits per dimension. I was looking at training statistics, which of course are lower than test statistics.

from glow-pytorch.