bhneo / decorrelated_bn Goto Github PK

Hi,

I've tried to use the DecorelationNormalization layer in a recent version of Tensorflow with my CNN architecture and it gives nan values in the training. I had to make some small changes to the normalization.py file so that it works with my Tensorflow version, so I am not sure whether it is something that I changed or not (you can find my code below). Do you have any insight on that?

https://github.com/fotisdr/decorrelated_bn/blob/master/common/normalization.py

I should add that I tried both 'zca_wm' and 'iter_wm' methods and they both give nans. The former also gives the following error during training: Eigen::BDCSVD failed with error code 3

Thanks a lot!
Fotis

EDIT: If this can be of any help, I've identified the issue to the computed trace getting negative values, which gives nan when computing the tf.sqrt(trace) at the end of the get_inv_sqrt function.