Return types for Complex Initializers about cvnn HOT 3 CLOSED

Actually, your question makes a lot of sense. It would be more logic to make cvnn.initializers.ComplexGlorotNormal() return a complex data type instead of a real one. And there is a reason I made it return a float. The reason is simply:
Tensorflow optimizer raises an exception if trainable parameters (kernels, bias, weights) are complex.

My solution was, therefore, to have two variables, for example, kernel_r and kernel_i to make the real and imaginary part and simulate the operations. Some times I can even then create an intermediate variable kernel = tf.complex(kernel_r, kernel_i) and it works (some points there for tensorflow 😤).

You can see how I init weights either here or here for example.

The question is now, why don't I use the tensorflow initializer if I am going to init as real dtype? Why use my own initializer?
The answer is xavier and he has properties of variance, that when using the TensorFlow initializer for both real and imaginary they loose those properties. In short, even though ComplexGlorotNormal() gives a float output, it is not equivalent as tf.initilizers.GlorotNormal() and the former should be use for complex layers if you want to maintain the good properties of these initializers.

from cvnn.

I think my question has been solved when I read codes related to convolutional layers. Sorry to bother you.

from cvnn.

Thanks for your reply. You’ve helped make it clearer for me.

from cvnn.