ComplexConv2D with bias vector slows down training a lot about cvnn HOT 7 OPEN

Hello @osannolik, and thank you for the bug report.

This is an interesting insight. I was not expecting this to happen.
Here is how the bias is added. Indeed it seems like too much code for something so simple.
I also could optimize conv2D using gauss trick to do 3 convolutions instead of 4.

Due to the holidays, I won't sit with this right now. I might fix it in January, If you want, I invite you to also do a pull request if you need it ASAP.

from cvnn.

It would be nice to have it fixed as it is a pretty good optimization that would benefit more or less all users of ComplexConv2D. Not sure if I have the time to fix it myself in the near future unfortunately.

from cvnn.

Any updates on this? I recently started using ComplexConv2D and saw my training time slow down by a factor of 3~4.

I haven't tried enabling or disabling the bias specifically, but I can give that a try to see if that is the cause.

from cvnn.

Are you using batch norm? That is the main speed issue with my library.

from cvnn.

No, I'm not using any batch norm or layer norm at all!

from cvnn.

Unfortunately, I now work a full time engineer and no longer have time to keep developing/upgrading this library. I am sorry for those users but I don't think this feature is coming in the near future.

from cvnn.

I've been told that dividing by just the norm of the complex value would be good enough and it's quite fast to compute. I will leave it as an option in this issue.

from cvnn.