Comments (4)
Indeed, the Complex BatchNorm is not optimized and is not previewed to be optimized in the short term. I am sorry for the trouble caused. The reason is similar as what happens with ComplexPyTorch.
from cvnn.
I was having the same problem and came up with this simple solution. According to the authors of ComplexPyTorch performing batch nomalization in a 'naive' way i.e. separately on the real and imaginary parts does not have a significant impact when compared to the complex formulation of Trabelsi et al.
Here's a TF version of their NaiveComplexBatchNorm layer, which can be used with the keras functional API.
import tensorflow as tf
from tensorflow.keras.layers import BatchNormalization
def naive_complex_batch_normalization(inputs: tf.Tensor) -> tf.Tensor:
real, imag = tf.cast(tf.math.real(inputs), tf.float32), tf.cast(tf.math.imag(inputs), tf.float32)
real_bn, imag_bn = BatchNormalization()(real), BatchNormalization()(imag)
return tf.cast(tf.complex(real_bn, imag_bn), tf.complex64)
@NEGU93, would you be interested in a PR implementing this as a proper tf.keras.layers.Layer
class?
from cvnn.
Sure, not sure what they are based on to guarantee that, from my point of view, doing a naive implementation may have a very negative impact on the phase, which is a crucial aspect of CVNN merits Ref.
But well, using CReLU should have a similar impact, and it still works well, so... Why not?
Please, submit your PR! and thank you for the contribution!
from cvnn.
Here is an implementation of a small 1D CNN for example until that PR would be integrated into the cvnn package:
def get_model(input_len=1000, activation_func='crelu'):
inputs = layers.complex_input(shape=(input_len, 1))
conv0 = layers.ComplexConv1D(64, 7, input_shape=(input_len, 1), activation=activation_func)(inputs)
bn_r0 = keras.layers.BatchNormalization()(tf.cast(tf.math.real(conv0), tf.float32))
bn_i0 = keras.layers.BatchNormalization()(tf.cast(tf.math.imag(conv0), tf.float32))
p0 = layers.ComplexAvgPooling1D(pool_size=2)(tf.cast(tf.complex(bn_r0, bn_i0), tf.complex64))
out = layers.ComplexConv1D(32, 3, activation=activation_func)(p0)
return tf.keras.Model(inputs, out)
from cvnn.
Related Issues (20)
- Error: Inputs to a layer should be tensors. Got: <cvnn.layers.core.ComplexInput object at ...> HOT 1
- ValueError: Unknown loss function:ComplexAverageCrossEntropy HOT 3
- Complex data type error with TensorFlow Functional API HOT 2
- Model subclassing compatibility HOT 4
- load CVNN model with succes HOT 1
- Implement complex-valued constraint parameter HOT 7
- Custom Activation Functions with tensorflow 2.8.2 HOT 1
- Pytorch implementation HOT 3
- ComplexConv2D with bias vector slows down training a lot HOT 7
- "WARNING:tensorflow: You are casting an input of type complex64 to an incompatible dtype float32. This will discard the imaginary part and may not be what you intended." HOT 5
- ModuleNotFoundError: No module named 'cvnn.montecarlo' HOT 1
- Unknown activation function 'cart_relu': Please ensure this object is passed to 'custom objects' argument HOT 5
- Cant find Complex Softmax which takes complex input and output complex output HOT 1
- Best Activation Function in Complex Domain HOT 1
- using this function layers.complex_input(shape=input_shape + (3,)) gives off dtype error HOT 2
- Problem with loading complex valued model HOT 2
- Equivalent Data PreProcessing for complex-valued input
- Data Parallel Distributed support HOT 4
- Best way to convert Real data into complex data type HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cvnn.