Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Can you build complex (in terms of complex numbers) neural networks in Keras or Tensorflow or something similar?
This would mean the inputs, weights, activation functions, and outputs would all potentially use complex numbers. I know this can be done in theory, but all the software packages I have come across seem to assume only real numbers are used for the weights and inputs.
Here they implement Complex-Valued Deep Neural Network architectures in Keras with Theano backend, and compare them with real-valued counterparts.
Neural networks learn a function mapping from the input space to the output. This can be framed as a classification or a regression problem. In both of these cases the function determines the most information rich inputs which will lead to a minimized loss. We minimize the loss with respect to the input space.
If we have a complex valued input this is no problem. Let's assume two complimentary extreme cases. If we have a complex image, however, the complex part is completely randomly distributed. Then this problem would be equivalent to only using the real part of the image as the input. This is due to the complex part not having any pertinent information which would lead to the classification or regression. This is equally true, if the real part of the image part is completely random but the complex part contains all the information.
For the in between situation where we do not know the information distribution in the real and imaginary space then you can input of these components into the network.
For a 64*64 complex image. This will result in $2*64^2$ input nodes each accepting a floating point value, if the first layer is a densely connected layer. This is commonly used for MRI images. So the nodes can identify which real or complex values are needed to infer meaning and provide an output.
I have not seen any implementation that directly applies neural networks on complex values. But derivatives of complex values exist. You would need nodes which have complex functions such that the gradient is applied to both the real and imaginary part. From a guess, I suppose it could be possible. But the common method is just to split the real and imaginary part and feeding them to the network.
