Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I have searched online, but am still not satisfied with answers like this and this.
My intuition is that fully connected layers are completely linear. That means no matter how many FC layers are used, the expressiveness is always limited to linear combinations of previous layer. But mathematically, one FC layer should already be able to learn the weights to produce the exactly the same behavior. Then why do we need more? Did I miss something here?
There is a nonlinear activation function inbetween these fully connected layers. Thus the resulting function is not simply a linear combination of the nodes in the previous layer.
