Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I am building a CNN from scratch, and I am trying to change the number of convolutional layers to see what happens.
I have noticed that decreasing the number of convolutional layers increases the number of parameters.
This fact really surprised me, since I was expecting to see a decrease in the number of parameters.
Why decreasing the number of convolutional layers inside a CNN increases the number of parameters?
Generally removing the number of convolutional layers indeed decreases the number of parameters in a network, but given your situation I suspect you are using fully connected layers after the convolutional layers in your network. When removing some convolutional layers it means that the image/tensor size before being passed to your fully connected layers is larger and contains more pixels. Since each neuron in the fully connected layer is connected to all pixels in the layer before it the increase in the number of parameters caused by the larger image size is larger than the decrease in the number of parameters caused by the removal of the convolutional layers (leading to an overall increase in the number of parameters in your network).
When calculating parameters between a FC and a CONV layer: the formula is
((Conv layer height * width * channel) + 1 ) * units in FC layer
So, when you remove a CONV layer, the previous CONV layer gets connected to FC layer. This results in higher number of parameters.
For Example:
Lets suppose there are two conv layer and a FC layer connected consecutively.
64 * 64 * 3 ----> 32 * 32 * 8 ----> 64 * 1
Initial parameters: (32 * 32 * 8 * 64) + (8 * (1 + (2 * 2 * 3)))= 524288 + 104 = 524392
parameters after removing a CONV layer: 64 * 64 * 3 * 64 = 786432
