Fully connected layer in deep learning

Question

How to determine the best number of the fully connected layers in CNN? Can I use only one fully connected layer in CNN? How to determine the dimension of the fully connected layer output?

Answer 1

In CNNs, the convolutional layers are used to extract features in the input in order to reduce the cost function. These extracted convolutional features should be classified using dense layers. Consequently, the use of dense layers is for classifying convolutional extracted features. Based on the complexity of the high level features, extracted features in deep convolutional layers, you can have one or more layers.

Take a look at How to set the number of neurons and layers in neural networks to investigate how to set the number of neurons in dense layers.

Can I use only one fully connected layer in CNN?

Yes, you can. For instance, you can use only one layer for MNIST data set and get an acceptable learning.

How to determine the dimension of the fully connected layer output?

The dimension of each fully connected layer is equal to the number of neurons in that layer. For instance, suppose you have a weight matrix $W$ which is $10\times20$. the latter number represents the number of output neurons. Consequently, the output dimension belongs to $\mathbb{R}^{20}$.