9
$\begingroup$

I am new to CNNs and NNs. I am reading this blog: CNN and I am confused about this part:enter image description here What confuses me is the operation that will be performed on an input vector/matrix. Will we be using a typical ANN equation: "O = W.T * input"?. And then a sigmoid on top of it?

$\endgroup$

2 Answers 2

8
$\begingroup$

Yes, essentially a typical CNN consists of two parts:

  • The convolution and pooling layers, whose goals are to extract features from the images. These are the first layers in the network.

  • The final layer(s), which are usually Fully Connected NNs, whose goal is to classify those features.

The latter do have a typical equation (i.e. $f(W^T \cdot X + b)$), where $f$ is an activation function. Usually in the context of CNNs, $f$ is a ReLU, except for the activation function of the final layer, which is selected according to the nature of the problem. The most common cases are:

  • Sigmoid activation functions work for binary classification problems.
  • Softmax activation functions work practically for both binary and multi-class classification problem.
  • For regression problems, the final layer has no activation.

One final note I'd like to make is that before entering the first FC layer, the output of the previous layer is flattened. By this I mean that the (typically 3) dimensions of that tensor are layed out into one large dimension.
For example a tensor with a shape of $(5, 5, 32)$, when flattened would become $(5 \cdot 5 \cdot 32) = (800)$.

$\endgroup$
2
$\begingroup$

Basically, yes. But in order to pass input from a convolutional, or max pooling layer to a fully-connected one, you need to "flatten" the input tensor. That is, either to flatten the tensor/multi-dimensional array from the convolutional layer or to use something like Global Average Pooling that will reduce the tensor to a vector.

You can check code snippets in different frameworks, that will help you understand the process.

Also, should be noted that fully connected layers are used not only as the last layer that outputs class probabilities in a CNN, check for example VGG networks, they have 2-3 fully connected layers at the end.

And the last remark, to get class scores you usually (not always!) use Softmax, not simple sigmoid. Softmax ensures that the sum of the values in your output vector is equal to 1.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.