# What is the intuition behind using 2 consecutive convolutional filters in a Convolutional Neural Network?

I understand the purpose of Convolutional filters (or kernels). I visualize them as learnable feature extractors. E.g. Extract vertical edges or horizontal edges, etc.

Could somebody kindly explain to me the intuition behind stacking 2 or more consecutive convolution filters? Why couldn't the 2 filters be merged into 1?

Picture from AndrewNg's video

AndrewNg's video

I have hyperlinked Andrew Ng's lectures on Machine learning as a reference https://youtu.be/bXJx7y51cl0?t=6m15s

Could somebody kindly explain to me the intuition behind stacking 2 or more consecutive convolution filters. Why couldn't the 2 filters be merged into 1?

No, when having two consecutive convolution layers can't be combined into one. The subsequent filter's inputs are the features extracted from the previous one. This results in the second layer's features are of higher-level than the previous.

This is the basis of the whole CNN. Having multiple convolutional layers stacked along the depth of the network, allows the network to extract high-level features (not just edges and corners) from the input images.

Edit.

The first convolutional layer of a CNN is essentially a standard image filter (+ a ReLU). Its goal is to take a raw image and extract basic features from it (e.g. edges, corners). These are referred to as low-level features.

The second convolutional layer, instead of the raw image, accepts the features extracted by the first as its input. This allows it to combine these basic shapes into more complex features.

The features extracted become more and more complex as we go further down the network. Layers near the middle of the network extract the so called mid-level features, while the final layers extract high-level features.

CNNs are powerful tools because it is trained to extract the best features for each task. This results in the network extracting different features for different tasks.

For example, take the image below:

While the low-level features are the same for each task, mid and high-level features differ greatly. For instance, for the first CNN that recognizes faces:

• The low-level features are basic geometric shapes (edges, corners, circles, etc.).
• The mid-level features combine the previously extracted shapes to extract more complex features (eyes, noses, lips), specific to the task of facial recognition.
• Finally, the high-level features combine the previous to extract even more complex features (in this case different faces).
• I appreciate your taking time to answer this question. Can you go a step deeper and explain with an intuitive example This results in the second layer's features are of higher-level than the previous. – Sau001 Sep 9 '18 at 10:14
• I guess it is worth mentioning that the images are outputs of layers, not their weights. – Media Sep 9 '18 at 10:41
• Thanks again for dwelling on this further. You are getting close. You are right about the first layer performing the task of detecting edges, corners and circles. The mid-level features combine the previously extracted shapes to extract more complex features (eyes, noses, lips),..... But again the question keeps haunting me Why can't I have a single convolutional kernel/filter which will detect the shape of an eye, assuming for this argument that the shape of an eye is a circle placed symmetrically inside a flattened rhombus.? – Sau001 Sep 9 '18 at 10:55
• @Sau because there should be next layers to put lines alongside each other to find the shape of eyes. – Media Sep 9 '18 at 10:58
• A bit late on this thread - but for the benefit of others reading this thread - One can't stack them because you add non linear activation functions between in the layers. More about activation functions - ai.stackexchange.com/questions/5493/…. The basic idea is - For the CNN to learn to map data to a label, we want to model a non-linear mapping. This non-linearity is modeled using the activation functions – MonsieurBeilto Mar 27 at 3:32

Basically, you can have multiple convolutional modules in one layer. It is called grouping and was introduced in AlexNet. The inputs are the same in this case and the outputs of all convolutional modules should be concatenated after passing the input.

I quote from the link the benefit of grouping in conv nets.

Group convolutions provide elegant language for talking about lots of situations involving probability ... Group convolutions naturally extend convolutional neural networks, with everything fitting together extremely nicely. Since convolutional neural networks are one of the most powerful tools in machine learning right now, that’s pretty interesting...

• Nice answer! I wasn't aware that layer stacking was a thing... – Djib2011 Sep 9 '18 at 11:13

Formally speaking: 2 layers cannot be stacked into one, because there is a non-linearity involved between them.

In general the structure of a CNN is $image \rightarrow conv1 \rightarrow f(conv1)\rightarrow conv2 \rightarrow f(conv2)$. There is no way you can convert this into a single layer unless $f(x)$ is the linear function, in which case the CNN might not be very good.

And non-linearity is added to extract features, suppress non-useful features, etc which is not probably well understood. But still I suggest you to look at this excellent answer, if you want to better understand the use of non-linearity in CNN's:

Do scientists know what is happening inside artificial neural networks?

• Are you familiar with grouping? – Media Sep 9 '18 at 10:54
• @Media no..why? – DuttaA Sep 9 '18 at 10:54
• It stacks the conv layers in one layer :D – Media Sep 9 '18 at 10:57
• @Media how is it possible though..if there is a non-linearity involved? – DuttaA Sep 9 '18 at 10:58
• Basically their input is the same. – Media Sep 9 '18 at 11:01