In CNN, why do we increase the number of filters in deeper Convolution layers for complex images?

I have been doing this online course Introduction to TensorFlow for AI, ML and DL. Here in one part, they were showing a CNN model for classifying human and horses. In this model, the first Conv2D layer had 16 filters, followed by two more Conv2D layers with 32 and 64 filters respectively. I am not sure how the number of filters is correlated with the deeper convolution layers.

For this you need to understand what filters actually do.

Every layer of filters is there to capture patterns. For example, the first layer of filters captures patterns like edges, corners, dots etc. Subsequent layers combine those patterns to make bigger patterns (like combining edges to make squares, circles, etc.).

Now as we move forward in the layers, the patterns get more complex; hence there are larger combinations of patterns to capture. That's why we increase the filter size in subsequent layers to capture as many combinations as possible.

• Thank you so much for the clarification. :) Jul 12 '19 at 9:36
• Good answer. Just a remark to be more exact, I would say "combinations" and not "permutations". Jul 13 '19 at 18:08
• Yes right. Thanks for pointing !! Jul 14 '19 at 5:51
• I'd like to add that in the case that OP is talking about, the filter size hasn't increased. The amount of filters has (16 -> 32 -> 64). But the size remains 3x3. Jan 17 '20 at 14:31

The higher the number of filters, the higher the number of abstractions that your Network is able to extract from image data. The reason why the number of filters is generally ascending is that at the input layer the Network receives raw pixel data. Raw data are always noisy, and this is especially true for image data.

Because of this, we let CNNs extract first some relevant information from noisy, "dirty" raw pixel data. Once the useful features have been extracted, then we make the CNN elaborate more complex abstractions on it.

That is why the number of filters usually increases as the Network gets deeper, even though it doesn't necessarily have to be like that.

• Thank you so much for the clarification. :) Jul 12 '19 at 9:36