Should the different layers of deep learning models have same size or they should be changed based on a rule

Question

I see a lot of people varying the width of each layer in a deep neural network. ie. Input->50->100->150->Output. I'm curious what, if any, are the advantages of this structure over static layer widths ie. Input->50->50->50->Output.

Given the following three model structures what would be the real-world implications and differences between them and how they interpret data?

Example 1:

Input -> 100 -> 100 -> 100 -> Output

Example 2:

Input -> 50 -> 100 -> 150 -> Output

Example 3:

Input -> 150 -> 100 -> 50 -> Output

Answer 1

Actually it is not completely clear which deep neural nets you are referring to but I guess you are referring to Dense, aka fully connected, networks. It depends on your data but for simplification, based on the answer here, neurons of the first hidden layer try to find lines for separating the data, and the neurons of the subsequent layers try to consider them simultaneously, I suggest reading the hole answer. As you increase the number of neurons in the first layer, you try to increase the number of lines for separation, The last shape of the link, and as you try to increase the number of e.g. the second layer, the number of convex shapes will increase. If you increase them too much you will over-fit your data.

Increasing the number of filters, neurons or kernels, in convolutional nets has another meaning. If you increase the number of filters, it means that you try to find more features that can represent your data better. In conv layers you actually try to find features of data. Although you are somehow doing same procedure in a dense net but because the neurons are less connected and they get updated similarly in conv nets, the features learned by conv nets are completely different from dense layers. for more information I recommend you taking a look at here.

Answer 2

It's really dependent on the problem you are solving. Neural network layers attempt to get more and more abstract representations of your input, tailored towards your output. So the factors that influence your output need to be able to be represented well at each abstraction level. Each abstraction level represents more and more non-linear interactions between your inputs and hopefully less and less noise.

Let's take convolutional neural networks as an example, because it's slightly easier to interpret what is happening. The first levels are looking for small patterns while the later levels are looking for combinations of these smaller patterns that are relevant. If there are a lot of different small patterns that are relevant for your task but the later layers are only looking for a few different combinations of these filters your 3rd example would be appropriate. On the other hand, if you have some edge detectors and then some basic shapes in the first few abstraction layers, but you are trying to distinguish between 1000 classes that are all built up from these basic shapes, then you would need a more extreme variant of the second example. The first example is somewhere in between.