I have not cross-validated my thoughts on overfitting with others (but this answer might be a try).
When we talk about data, we know there are both generalized relationship between input and output, which is what you want your model to learn, and random noises. You can find some plots and a more detailed description here on wikipedia.
In short, being under-fitted means your model have not yet fitted to the real mapping $f(x)\mapsto y$ you intended to teach it. Being over-fitted means your model fitted to much to the random noises in training set, which cannot be generalized to CV set or test set.
Now back to the story of models. There are a bunch of reasons a model can be overfitted, e.g. inappropriate training, wrong choice of model, etc. That being said, on the model side, the prerequisite is quite simple:
A more complex model, in terms of degree of freedom, fitted to a simpler mapping based on a relatively small training set, is more likely to overfit.
A simply linear regression with few features is not likely to overfit, as the degree of freedom, i.e. num of hyper-parameters, is too low.
For neural network, let's say MLP-NN, I can think of two problems with an overlarge hidden layer:
- You model is likely to overfit if you don't train it carefully, as the model is quite complicated, while the input has few features, which means the mapping might be relatively simpler.
- Though it increases the complexity and computational costs, it might not help with performance. Let's say you applied some technique like drop-out with a large dataset, and successfully trained your model without significant overfitting. However, the real mapping is usually not so complicated on the few features you selected. You might find most of your hidden neurons' behaving nearly identically, and thus can be removed. A better way to account for more complex mapping might be introducing more features.
