What is a general approach or planning when it comes to selecting the number of hidden layers in a neural network? Are there datasets/industries/classification v. linear that would help in choosing the number of hidden layers?

I mostly deal with small, financial datasets (less than 10,000 records) and want to put more planning into the number of layers that I use. What should I be taking into account?


You generally won't know ahead of time how many layers to use for a particular project, so you'll have to try a variety (1-5 layers, etc). However, one key constraint that will provide an upper bound is that you don't want so many layers that your model starts to overfit. 10,000 records is a very small dataset, so I probably wouldn't try more than 3 hidden layers.

Some problems (e.g., object recognition in images) can have more layers depending on the design of the network. For example, using convolutions on images reduces overfitting, so you could have more layers. However, convolutions probably won't work on financial data. Convolutions work best on spatial data. Lastly, the number of layers is independent of whether the output is classification vs regression.

| improve this answer | |
  • $\begingroup$ Convolutions can work on any data where there is a sensible way of representing each data record as an image (2D array). $\endgroup$ – DrMcCleod Aug 9 '16 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.