I am trying to build an MLP classifier model on a dataset containing 30000 samples and 23 features.
What are the standards I need to consider while selecting the number of hidden layers and number of nodes in each hidden layer?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Try logistic regression first. Your data is small enough that logistic regression should perform well if your data is linearly separable. MLPs are generalizations of generalized linear models. If logistic regression performs poorly, you know you need to add a deeper architecture in an MLP. Start out small with a sample of your training data first. $\endgroup$– JonCommented Apr 21, 2018 at 15:51
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
- First try a simple model: The input layer and the output layers dimension are defined by your data / your problem definition. Then train a model without any hidden layer.
- See how good it performs. Is it good enough? If yes, you're done. If no, continue
- Add a hidden layer of reasonable size or adjust a hidden layers size. Go to step (2).
The "reasonable" size part might be difficult. As a guidance: If you have only a single node, it is certainly too small for a 1000 class problem. It might be big enough for a 2 - 3 class problem. I would usually suggest to keep the size of the features per layer roughly constant or at most reduce it by 1/10 or triple it. But that is only gut feeling.
The reason for my preference for simple models is Occam's razor, the fact that they are often faster, easier to analyze and to manually improve.
For more information about topology learning and rules how to design neural networks, see:
Thoma, Martin. "Analysis and Optimization of Convolutional Neural Network Architectures." arXiv preprint arXiv:1707.09725 (2017).
Especially chapter 2.5 and chapter 3.
answered Apr 21, 2018 at 9:50
-
$\begingroup$ Thanks for the answer @Martin Thoma. I will refer the link and post the outcomes. $\endgroup$– deepguyCommented Apr 21, 2018 at 14:37
$\begingroup$
$\endgroup$
Because your inputs have numerous features, it is not clear whether your data-set is linearly separable using hyperplanes as a hypothesis or not due to the fact that you can not visualize your data and see the outcomes. First, try to add one simple neuron which has a linear calculation and a nonlinearity activation. The cost function of a single neuron is convex and it shows you whether your data is linearly separable or not in that feature space. If you have high error rate then you have to add neurons. For figureing out how you should add extra neurons and layers take a look at here which may help you.
answered Apr 21, 2018 at 14:52