1
$\begingroup$

I am trying to build an MLP classifier model on a dataset containing 30000 samples and 23 features. What are the standards I need to consider while selecting the number of hidden layers and number of nodes in each hidden layer?

$\endgroup$
  • $\begingroup$ Try logistic regression first. Your data is small enough that logistic regression should perform well if your data is linearly separable. MLPs are generalizations of generalized linear models. If logistic regression performs poorly, you know you need to add a deeper architecture in an MLP. Start out small with a sample of your training data first. $\endgroup$ – Jon Apr 21 '18 at 15:51
2
$\begingroup$
  1. First try a simple model: The input layer and the output layers dimension are defined by your data / your problem definition. Then train a model without any hidden layer.
  2. See how good it performs. Is it good enough? If yes, you're done. If no, continue
  3. Add a hidden layer of reasonable size or adjust a hidden layers size. Go to step (2).

The "reasonable" size part might be difficult. As a guidance: If you have only a single node, it is certainly too small for a 1000 class problem. It might be big enough for a 2 - 3 class problem. I would usually suggest to keep the size of the features per layer roughly constant or at most reduce it by 1/10 or triple it. But that is only gut feeling.

The reason for my preference for simple models is Occam's razor, the fact that they are often faster, easier to analyze and to manually improve.

For more information about topology learning and rules how to design neural networks, see:

Thoma, Martin. "Analysis and Optimization of Convolutional Neural Network Architectures." arXiv preprint arXiv:1707.09725 (2017).

Especially chapter 2.5 and chapter 3.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Thanks for the answer @Martin Thoma. I will refer the link and post the outcomes. $\endgroup$ – deepguy Apr 21 '18 at 14:37
1
$\begingroup$

Because your inputs have numerous features, it is not clear whether your data-set is linearly separable using hyperplanes as a hypothesis or not due to the fact that you can not visualize your data and see the outcomes. First, try to add one simple neuron which has a linear calculation and a nonlinearity activation. The cost function of a single neuron is convex and it shows you whether your data is linearly separable or not in that feature space. If you have high error rate then you have to add neurons. For figureing out how you should add extra neurons and layers take a look at here which may help you.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.