For example, i have a 4000 samples/data points and we have to categorize them into 4 classes.

while building MLP RNN multi-class text classification model, which has 4 classes.

For building model,

1.initially how many neurons should we take in input layer?

2.how many hidden layers & number of neurons in each layer should we consider initially?

3.how to take decision about activation functions (i.e) what functions to add at what layer or at neurons?

  1. how to decide the threshold for multi-class classification in this use-case?

what is the basic approach or assumption to consider the initial values?

  • $\begingroup$ may i know the reason why this question is down voted? $\endgroup$
    – tovijayak
    Commented Jun 27, 2023 at 11:06
  • $\begingroup$ a few likely reasons: multiple questions in one post, lack of evidence of research/an attempt, and that this set of questions has been asked many times $\endgroup$
    – Andy
    Commented Jun 27, 2023 at 20:01
  • 1
    $\begingroup$ agree, seems multiple questions. but, all should related one another and hope small questions. rapidly research is going on, so,, time to know any updates on this one. $\endgroup$
    – tovijayak
    Commented Jun 28, 2023 at 1:12

1 Answer 1


To answer your question:

  • First of all, there is no as such rule of thumb for selecting no of layers or no of neurons.
  • But you can follow some tips:

Selecting the number of layers and neurons in a Recurrent Neural Network (RNN) is more of an art than a science, and it often involves a lot of trial and error. However, here are some general guidelines:

  1. Number of Layers: In general, more layers can help the model learn more complex patterns, but it also increases the risk of overfitting and requires more computational resources. For many tasks, 1-3 layers of RNNs are sufficient. If you're dealing with more complex tasks, such as language translation or speech recognition, you might need more layers.

  2. Number of Neurons: The number of neurons in a layer determines the amount of information that can be stored in the network. More neurons allow the network to learn more complex patterns, but also increase the risk of overfitting and require more computational resources. A common practice is to start with a relatively small number of neurons, such as 128 or 256, and increase it if necessary.

  3. Validation Performance: The most reliable way to determine the optimal number of layers and neurons is to monitor the performance of the model on a validation set. If the performance on the validation set starts to decrease, it might be a sign that the model is overfitting and that you should reduce the complexity of the network.

  4. Early Stopping: Another common technique is to use early stopping. This means that you train the network for a large number of epochs but stop the training as soon as the performance on the validation set starts to decrease.

  5. Grid Search or Random Search: You can also use techniques like grid search or random search to systematically explore different combinations of hyperparameters, including the number of layers and neurons.

Remember, these are just guidelines and the optimal configuration can vary depending on the specific task and dataset.

Hope I answered your quesiton.

  • $\begingroup$ Thanks for the points @Harshad Patil, am looking for more insights. For example, if the input layer has 50 neurons and the output layer has 10 neurons, then a good starting point for the number of neurons in the hidden layer would be somewhere between 10 and 50. similarly, am looking for any notes for points (1,2,3,4) $\endgroup$
    – tovijayak
    Commented Jun 26, 2023 at 16:44
  • $\begingroup$ @tovijayak how did you arrive at between 10 and 50? that's not clear to me $\endgroup$
    – Andy
    Commented Jun 26, 2023 at 18:06
  • $\begingroup$ i just read in some article as this is a suggestion for initial setup. @Andy $\endgroup$
    – tovijayak
    Commented Jun 27, 2023 at 1:24
  • $\begingroup$ These are the basic steps to start with. One more example. If you have 10 columns then start with no of neurons = multiplies of 10 and it could be 20,30,50, 100 etc. But for layers, it's an art. You need to fine tune it until you get the best score. $\endgroup$ Commented Jun 27, 2023 at 3:41
  • $\begingroup$ ok @HarshadPatil, i have raised this question just to know any thumb-rules / notes (what we have in research - as of now) for designing the model. $\endgroup$
    – tovijayak
    Commented Jun 27, 2023 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.