1
$\begingroup$

What are the limitations of having too many hidden units in the Neural Network Does it take more memory or takes longer time to train the model

$\endgroup$

2 Answers 2

1
$\begingroup$

The number of hidden units is a hyperparameter which increases the model capacity. Therefore, increasing it not only increases time and memory requirements but can also lead to overfitting (also see section 11.4.1 in the Deep Learning Book).

$\endgroup$
0
$\begingroup$

As you increase your hidden layers in a neural network, the weights associated with the nodes are also increased. They increase memory requirements for your model. Also, as there are more weights, more time is needed to optimize them, hence, increase the time taken for processing.
As layers are increased our model accuracy starts to degrade due to over-fitting.

ResNet is a great example , where they skip layers in an extremely deep neural networks to achieve great performance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.