0
$\begingroup$

I just need to check my understanding regarding the dropout regularization technique. According to my understanding, neurons can interact with each other in a way that they can fix each other's mistakes. Therefore, this can make the model learns the training data too well, and hence, overfitting can occur. Dropout is a technique in which neurons are dropped randomly per each iteration of training that involves a mini-batch. The effect of this is simply we enforce the neurons to depend on themselves and overcome this co-adaption effect. Is that right?

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes, it is correct. If you compare Neural Network to the human brain, it would be similar to new learning and prediction adaptability (otherwise it would be regarded as "psychorigid").

No dropout at all is like learning something too well, without being able to learn something else new. Indeed, a NN cannot be able to modify its weights and learn new things, because weights are already fixed and cannot correct themselves.

That's why the NN learning process needs a good balance with some error margin, just like the human brain needs some doubts when knowing something.

So the dropout is mainly useful to make the NN continue learning through many iterations. Otherwise, it would reach a threshold and new data cannot be learned.

Of course, too many dropouts would result in poor learning.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.