0
$\begingroup$

If one is training a basic FFNN (Feed-Forward Neural Network), one would apply regularizations like dropout, l1, l2 and gaussian noise, so that the model is robust and gives better results for unseen data. But my question is, once the model gives fairly good results, isn't it advisable to remove the reguarizations then train the model again for some time, so that its predictions are more accurate?

$\endgroup$
  • $\begingroup$ please add the full name of the abbreviation FFNN when first mentioned for clarification. $\endgroup$ – Peter Jun 19 at 19:42
  • $\begingroup$ If we train the generalised model again, there are chances that it might overfit again. $\endgroup$ – Shubham Panchal Jun 20 at 2:02
0
$\begingroup$

L1 and L2 regularizations matter only during training, they are a way to update the Network's weights in (hopefully) the right way. Once you use your model for prediction that doesn't matter anymore.

Dropout is active only during training. Once it's done, the Network usese all the trained nodes to make a prediction.

In other words, no need to remove regularization techniques manually.

$\endgroup$
  • $\begingroup$ I don't think you have fully understood the question, my question is: Is it okay to retrain the model for a while after removing the regularization? $\endgroup$ – ѕняєє ѕιиgнι Jun 20 at 16:59
  • $\begingroup$ Absolutely not, that would push the model back to overfitting $\endgroup$ – Leevo Jun 20 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.