I am training a neural network which is overfitting. Even when I increase the number of parameters, the test lost plateaus while the training loss keeps decreasing. Can regularization (like an L1 or L2 loss term) help me reach a lower test loss in this case, or would I be better off just augmenting my dataset? Is there anything obvious to try that I am missing?

  • $\begingroup$ Of course, regularization can help your validation loss decrease and it is very easy to add to a network (e.g. dropout). $\endgroup$
    – noe
    Commented Apr 11 at 6:10


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.