I'm training a deep network (CNN-LSTM-CRF) for Named Entity Recognition. Is there a reason that increasing the number of parameters would make the network less able to overfit a small training set (~20 sentences), or does this indicate a serious bug in the code?

  • 3
    $\begingroup$ No, it means you have to train it more. $\endgroup$ Jul 9, 2018 at 1:36
  • 2
    $\begingroup$ "less able to overfit" -- explain how it is that you are measuring this, and we can help more. $\endgroup$
    – Him
    Jul 9, 2018 at 4:11

1 Answer 1


It is not necessarily a red flag. Of course, without seeing the code, it is impossible to say that for sure.

What is likely happening here is that adding parameters to your model, it makes it harder for it to converge to some minimum. More parameters roughly mean that your model is able to explain "more complex stuff". And since you have a small amount of data, the explanation should remain rather simple. Therefore, your model is trying to explain something simple in a complicated way, and it might not be easy to do so.

Also, are you using dropout or regularization? If yes, this might also be an issue as these are explicitly use to avoid overfitting.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.