0
$\begingroup$

I am kind of new to NLP and text classification with Convolutional Neural Nets, and I have trained my first models quite recently. I am a little bit concerned with overfitting. I am doing multilabel classification, so my output is a list of 9 numbers (one per label) containing probabilities. I have to set a threshold to the output to get a list of 0s and 1s and assign labels to sentences. When I train the models, I draw a couple of plots to check if there is overfitting or not.

My results from the CNN

I draw the variation of the F1 score, precision and recall depending on the threshold I set on the output from the model. I see that if I try the trained model with the training data the plots show a perfect F1 score, precision and recall but it shows something much poorer with the test data. In the plots, the orange line corresponds to the training data and the blue one to the test data. Shouldn’t both curves be closer? It seems to me that the training data is perfectly fit whereas the test data is further behind. Is this a good way of checking overfitting? Or is there any other way of checking it?

$\endgroup$
1
$\begingroup$

One way to detect overfitting is to monitor train- and validation error. If both are diverging during training an overfitting situation is given. With Keras you could set verbose=1 while training to monitor the metrics or you could plot your model history:

History = model.fit(...)
History.history #holds val and train error for all trained epochs
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.