I am kind of new to NLP and text classification with Convolutional Neural Nets, and I have trained my first models quite recently. I am a little bit concerned with overfitting. I am doing multilabel classification, so my output is a list of 9 numbers (one per label) containing probabilities. I have to set a threshold to the output to get a list of 0s and 1s and assign labels to sentences. When I train the models, I draw a couple of plots to check if there is overfitting or not.
I draw the variation of the F1 score, precision and recall depending on the threshold I set on the output from the model. I see that if I try the trained model with the training data the plots show a perfect F1 score, precision and recall but it shows something much poorer with the test data. In the plots, the orange line corresponds to the training data and the blue one to the test data. Shouldn’t both curves be closer? It seems to me that the training data is perfectly fit whereas the test data is further behind. Is this a good way of checking overfitting? Or is there any other way of checking it?
