Overfit from the loss curve

Question

I have a binary classification task. I have shown the loss curve here. I have decreased the learning rate by 1/10 every 15 epochs. There is also dropout put in the model. As you can see, I am trying to figure out the optimal point for model training. My initial assumption was that the point came at around epoch 28 since the validation error almost remains constant and then increases ever so slightly. However, I still wanted to know if this is fine or the model is indeed overfitting.

Another concern I have is that the training and validation curves are very very close to each other. Is this an expected behavior?

Being a newbie I would really appreciate any help in here

Answer 1

I'll go through your question one by one:

---

My initial assumption was that the point came at around epoch 28 since the validation error almost remains constant and then increases ever so slightly. However, I still wanted to know if this is fine or the model is indeed overfitting.

Commond knowledge from DS/ML handbooks says you're right. You should keep training until validation Loss doesn't start going up. Please keep in mind that some overfitting is inevitable (a good model is not one that eliminates overfitting, that would be impossible, but a model that is able to keep it at bay).

---

Another concern I have is that the training and validation curves are very very close to each other. Is this an expected behavior?

You should keep training even after validation loss gets higher than training loss, and stop only when validation loss starts growing. Ideally, you should have a very slight overfitting, with validation set just above training, and its slope is flat. That means: I can't train my model more than that without making things worse.

Answer 2

Yes, it looks like your model is slowly entering the overfitting area after the 28th epoch since the training loss is decreasing and the validation loss is slowly increasing.

That point, after which the validation loss increases is considered as a good point to stop. See the graph on the right in this article.

Another way to confirm that you should stop after that point is to calculate predictions with your model (from some later epoch) and check metrics (e.g. precision). Those metrics should be worse than with model from epoch which has validation loss minimum.

"Another concern I have is that the training and validation curves are very very close to each other. Is this an expected behavior?"
It might be that your training and validation data are too similar. Also, be careful that you didn't mistakenly provide same data for training and validation. But, in general, that can happen - curves can be close. It depends on the problem and data.