Reason for Training and test loss sudden increment after some epochs keras

Question

We know that if training and test loss are different from each other, our model is over-fitting. However, if both get high after some epochs, how can we justify it?

One way to solve it is to reduce the learning rate to 0.0001.

But, actually I would like to know the theoretical reason behind this sudden increment.

Answer 1

This might be happening because of high learning rate.

The loss function is convex and the model needs to reach the minima. Learning rate defines the magnitude of step you take towards minima. If you take high learning rate, you might overshoot the minima, hence sudden increase in loss value.