There are several scenarios that can occur while training and validating:
- Both training loss and validation loss are decreasing, with the training loss lower than the validation loss.
- Both training loss and validation loss are decreasing, with the training loss higher than the validation loss.
- Training loss decreasing, but validation loss is increasing.
I am aware that overfitting occurs in scenario 3, but does overfitting occur in scenario 1? If so, does this mean that overfitting only occurs when either scenario 1 or scenario 3 occur? Otherwise, if overfitting only occurs in scenario 3, does this mean that overfitting only occurs when validation loss is increasing?