0
$\begingroup$

I (probably) well overfitted/overtrained a model. But I was just curious as to what might cause this type of behaviour. I carried on training (Epoch 1/50 is not the first epoch of training this model). You can see the mse (loss) is v low. It slowly decreases over epochs 1-40. Then soon it explodes. What causes this type of behaviour when training models?

55706/55706 [=======] - 109s 2ms/step - loss: 0.0059 - coeff_determination: 0.9688
…
Epoch 5/50
55706/55706 [=======] - 105s 2ms/step - loss: 0.0033 - coeff_determination: 0.9828
...
Epoch 40/50
55706/55706 [=======] - 89s 2ms/step - loss: 0.0018 - coeff_determination: 0.9906
Epoch 41/50
55706/55706 [=======] - 110s 2ms/step - loss: 0.1853 - coeff_determination: 0.0299
Epoch 42/50
55706/55706 [=======] - 97s 2ms/step - loss: 0.3120 - coeff_determination: -0.6346
Epoch 43/50
55706/55706 [=======] - 99s 2ms/step - loss: 0.3120 - coeff_determination: -0.6345```
Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Please share the model you are using.... This maybe due to many reasons $\endgroup$
    – Ashwiniku918
    Apr 2, 2022 at 18:54

1 Answer 1

Reset to default
0
$\begingroup$

This behaviour might be related to the learning rate, first getting you closer to the loss' minimum, then overshoots it. Consider different learning rates, including the adaptive types.

Also, try different regularization methods.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.