0
$\begingroup$

I (probably) well overfitted/overtrained a model. But I was just curious as to what might cause this type of behaviour. I carried on training (Epoch 1/50 is not the first epoch of training this model). You can see the mse (loss) is v low. It slowly decreases over epochs 1-40. Then soon it explodes. What causes this type of behaviour when training models?

55706/55706 [=======] - 109s 2ms/step - loss: 0.0059 - coeff_determination: 0.9688
…
Epoch 5/50
55706/55706 [=======] - 105s 2ms/step - loss: 0.0033 - coeff_determination: 0.9828
...
Epoch 40/50
55706/55706 [=======] - 89s 2ms/step - loss: 0.0018 - coeff_determination: 0.9906
Epoch 41/50
55706/55706 [=======] - 110s 2ms/step - loss: 0.1853 - coeff_determination: 0.0299
Epoch 42/50
55706/55706 [=======] - 97s 2ms/step - loss: 0.3120 - coeff_determination: -0.6346
Epoch 43/50
55706/55706 [=======] - 99s 2ms/step - loss: 0.3120 - coeff_determination: -0.6345```
$\endgroup$
1
  • 1
    $\begingroup$ Please share the model you are using.... This maybe due to many reasons $\endgroup$ Apr 2, 2022 at 18:54

1 Answer 1

0
$\begingroup$

This behaviour might be related to the learning rate, first getting you closer to the loss' minimum, then overshoots it. Consider different learning rates, including the adaptive types.

Also, try different regularization methods.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.