How can one interpret a drastic accuracy loss after ~38 epochs? Maybe more dropout should be added to the CNN network?
(x-axis shows the number of epochs)
1
$\begingroup$
$\endgroup$
2
$\begingroup$
$\endgroup$
There could be several reasons:
Numeric stability issues + overfitting. Some neural activation, or some weights, continue to increase/decrease monotonically until some function (e.g.
softmax) encounters a numerical stability issue. This could happen when your network is memorizing instead of properly learning.Improper learning rate. A well defined learning algorithm usually gradually decrease its learning rate as training proceeds. If that didn't work properly, or you are using a too large constant learning rate, some activation or weights could be exploding.
For the above two reasons, you may want to print the min and max values for the activations and weights in each layer during your training. If something looks like exploding (i.e. reaching something like 10E+38) you need to check on that. Also monitor if any values become NaN or infinity.
- A bug in your code for calculating accuracy or plotting. You may want to manually inspect the predicted labels and the ground truth labels to confirm whether your algorithm is performing poorly after epoch 38.
