How can one interpret a drastic accuracy loss after ~38 epochs? Maybe more dropout should be added to the CNN network?

Training vs Test Accuracy

(x-axis shows the number of epochs)


There could be several reasons:

  • Numeric stability issues + overfitting. Some neural activation, or some weights, continue to increase/decrease monotonically until some function (e.g. softmax) encounters a numerical stability issue. This could happen when your network is memorizing instead of properly learning.

  • Improper learning rate. A well defined learning algorithm usually gradually decrease its learning rate as training proceeds. If that didn't work properly, or you are using a too large constant learning rate, some activation or weights could be exploding.

For the above two reasons, you may want to print the min and max values for the activations and weights in each layer during your training. If something looks like exploding (i.e. reaching something like 10E+38) you need to check on that. Also monitor if any values become NaN or infinity.

  • A bug in your code for calculating accuracy or plotting. You may want to manually inspect the predicted labels and the ground truth labels to confirm whether your algorithm is performing poorly after epoch 38.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.