Can anyone explain the following observation?

Why did the accuracies keep to be a straight line with a very smooth decrease of loss?

By the way, why is the loss lines so beautifully smooth for the first 400 epochs?

Is this because of the learning rate or other reasons?

enter image description here


1 Answer 1


The accuracy depends on a threshold, whereas the loss doesn't. ML software tends to assume a threshold of 0.5, which is not a good fit in cases where there's some class imbalance.

I believe that, until epoch 500, your model is learning (loss is going down), but the default threshold doesn't allow you to see it in terms of accuracy.

If you pick another threshold you might see different results.

Regarding the loss going from smooth to noisy, it might be that it is learning the "easy cases" first, and then decreasing very easily and smoothly, and after epoch 400-500 it starts to overfit some of "hard cases", thus the loss becoming noisier.

  • $\begingroup$ Would you explain how can I change the threshold? Is it a parameter in the model function? Shall I increase it or lower it? I mean increasing the learning rate can effectively shorten the straight line but I think that is not the right way to deal with this.Thanks $\endgroup$
    – Leo
    Jan 28, 2021 at 5:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.