0
$\begingroup$

I'm training a cNN for binary classification. I used a batch size of 128, and the loss is decreasing and accuracy is increasing over epoches. The accuracy reached over 0.99 eventually, and the loss reached below 0.3. But after a few more epoches, the model converges to loss 0.6 and accuracy 0.5. An inspection of the model shows that it always predicts 0.5. I'm using binary cross entropy as the loss function. For each epoch, all data points are shuffled. I'm using SGD, learning rate is 0.01.

Am I hitting a local minimum with low accuracy rate but fairly good loss value? What is the suggested approach to deal with this? Also, why is it possible to have a low loss function with one single predicted class?

Improve this question
$\endgroup$
12
  • $\begingroup$ Is there any class imbalance? $\endgroup$
    – noe
    Commented Mar 1, 2018 at 19:02
  • $\begingroup$ @ncasas No, approximately 50% have 1 class and the other 50% have the the other class. $\endgroup$
    – xuhdev
    Commented Mar 1, 2018 at 19:03
  • $\begingroup$ Are you using binary cross entropy as loss function? $\endgroup$
    – noe
    Commented Mar 1, 2018 at 19:32
  • $\begingroup$ @ncasas Yes, I am. $\endgroup$
    – xuhdev
    Commented Mar 1, 2018 at 21:30
  • $\begingroup$ What kind of data shuffling are you using before batching? $\endgroup$
    – noe
    Commented Mar 2, 2018 at 9:09

1 Answer 1

Reset to default
1
$\begingroup$

From your description, it might be an issue of too high learning rate. When this happens, the weights cannot reach closer to the minimum and the loss does not go down.

Accordingly, I would suggest reducing the learning rate to 0.001 and using Adam.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.