0
$\begingroup$

I'm training a cNN for binary classification. I used a batch size of 128, and the loss is decreasing and accuracy is increasing over epoches. The accuracy reached over 0.99 eventually, and the loss reached below 0.3. But after a few more epoches, the model converges to loss 0.6 and accuracy 0.5. An inspection of the model shows that it always predicts 0.5. I'm using binary cross entropy as the loss function. For each epoch, all data points are shuffled. I'm using SGD, learning rate is 0.01.

Am I hitting a local minimum with low accuracy rate but fairly good loss value? What is the suggested approach to deal with this? Also, why is it possible to have a low loss function with one single predicted class?

$\endgroup$
12
  • $\begingroup$ Is there any class imbalance? $\endgroup$
    – noe
    Mar 1 '18 at 19:02
  • $\begingroup$ @ncasas No, approximately 50% have 1 class and the other 50% have the the other class. $\endgroup$
    – xuhdev
    Mar 1 '18 at 19:03
  • $\begingroup$ Are you using binary cross entropy as loss function? $\endgroup$
    – noe
    Mar 1 '18 at 19:32
  • $\begingroup$ @ncasas Yes, I am. $\endgroup$
    – xuhdev
    Mar 1 '18 at 21:30
  • $\begingroup$ What kind of data shuffling are you using before batching? $\endgroup$
    – noe
    Mar 2 '18 at 9:09
1
$\begingroup$

From your description, it might be an issue of too high learning rate. When this happens, the weights cannot reach closer to the minimum and the loss does not go down.

Accordingly, I would suggest reducing the learning rate to 0.001 and using Adam.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.