1
$\begingroup$

I'm just getting started into the field of deep learning, and I completed my first model training using PyTorch.

I decided to use a pre-build model from torchvision, more specifically the mobilenet_v2 (https://pytorch.org/docs/stable/_modules/torchvision/models/mobilenet.html#mobilenet_v2) on a custom dataset for binary classification.

I manage to get 100% accuracy for both training and test sets (this particular dataset is not a difficult one, so nothing weird about that)

What I don't understand is why the test loss seems to be several orders of magnitude higher than the train loss, while the accuracy is 100% in both cases. Could someone here explain me what is happening here?

I show you the training stats below:

Epoch: 1/15, Train Loss: 0.22990, Train accuracy: 88.48%, Test Loss: 2.10275, Test accuracy: 99.22%

Epoch: 2/15, Train Loss: 0.03988, Train accuracy: 99.41%, Test Loss: 2.08563, Test accuracy: 99.22%

Epoch: 3/15, Train Loss: 0.02211, Train accuracy: 99.41%, Test Loss: 2.05521, Test accuracy: 100.00%

Epoch: 4/15, Train Loss: 0.01173, Train accuracy: 99.80%, Test Loss: 2.05332, Test accuracy: 100.00%

Epoch: 5/15, Train Loss: 0.00499, Train accuracy: 100.00%, Test Loss: 2.04989, Test accuracy: 100.00%

Epoch: 6/15, Train Loss: 0.00381, Train accuracy: 100.00%, Test Loss: 2.04952, Test accuracy: 100.00%

Epoch: 7/15, Train Loss: 0.00171, Train accuracy: 100.00%, Test Loss: 2.04999, Test accuracy: 100.00%

Epoch: 8/15, Train Loss: 0.00072, Train accuracy: 100.00%, Test Loss: 2.04971, Test accuracy: 100.00%

Epoch: 9/15, Train Loss: 0.00045, Train accuracy: 100.00%, Test Loss: 2.04938, Test accuracy: 100.00%

Epoch: 10/15, Train Loss: 0.00035, Train accuracy: 100.00%, Test Loss: 2.04932, Test accuracy: 100.00%

Epoch: 11/15, Train Loss: 0.00029, Train accuracy: 100.00%, Test Loss: 2.04920, Test accuracy: 100.00%

Epoch: 12/15, Train Loss: 0.00025, Train accuracy: 100.00%, Test Loss: 2.04922, Test accuracy: 100.00%

Epoch: 13/15, Train Loss: 0.00022, Train accuracy: 100.00%, Test Loss: 2.04906, Test accuracy: 100.00%

Epoch: 14/15, Train Loss: 0.00020, Train accuracy: 100.00%, Test Loss: 2.04914, Test accuracy: 100.00%

Epoch: 15/15, Train Loss: 0.00018, Train accuracy: 100.00%, Test Loss: 2.04905, Test accuracy: 100.00%

Improve this question
$\endgroup$
1
  • $\begingroup$ Yeah, that doesn't seem right. I suggest you show less of the output and more of your code (e.g., what functions you are calling to get the losses and accuracies). $\endgroup$
    – cag51
    Sep 15, 2020 at 3:27

1 Answer 1

Reset to default
1
$\begingroup$

In binary classification we have either 0 or 1. You must be using softmax function at the end layer to get the output . It ranges from (0,1)
Loss: Generally speaking , it is the difference between predicted and actual value.
Suppose
CASE 1:predicted value is 0.6 real answer is 1.
CASE 2:predicted value is 0.9 real answer is 1.
Although they both have the correct answers(because they both round off to become 1) , loss in case 2 is less than case 1.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Well, in this particular case I was using a sigmoid function because the end layer had only 1 output. But it shouldn't make any different. $\endgroup$
    – Moonstone5629
    Sep 15, 2020 at 11:04
  • $\begingroup$ I have checked the outputs of my network and indeed all of them are between 0.4 and 0.6 so this seems to be the reason. Could this mean that my network is overfitting? $\endgroup$
    – Moonstone5629
    Sep 15, 2020 at 11:06
  • $\begingroup$ I meant sigmoid instead of softmax. sorry for the confusion. $\endgroup$
    – Shiv
    Sep 15, 2020 at 11:11
  • $\begingroup$ softmax is for multiclass classification. $\endgroup$
    – Shiv
    Sep 15, 2020 at 11:13
  • $\begingroup$ Your model is overfitting to some extent, you can try L1 or L2 regularisation or dropout just to check. $\endgroup$
    – Shiv
    Sep 15, 2020 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.