1
$\begingroup$

I'm just getting started into the field of deep learning, and I completed my first model training using PyTorch.

I decided to use a pre-build model from torchvision, more specifically the mobilenet_v2 (https://pytorch.org/docs/stable/_modules/torchvision/models/mobilenet.html#mobilenet_v2) on a custom dataset for binary classification.

I manage to get 100% accuracy for both training and test sets (this particular dataset is not a difficult one, so nothing weird about that)

What I don't understand is why the test loss seems to be several orders of magnitude higher than the train loss, while the accuracy is 100% in both cases. Could someone here explain me what is happening here?

I show you the training stats below:

Epoch: 1/15, Train Loss: 0.22990, Train accuracy: 88.48%, Test Loss: 2.10275, Test accuracy: 99.22%

Epoch: 2/15, Train Loss: 0.03988, Train accuracy: 99.41%, Test Loss: 2.08563, Test accuracy: 99.22%

Epoch: 3/15, Train Loss: 0.02211, Train accuracy: 99.41%, Test Loss: 2.05521, Test accuracy: 100.00%

Epoch: 4/15, Train Loss: 0.01173, Train accuracy: 99.80%, Test Loss: 2.05332, Test accuracy: 100.00%

Epoch: 5/15, Train Loss: 0.00499, Train accuracy: 100.00%, Test Loss: 2.04989, Test accuracy: 100.00%

Epoch: 6/15, Train Loss: 0.00381, Train accuracy: 100.00%, Test Loss: 2.04952, Test accuracy: 100.00%

Epoch: 7/15, Train Loss: 0.00171, Train accuracy: 100.00%, Test Loss: 2.04999, Test accuracy: 100.00%

Epoch: 8/15, Train Loss: 0.00072, Train accuracy: 100.00%, Test Loss: 2.04971, Test accuracy: 100.00%

Epoch: 9/15, Train Loss: 0.00045, Train accuracy: 100.00%, Test Loss: 2.04938, Test accuracy: 100.00%

Epoch: 10/15, Train Loss: 0.00035, Train accuracy: 100.00%, Test Loss: 2.04932, Test accuracy: 100.00%

Epoch: 11/15, Train Loss: 0.00029, Train accuracy: 100.00%, Test Loss: 2.04920, Test accuracy: 100.00%

Epoch: 12/15, Train Loss: 0.00025, Train accuracy: 100.00%, Test Loss: 2.04922, Test accuracy: 100.00%

Epoch: 13/15, Train Loss: 0.00022, Train accuracy: 100.00%, Test Loss: 2.04906, Test accuracy: 100.00%

Epoch: 14/15, Train Loss: 0.00020, Train accuracy: 100.00%, Test Loss: 2.04914, Test accuracy: 100.00%

Epoch: 15/15, Train Loss: 0.00018, Train accuracy: 100.00%, Test Loss: 2.04905, Test accuracy: 100.00%

$\endgroup$
1
  • $\begingroup$ Yeah, that doesn't seem right. I suggest you show less of the output and more of your code (e.g., what functions you are calling to get the losses and accuracies). $\endgroup$ – cag51 Sep 15 '20 at 3:27
1
$\begingroup$

In binary classification we have either 0 or 1. You must be using softmax function at the end layer to get the output . It ranges from (0,1)
Loss: Generally speaking , it is the difference between predicted and actual value.
Suppose
CASE 1:predicted value is 0.6 real answer is 1.
CASE 2:predicted value is 0.9 real answer is 1.
Although they both have the correct answers(because they both round off to become 1) , loss in case 2 is less than case 1.

$\endgroup$
5
  • $\begingroup$ Well, in this particular case I was using a sigmoid function because the end layer had only 1 output. But it shouldn't make any different. $\endgroup$ – isevilla Sep 15 '20 at 11:04
  • $\begingroup$ I have checked the outputs of my network and indeed all of them are between 0.4 and 0.6 so this seems to be the reason. Could this mean that my network is overfitting? $\endgroup$ – isevilla Sep 15 '20 at 11:06
  • $\begingroup$ I meant sigmoid instead of softmax. sorry for the confusion. $\endgroup$ – Shiv Sep 15 '20 at 11:11
  • $\begingroup$ softmax is for multiclass classification. $\endgroup$ – Shiv Sep 15 '20 at 11:13
  • $\begingroup$ Your model is overfitting to some extent, you can try L1 or L2 regularisation or dropout just to check. $\endgroup$ – Shiv Sep 15 '20 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.