0
$\begingroup$

I have built a CNN in pytorch for classifying the Fashion-MNIST dataset (10 classes). The images are 28x28.

I have constructed the final layer in my model as an output of 50. (i.e. $nn.Linear(100, 50)$). Also I am using cross entropy loss.

I am confused about how loss is calculated for these data sizes. From what I had known about backpropagation and loss function, the output of the neural net is compared with the expected result.

For example, using mean square error, the loss function is $(output - expected)^2$. So if I had a binary classifier, say the class labels are $({0,1})$ then the output of the neural network would need to be one dimension to compute the loss.

Now if I had three classes, how would you calculate loss? How many outputs would you need? Since the expected class label is still just a single digit, I don't see how loss can be calculated if the output of the neural network is more than one dimension.

For example, if the output is $[ x1, x2, x3]$ and the expected class label is $y$, I don't see how loss could be calculated since the dimensions don't agree.

So how is loss computed against a class label when the output of a neural network isn't a single digit?

$\endgroup$
  • $\begingroup$ Why would you make final layer larger than number of classes? $\endgroup$ – Piotr Rarus - Reinstate Monica Dec 5 '19 at 8:48
  • $\begingroup$ Well, it didn't give an error. And the accuracy ended up being better surprisingly. $\endgroup$ – Darklink9110 Dec 5 '19 at 14:24
1
$\begingroup$

There are many great tutorials covering how exactly cross entropy loss works. The key thing is : the classifier outputs probability for each class and not a single label. Taking your example, if there are 3 classes then the network will output something like [0.5, 0.25, 0.25] giving probabilities for each class. You can treat expected output as [1, 0, 0] if the output is, say, label 0. Now the dimensions agree and you can calculate the loss.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.