How is loss computed for multiclass CNN with an output layer larger than the number of classes?

Question

I have built a CNN in pytorch for classifying the Fashion-MNIST dataset (10 classes). The images are 28x28.

I have constructed the final layer in my model as an output of 50. (i.e. $nn.Linear(100, 50)$). Also I am using cross entropy loss.

I am confused about how loss is calculated for these data sizes. From what I had known about backpropagation and loss function, the output of the neural net is compared with the expected result.

For example, using mean square error, the loss function is $(output - expected)^2$. So if I had a binary classifier, say the class labels are $({0,1})$ then the output of the neural network would need to be one dimension to compute the loss.

Now if I had three classes, how would you calculate loss? How many outputs would you need? Since the expected class label is still just a single digit, I don't see how loss can be calculated if the output of the neural network is more than one dimension.

For example, if the output is $[ x1, x2, x3]$ and the expected class label is $y$, I don't see how loss could be calculated since the dimensions don't agree.

So how is loss computed against a class label when the output of a neural network isn't a single digit?

Answer 1

There are many great tutorials covering how exactly cross entropy loss works. The key thing is : the classifier outputs probability for each class and not a single label. Taking your example, if there are 3 classes then the network will output something like [0.5, 0.25, 0.25] giving probabilities for each class. You can treat expected output as [1, 0, 0] if the output is, say, label 0. Now the dimensions agree and you can calculate the loss.