Why do we have multiple neurons in the output layer of a neural network?

Why do we have two neurons in the output layer?

What does each neuron mean?

If our classifier is a binary classifier, will we have only one neuron in the output layer?

Here is a picture of the neural network:

If you want multiple things out of your network you need multiple output nodes. In the case of multiclass classification you want multiple outputs, one for each class. These represent the probability distribution over the different classes. For binary you can get away with only one output because the other class has 1-P as the probability. You could say the same for multiclass that you need one less, however this is commonly parameterized with a softmax which needs all values for the denominator.
The interpretation of outputs for regression tasks is like estimating a real value, this is why linear activation functions are used as the last layers' activation. For classification tasks it is like finding the probability. The probability that your network finds depends on the activation function used in the last layer. If your classes are mutually exclusive, and each input should have just one label, for OCR tasks each image of number refers to just a single number so classes are mutually exclusive, the activation should be Softmax and it represents the probability of belonging to each class. If the input may contain different classes, e.g. an image containing both cat and dog, you can use Sigmoid activation which each output represents the chance of existing of each class. You can also take a look at here.