1) What are the appropriate activation and loss functions for multi-class classification problem?

Is it so that:

  • Up to 2 classes $\rightarrow$ Binary classification $\rightarrow$ Activation: Sigmoid $\rightarrow$ Loss: binary_crossentropy
  • From 3 classes $\rightarrow$ Multi-class classification $\rightarrow$ Activation: Softmax $\rightarrow$ Loss: categorical_crossentropy

If so then...

  • 'Dogs and cats' are binary classification but 'dogs, cats and horses' multi-class classification?

  • If we have 2 classes then we can't output probability ranges for prediction results?

2) Is it so that for multi-class classification, the last dense layer must have a number of nodes equal to the number of classes?

In the following example there are two classes (dogs and cats) and last dense layer has one node (why not 2 nodes?). If we have 3 or 10 classes then last node should have 3 or 10 nodes?

enter image description here


In my experience the activations are irrelevant to whether the classes are binary or multi-class. Their effectiveness depends on the kind of data you have.

Furthermore, there is one node in the layer because you are doing a binary classification. In other words, when the value of the last node is low or below a threshold the class is 0 and 1 if otherwise(value of last node is high or above a thresold).

And yes, if there are more classes you need as many nodes in the last Dense Layer. You can also you 2 nodes when you only have two classes in your data and predicting the one with the highest score/output.

  • $\begingroup$ Why 'Dogs and cats' are binary but 'dogs, cats and horses' multi-class classification? $\endgroup$ – Jane Mänd Dec 9 '19 at 13:28
  • $\begingroup$ binary means two and hence why cats and dogs are binary(two classes). They are still multi-class - hence you can use two nodes too but usually only one node is used because it is enough. Is it clear now? $\endgroup$ – Mohit Motwani Dec 9 '19 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.