Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I'm working on Multi-Label problem i.e output can predict 1 or more label as an output and hence training data also have multiple labels.
Somehow I'm not able to map such ML model training. Please provide me the answer of below mentioned questions:
I found one old thread Multi Label old query, but again Im not able to understand the solution.
In a normal multi-class setup we use softmax over last layer because we know that the sum of probability of the classes is 1 as only one of these class is the actual answer. After that we pass the output to a suitable loss function like BCE (binary cross entropy).
For multi-label we know that each class can be the output so the sum of probability of these classes will not necessarily be 1! However, the individual probability of each class still need to be between 0 and 1 to be a valid probability distribution. Therefore, simply apply a sigmoid layer on top of each class output and pass this to BCE loss which treats each class as a separate mini BCE loss and finally adds these losses across all the classes to get the final BCE.
For e.g., in Pytorch you just apply sigmoid and pass it to the BCE such that the dimensions of the output passed to BCE layer is (batch size, num of total class).
Look at the BCE loss section in the pytorch docs, where the dimensions of the input to BCE loss is (N, *) which implies the dimension which I just pointed out: https://pytorch.org/docs/stable/nn.html
For multi class classification , after applying softmax over last layer we generally use categorical cross entropy as loss function rather than BCE.
