I do multi-label text classification using Bi-LSTM classifier, that means there are instances in the dataset of 11 classes that have more than 1 label. When I use the "sigmoid" activation function with "binary_crossentropy" loss function, I get higher accuracy than using "softmax" activation function with "categorical_crossentropy" loss function. why?
1 Answer
You are doing multi-label classification.
Softmax function forces the output probabilities to have a sum equals to 1. So you can't have a final output like [0, 1, 0, 1]
(which you would like for a multi-label classification). Sigmoid does not have such constraint. Softmax is not suited for multi-label classification.
It probably is the reason of your results.
You should dig further into cross-entropy loss:
Cross-entropy for classification
Should I use a categorical cross-entropy or binary cross-entropy loss for binary predictions?