0
$\begingroup$

I do multi-label text classification using Bi-LSTM classifier, that means there are instances in the dataset of 11 classes that have more than 1 label. When I use the "sigmoid" activation function with "binary_crossentropy" loss function, I get higher accuracy than using "softmax" activation function with "categorical_crossentropy" loss function. why?

$\endgroup$

1 Answer 1

1
$\begingroup$

You are doing multi-label classification.

Softmax function forces the output probabilities to have a sum equals to 1. So you can't have a final output like [0, 1, 0, 1] (which you would like for a multi-label classification). Sigmoid does not have such constraint. Softmax is not suited for multi-label classification.

It probably is the reason of your results.

You should dig further into cross-entropy loss:
Cross-entropy for classification
Should I use a categorical cross-entropy or binary cross-entropy loss for binary predictions?

$\endgroup$
1
  • $\begingroup$ Many thanks, etiennedm $\endgroup$ Feb 15, 2022 at 20:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.