0
$\begingroup$

Regarding Tensorflow/Keras SparseCategoricalcrossEntropy.

SparseCategoricalcrossEntropy(from_logits=True) expects the logits that has not been normalized by softmax. Then does SparseCategoricalcrossEntropy applies softmax internally?

I suppose applying softmax would not be required internally as taking argmax would give the index expected.

Please help clarify.

$\endgroup$

1 Answer 1

1
$\begingroup$

Yes, Softmax function is called when logit=True

Infact, if we check the keras code [Link], the softmax output is ignored in every condition and tf.nn.sparse_softmax_cross_entropy_with_logits is called. This function calculate softmax prior to cross_entropy as explained [Here]

I suppose applying softmax would not be required internally as taking argmax would give the index expected.

Predicting the class is not the only task here. It has to calculate the appropriate loss value to provide feedback to the network
If the target is in probability values[OHE] i.e. sum=1, not an open-ended logit i.e. [-inf, inf]. So the output should be similarly normalized prior to the loss calculation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.