1
$\begingroup$

For the classification problems, what loss functions can I choose ?

For the translation problem how do I decide whether the translation is good and how to choose a loss function?

And what about the question and answer problem ?

$\endgroup$

1 Answer 1

2
$\begingroup$

For classification, you use binary cross-entropy if it's binary classification or categorical-cross entropy if it's multiclass-classification. In both cases, you compute the loss against the true class labels.

For text generation, including translation, you use categorical cross-entropy against the target sequence.

For question-answering (QA), it depends if it's extractive QA or generative QA. If it's extractive, you compute the categorical cross-entropy against the marks of start and end of the answer within the text. If it's generative, you compute the categorical cross-entropy against the expected output sequence.

$\endgroup$
4
  • $\begingroup$ So in a nutshell, it is categorical cross-entropy, right ? $\endgroup$
    – Qiulang
    Commented Apr 24 at 2:39
  • $\begingroup$ Yes, except for binary classification, where you should use binary cross-entropy. $\endgroup$
    – noe
    Commented Apr 24 at 5:33
  • $\begingroup$ @Qiulang did the answer solve the doubt from your question? If so, please consider marking it as correct (see the help center). If not, please leave a comment describing what is not clear or why you think it's not correct. $\endgroup$
    – noe
    Commented May 1 at 17:02
  • 1
    $\begingroup$ Sorry I have already upvoted but I did not accept your answer because I am new to this area so I am not sure if cross-entropy is the "only" loss function I can choose. I need to study more about it before I can accept your answer. $\endgroup$
    – Qiulang
    Commented May 5 at 10:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.