0
$\begingroup$

I've seen a lot of RNN/Transformer models use cross entropy loss with softmax. but isn't language modeling a multilabel classification task? what happens if we replace cross entropy loss with binary cross entropy loss? what are the pros and cons?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.