I've seen a lot of RNN/Transformer models use cross entropy loss with softmax. but isn't language modeling a multilabel classification task? what happens if we replace cross entropy loss with binary cross entropy loss? what are the pros and cons?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.