1
$\begingroup$

i'm kinda new with neural machine translation. I've read some papers, authors usually limit the size of vocabulary by replace rare words by unk token. In this paper, they said that "...NMT model cannot learn the translation of rare words...". I want to understand why is hard for NMT mode to learn rare words and also the impact of word counts to NMT model. Thanks

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Rare words are not a problem only for NMT, they are a problem for MT in general. The reason is simple: in order to accurately translate a word in any particular context, the model needs to see as many examples as possible during the training stage. By definition the training data contains very few occurrences of rare words (especially hapax words which occur only once), so the model doesn't have enough information to learn their translation properly.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.