1
$\begingroup$

how non-contextual embedding (Word2Vec, Glove, FastText) handle OOV (incase if given word is not available in vocabulary)

$\endgroup$
1
  • $\begingroup$ Hi @tovijayak. If you find the answers to your question useful, please consider upvoting them. Also, please consider accepting one (with the tick mark ✓ next to it) if you consider it correct or, alternatively, please describe in a comment why you consider it incorrect or not clear enough. $\endgroup$
    – noe
    Jun 18, 2023 at 8:42

1 Answer 1

1
$\begingroup$

In the case of word2vec and GloVe, the out-of-vocabulary (OOV) problem is usually addressed by simply ignoring the OOV word. That is, act as if the word were not present in the text since the beginning. There is not much of an alternative, because they are simply tables that associate a word with a vector. If some word is not on the table, then you may try to find a similar word by other means (e.g. a synonym dictionary) and use its vector instead, but normally it is not worth it and the most cost-effective solution is to simply drop the word.

For FastText the issue is different because it works with character n-grams: a word embedding is simply the sum of its n-gram vectors plus the vector of the word itself. Therefore, if the n-grams of the OOV word are in the embedding, FastText would still be able to give you a representation for it.

$\endgroup$
6
  • $\begingroup$ that means in FastText, embedding works on character/ subword level? @noe $\endgroup$
    – tovijayak
    Jun 16, 2023 at 10:00
  • $\begingroup$ FastText embeddings receive information at both character n-gram and word levels and compute the embeddings at word level. $\endgroup$
    – noe
    Jun 16, 2023 at 10:02
  • $\begingroup$ in above for FastText: you referred 'word embedding is the sum of n-gram vectors PLUS the vector of the word itself. how do we get vector of the word (since the word mapping is not available ) @noe $\endgroup$
    – tovijayak
    Jun 16, 2023 at 10:07
  • $\begingroup$ The vector of the word itself is also part of the embeddings table, along with the n-grams. $\endgroup$
    – noe
    Jun 16, 2023 at 10:08
  • $\begingroup$ @tovijayak do you have any further doubts about my answer? $\endgroup$
    – noe
    Jun 19, 2023 at 14:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.