3
$\begingroup$

Since FastText sums up the vectors(order is not considered) of an OOV word's subwords, is it possible for two different OOV words to get the same vector ? If so, then can you give an example?

$\endgroup$
2
$\begingroup$

TL;DR

Theoretically it is possible, but it is unlikely.

1) Uncommon subwords

word1 = 'iiii'
word2 = 'jjjj'

word1_subwords = ['<ii', 'iii', 'iii', 'ii>']
word2_subwords = ['<jj', 'jjj', 'jjj', 'jj>']

In this example, there are basically 6 subwords: ['<ii', '<jj', 'iii', 'jjj', 'ii>', 'jj>'], but these are not common subwords in general. So, there is a possibility that the embedding for all the subwords is the same (e.g. [0,0,...,0,0]), making their sum all the same.

2) Homographs

word1 = 'lie' # meaning: tell something untruthful
word2 = 'lie' # meaning: to rest on a horizontal position

In this example, there are two homograph words. These are different words but they have the same spelling. Since FastText only take syntax into account, they will have the same subword embedding sum.

$\endgroup$
  • $\begingroup$ If for all practical purposes, fasttext returns unique vectors, then, will incorporating subword order rather than simple summation serve any purpose? $\endgroup$ – Atif Hassan Dec 19 '19 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.