Since FastText sums up the vectors(order is not considered) of an OOV word's subwords, is it possible for two different OOV words to get the same vector ? If so, then can you give an example?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
TL;DR
Theoretically it is possible, but it is unlikely.
1) Uncommon subwords
word1 = 'iiii'
word2 = 'jjjj'
word1_subwords = ['<ii', 'iii', 'iii', 'ii>']
word2_subwords = ['<jj', 'jjj', 'jjj', 'jj>']
In this example, there are basically 6 subwords: ['<ii', '<jj', 'iii', 'jjj', 'ii>', 'jj>'], but these are not common subwords in general. So, there is a possibility that the embedding for all the subwords is the same (e.g. [0,0,...,0,0]), making their sum all the same.
2) Homographs
word1 = 'lie' # meaning: tell something untruthful
word2 = 'lie' # meaning: to rest on a horizontal position
In this example, there are two homograph words. These are different words but they have the same spelling. Since FastText only take syntax into account, they will have the same subword embedding sum.
answered Dec 19, 2019 at 8:44
-
$\begingroup$ If for all practical purposes, fasttext returns unique vectors, then, will incorporating subword order rather than simple summation serve any purpose? $\endgroup$ Commented Dec 19, 2019 at 14:33