# Why would you use word embeddings to find similar words?

One of the applications of word embeddings (such as GloVe) is finding words of similar meaning. I just had a look at some embeddings produced by glove on large datasets and I found that the nearest neighbors of a given word are often fairly irrelevant. Eg. ‘dad’ is the closest neighbor of ‘mom’, ‘dealership’ is the seventh closest neighbor of ‘car’.

In light of this, if you wanted to find words of similar semantics why would you prefer using embeddings instead of just downloading a database of synonyms from an online dictionary that is compiled by humans?

• First of all, word embeddings do not find synonims, they find words appearing in a similar context (because of which "dad" being similar to "mom" and "dealership" being similar to "car" is exactly what you want). They depend very much on the text you train them on - because for different corpora you may want to have different similarities. Last but not the least, they provide quantitative meaasures (i. e. vectors) to be associated to words, allowing you to perform more or less complicated mathematics with it. Commented Jul 14, 2018 at 18:51
• @gented, regarding your first sentence above: "The Euclidean distance (or cosine similarity) between two word vectors provides an effective method for measuring the linguistic or semantic similarity of the corresponding words. Sometimes, the nearest neighbors according to this metric reveal rare but relevant words that lie outside an average human's vocabulary.", see GloVe
– gen
Commented Jul 14, 2018 at 19:31
• I do not understand your comment: how is the sentence that you cited in disagreement with my statement? Commented Jul 14, 2018 at 20:36

## 1 Answer

It depends on how similarity is defined. If similarity is defined as human-defined semantics, then a synset (i.e., synonym set) is most appropriate. If similarity is defined as frequent co-occurrence, then word embeddings are most appropriate. Even within semantic similarity, there are many approaches beyond synsets.

One advantage of word embeddings over synsets is the ability to automatically find similarity with multi-word term vocabulary. For example, the common word analogy - Man is to king as woman is to queen.

• When you say “there are many approaches beyond synsets”... Could you please list some of these approaches?
– gen
Commented Sep 14, 2018 at 20:15
• Vector space models can outperform synsets. Commented Sep 14, 2018 at 20:26
• could you be a bit more specific? Is there a paper you are thinking of? What vector space models? On what tasks?
– gen
Commented Sep 15, 2018 at 3:18
• "From Frequency to Meaning: Vector Space Models of Semantics" is a review paper on the subject faculty.cse.tamu.edu/huangrh/Spring18/word_vectors.pdf Commented Sep 15, 2018 at 21:40