One of the applications of word embeddings (such as GloVe) is finding words of similar meaning. I just had a look at some embeddings produced by glove on large datasets and I found that the nearest neighbors of a given word are often fairly irrelevant. Eg. ‘dad’ is the closest neighbor of ‘mom’, ‘dealership’ is the seventh closest neighbor of ‘car’.

In light of this, if you wanted to find words of similar semantics why would you prefer using embeddings instead of just downloading a database of synonyms from an online dictionary that is compiled by humans?

  • 1
    $\begingroup$ First of all, word embeddings do not find synonims, they find words appearing in a similar context (because of which "dad" being similar to "mom" and "dealership" being similar to "car" is exactly what you want). They depend very much on the text you train them on - because for different corpora you may want to have different similarities. Last but not the least, they provide quantitative meaasures (i. e. vectors) to be associated to words, allowing you to perform more or less complicated mathematics with it. $\endgroup$ – gented Jul 14 '18 at 18:51
  • $\begingroup$ @gented, regarding your first sentence above: "The Euclidean distance (or cosine similarity) between two word vectors provides an effective method for measuring the linguistic or semantic similarity of the corresponding words. Sometimes, the nearest neighbors according to this metric reveal rare but relevant words that lie outside an average human's vocabulary.", see GloVe $\endgroup$ – gen Jul 14 '18 at 19:31
  • $\begingroup$ I do not understand your comment: how is the sentence that you cited in disagreement with my statement? $\endgroup$ – gented Jul 14 '18 at 20:36

It depends on how similarity is defined. If similarity is defined as human-defined semantics, then a synset (i.e., synonym set) is most appropriate. If similarity is defined as frequent co-occurrence, then word embeddings are most appropriate. Even within semantic similarity, there are many approaches beyond synsets.

One advantage of word embeddings over synsets is the ability to automatically find similarity with multi-word term vocabulary. For example, the common word analogy - Man is to king as woman is to queen.

  • $\begingroup$ When you say “there are many approaches beyond synsets”... Could you please list some of these approaches? $\endgroup$ – gen Sep 14 '18 at 20:15
  • $\begingroup$ Vector space models can outperform synsets. $\endgroup$ – Brian Spiering Sep 14 '18 at 20:26
  • $\begingroup$ could you be a bit more specific? Is there a paper you are thinking of? What vector space models? On what tasks? $\endgroup$ – gen Sep 15 '18 at 3:18
  • 2
    $\begingroup$ "From Frequency to Meaning: Vector Space Models of Semantics" is a review paper on the subject faculty.cse.tamu.edu/huangrh/Spring18/word_vectors.pdf $\endgroup$ – Brian Spiering Sep 15 '18 at 21:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.