What is the best way to figure out the semantic similarity of words? Word2Vec is okay, but not ideal:

# Using the 840B word Common Crawl GloVe vectors with gensim:

# 'hot' is closer to 'cold' than 'warm'
In [7]: model.similarity('hot', 'cold')
Out[7]: 0.59720456121072973

In [8]: model.similarity('hot', 'warm')
Out[8]: 0.56784095376659627

# Cold is much closer to 'hot' than 'popular'
In [9]: model.similarity('hot', 'popular')
Out[9]: 0.33708479049537632

NLTK's Wordnet methods appear to just give up:

In [25]: print wn.synset('hot.a.01').path_similarity(wn.synset('warm.a.01'))

What are other options?

  • 1
    $\begingroup$ Can you tell why word2vec is not ideal? Word2vec is designed to capture semantic similarity of words, then why is it not ideal? $\endgroup$ – SHASHANK GUPTA Jul 21 '16 at 10:07
  • 3
    $\begingroup$ Yes, that's why I provided the examples in the question. To reiterate: 'hot' and 'cold' are antonyms, but they're more similar (according to word2vec) than 'hot' and 'warm' which are much closer to synonyms. 'Hot' can also mean 'popular' (e.g., a "hot item") but again "hot" and "cold" are closer to each other than "hot and "popular" are. So word2vec is not ideal because it scores antonyms (which are semantic opposites) as more similar than synonyms (which are semantically equivalent). $\endgroup$ – Thomas Johnson Jul 21 '16 at 16:02
  • $\begingroup$ Use a more appropriate embedding method then: Integrating Distributional Lexical Contrast into Word Embeddings for Antonym–Synonym Distinction $\endgroup$ – Emre Aug 10 '17 at 6:42

Word2vec does not capture similarity based on antonyms and synonyms. Word2vec would give a higher similarity if the two words have the similar context. Eg The weather in California was _____ . The blank could be filled by both hot and cold hence the similarity would be higher. This concept is called Paradigmatic relations.

If you are interested to capture relations such as hypernyms, hyponyms, synonyms, antonym you would have to use any wordnet based similarity measure. There are many similarity measures based on wordnet. You may check this link http://ws4jdemo.appspot.com/


In Text Analytic Tools for Semantic Similarity, they developed a algorithm in order to find the similarity between 2 sentences. But if you read closely, they find the similarity of the word in a matrix and sum together to find out the similarity between sentences. So, it might be a shot to check word similarity.

Also in SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation, they explain the difference between association and similarity which is probably the reason for your observation as well. For example, coffee and a cup. They are not similar but they are associative. So just considering similarity would give a different result. The authors suggest various models to estimate them.

  • $\begingroup$ Broken Link, "Text Analytic Tools for Semantic Similarity". $\endgroup$ – xtian Aug 8 '16 at 19:34

Word2vec is a good starting point for most scenarios. It does capture semantics by way of prediction using CBOW method. It allows translations (as most repeated example I can put here again), V(King) - V(Queen) ~~ V(men) - V(women) and so on.

So what is the problem? The issue lies in word sense ambiguity. Whenever the word itself has two different meaning in two different context, the word vector will tend to really be away from either context. Python ~ Boa (both snakes) and Python - Java (both programming languages)..

Any alternative?

For the very specific purpose of "synonyms" if you want Wordnet would be ideal place. It captures explicit relationship of two words rather than implicit relation based on usage and occurrences.

Wordnet is mostly crafted as a dictionary - where as word2vec is mined by usage.


In a context free grammar, I think it is really kind of impossible to determine the closeness of words. What you can do is use lexicon vectors and then if a word is close in values between two lexicons then the value should be close.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.