1
$\begingroup$

In natural language processing, the cosine similarity is often used to compute the similarity between two words. It is bounded between [-1, 1]. Supposedly, 1 means complete similarity, -1 means something like antonyms, and 0 means no relationship between the words, although I am unsure whether that fully holds true in praxis. For another application, I need to convert the cosine similarity to a probability between 0 and 1. A straightforward solution would be to take the absolute value of the cosine similarity, but does this make sense? My goal is simply to assign higher scores to words that occur in similar contexts (i.e. could be swapped out and still leave the sentence plausible).

$\endgroup$

1 Answer 1

2
$\begingroup$

You can try to normalize the similarity: norm_sim = (sim + 1) / 2, where sim is the cosine.

In this case, 0 means opposite similarity, 0.5 means no relationship between words and 1 means complete similarity.

In my opinion, taking the absolute value of the cosine wouldn't have much sense because a word that has complete similarity with another one gets the same value as its antonym.

source

If you want to assign an higher score to antonyms than to unrelated words, you could try assigning weights to the cosine values < 0 that reduce more the value as it approaches 0.

For example:

def weights(x):
  if x < 0:
   return ((0.75*x)+1)/2
  else:
   return (x+1)/2

I know this may be an awful choice of weights, but i think the idea is valid

$\endgroup$
3
  • $\begingroup$ Thank you for your answer. I thought about rescaling like this, but my issue is with assigning antonyms a lower score than words with no relationship. That way, when I want to swap out words for similar words, I may disregard antonyms that still make a sentence meaningful in favour of words with no relation. It seems my issue then may be more with cosine similarity than how to normalise it... $\endgroup$ Mar 17 at 21:06
  • $\begingroup$ Hi, I edited the answer $\endgroup$
    – Iya Lee
    Mar 17 at 21:25
  • 1
    $\begingroup$ Thanks, that's a creative idea. I might try it, although choosing arbitrary weights seems a bit hacky. But if it works, so what? $\endgroup$ Mar 18 at 8:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.