2
$\begingroup$

I am very new to NLP and although this seems like a basic question I don't know how to search for an answer online.

This is my problem: I have extracted and ranked keywords from 2 text sources:

enter image description here

A rank of 1, means this keyword is more important than a keyword with rank 5. Some keywords may not exist in one text but do in the other. In this case, where the keyword does not exist, there is no rank, therefore, Nan.

What method do I need to use to extract the similarity between the keyword ranks? I want to find out how similar the 2 texts are based on what keywords it contains and the ranks of these keywords.

I have tried cosine similarity by removing rows that contain Nan values and then treating text1Rank and text2Rank as vectors like this:

enter image description here

the 2 columns are the vectors I pass into the cosine similarity formula.

However, I do not think that this method weighs higher-ranked keywords more than lower-ranked keywords. Am I correct in thinking this?

If so, what method should I use to compare the 2 sets of ranks of keywords?

$\endgroup$
0
$\begingroup$

Cosine similarity won't work very well because it's only based on whether the rank at position $i$ is the same in vector 1 and vector 2.

For instance the vectors [3,2,4,1,5] vs [2,3,5,1,4] will have very low similarity because 4 positions out of 5 are different, even though there are only two swaps between (2,3) and (4,5).

A much better way to measure the similarity between two rankings is Spearman Rank Correlation.

Note that (if I'm not mistaken) in this case you could also directly use Pearson correlation, since the numeric values are already ranks. Spearman just ranks the values before applying Pearson correlation. So normally the two will give you the same result.

Note also that this method won't give more weight to the top of the ranking than the bottom. I'm not aware of a measure which does this, other than defining a custom weighted measure.

$\endgroup$
1
  • $\begingroup$ Thank you. I will try those methods. I will also try the weighted euclidean distance formula. $\endgroup$ – K Kreid Dec 12 '20 at 22:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.