I am very new to NLP and although this seems like a basic question I don't know how to search for an answer online.
This is my problem: I have extracted and ranked keywords from 2 text sources:
A rank of 1, means this keyword is more important than a keyword with rank 5. Some keywords may not exist in one text but do in the other. In this case, where the keyword does not exist, there is no rank, therefore, Nan.
What method do I need to use to extract the similarity between the keyword ranks? I want to find out how similar the 2 texts are based on what keywords it contains and the ranks of these keywords.
I have tried cosine similarity by removing rows that contain Nan values and then treating text1Rank and text2Rank as vectors like this:
the 2 columns are the vectors I pass into the cosine similarity formula.
However, I do not think that this method weighs higher-ranked keywords more than lower-ranked keywords. Am I correct in thinking this?
If so, what method should I use to compare the 2 sets of ranks of keywords?