I want to get the similarity between strings (in python) However, I also want to add a weight based on the size of the strings. If the cosine similarity is 1, but the string is of 2 words, I want the final similarity to be lower than a cosine similarity of 1 with 10 words per string.
Until now, this is what I'm doing:
def weight_cosine_similarity(num1, num2): """ num1: nb of words of string 1 num2: nb of words of string 2 """ distance = (abs(num1 - num2) + 1) # + 1 to avoid dividing by zero total_words = num1 + num2 sigm_total_words = 1 / (1 + math.exp(-total_words)) return sigm_total_words / distance
Two things impact the weight: 1. if number of words in string 1 is different than string 2, the weight will be smaller 2. if the total number of words is high, the weight will be higher (using sigmoid to get a number between 0 and 1)
The cosine similarity do the rest of the job.
My question: Do you know any better ways to better answer my problem, which is, to find a good weight when comparing two strings.
Thanks for your help.