0
$\begingroup$

I want to get the similarity between strings (in python) However, I also want to add a weight based on the size of the strings. If the cosine similarity is 1, but the string is of 2 words, I want the final similarity to be lower than a cosine similarity of 1 with 10 words per string.

Until now, this is what I'm doing:

def weight_cosine_similarity(num1, num2):
    """
    num1: nb of words of string 1
    num2: nb of words of string 2
    """

    distance = (abs(num1 - num2) + 1)  # + 1 to avoid dividing by zero
    total_words = num1 + num2
    sigm_total_words = 1 / (1 + math.exp(-total_words))

    return sigm_total_words / distance

Two things impact the weight: 1. if number of words in string 1 is different than string 2, the weight will be smaller 2. if the total number of words is high, the weight will be higher (using sigmoid to get a number between 0 and 1)

The cosine similarity do the rest of the job.

My question: Do you know any better ways to better answer my problem, which is, to find a good weight when comparing two strings.

Thanks for your help.

$\endgroup$
  • $\begingroup$ and what is the question? $\endgroup$ – Peter Nov 16 at 8:30
  • $\begingroup$ I added it in case it was not clear $\endgroup$ – nolw38 Nov 16 at 10:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.