NLP to calculate similarity ratio between sentences of max 5-6 words

Im looking for a relatively simple NLP algo that would help me rate the similarity between two sentences. These sentences usually range between 1-5 words approximately.

Context:

A user can create as many categories as he wishes to group his photos. I noticed that a lot of these categories are empty and when diving a bit deeper I see that a lot of the categories created by a user have almost identical names E.g. FRANCE VS FRANC | SUMMER VS SUMER | BEACH VS BEACH ( HEART EMOTE)

One assumption is that they are creating a category with a spelling mistake and instead of deleting , they create a new one.

Goal:

Quantify the amount of highly similar category pairs at a user level.

So my question is essentially two fold:

1. which straightforward NLP algorithm could do the job pretty well without being some convuloted neural network that a company like google uses. heard of cosine similarity for vector space but unsure

2. what would be an appropriate threshold for similarity ratios? I guess thats subjective but any advice is appreciated

For spelling mistakes, Levenshtein distance is a good way to start. This metric essentially calculates the distance between two words in terms of substitutions, deletions, and insertions between them.

For instance, "bear" can be written as "fear" with one substitution (changing the "b" to an "f"). In your case, FRANC is just the word FRANCE with a deletion (of the letter "E"). You can set an appropriate threshold based on how long the sentence is. For instance, for shorter sentences you may want a smaller threshold (a Levenshtein distance of 1), whereas for your 5 word sentences, you may be willing to accommodate a threshold of 3 or 4.

This library allows you to calculate Levenshtein distance fairly easily, as a percentage of the total length of the word. For instance, in the "bear"/"fear" example I mentioned above, the words are 75% similar (3 characters stay the same out of 4 total characters). You can experiment with the percentage threshold that works best for you, though again I would recommend adjusting the threshold based on the length of the sentence.

To calculate this using cosine similarity, you would first have to get a vector of the individual sentences. This could be done in many ways, from a simple algorithm like TF-IDF or by getting embeddings from a pretrained NLP model. Once you get the vector, cosine similarity is a good measure to calculate similarity between two sentences.

Regarding finding an appropriate threshold, there is no rule of thumb but you can use heuristics and manual screening to try various thresholds, starting from high thresholds like 0.9 and gradually trying various others to see what works for your problem

• any recommendation for a pretrained NLP model? Would you suggest to just used a TF-IDF algo instead? Apr 3, 2022 at 16:58
• Maybe you can, i did not suggest because your sequence does not have any context or semnatics so document embedding may not apply.. for word embedding i will little sceptical about spelling mistakes and in my experience averaging embeddings to get sentence embedding does not yield good results Apr 3, 2022 at 17:24
• i mean there will be a lot of spelling mistakes and that my objective, trying to figure out the % of those who have created "duplicate" categories . so tf-idf is still ok ? Apr 3, 2022 at 19:42
• Tf-idf will not work for spelling mistakes...When it comes to spelling mistakes it becomes a little diff problem to solve and that too when you look at sequence not words.. Apr 3, 2022 at 19:46
• what alternatives do I have ? Like there must be something out there that can identify high similarity between say "FRANCE" and "FRANC" Apr 3, 2022 at 21:15

For calculating similarity scores between 2 short sentences "Fuzz" would work good.

1. String Similarity The simplest way to compare two strings is with a measurement of edit distance.

For example, the following two strings are quite similar: NEW YORK METS NEW YORK MEATS

m = SequenceMatcher(None, "NEW YORK METS", "NEW YORK MEATS")
m.ratio() ⇒ 0.962962962963


Which gives a very high number because of their similarity

1. Partial String Similarity

fuzz.ratio("NEW YORK METS", "NEW YORK YANKEES") ⇒ 75

Source to look further

https://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/

https://pypi.org/project/fuzzywuzzy/

https://github.com/seatgeek/thefuzz