I need a way to map strings to a numeric space, where the mapping moves similar strings to the same number.
For example:
in
str1 = 'some random string one'
str2 = 'some rzndom string one'
str3 = 'some rndom string one'
str4 = 'a very different string'
we would want [str1
, str2
, str3
] all to be mapped to the same number, while str4
be mapped to a relatively distant number
We can assume all the strings are lowercase and have no punctuation.
What I've tried
So far, I've come up with using the sum of the ascii values and rounding down by some factor:
class StringMapper(TestCase):
def translate_string_to_numeric_space(self, strr: str, granularity: int) -> int:
list_of_char_vals = [ord(c) for c in strr]
sum_of_chars = sum(list_of_char_vals)
granularity_reduced_sum = self.round_down_to_nearest_multiple_of(sum_of_chars, granularity)
return granularity_reduced_sum
@staticmethod
def round_down_to_nearest_multiple_of(num: int, multiple: int) -> int:
return num - (num % multiple)
Then we get:
vals = [self.translate_string_to_numeric_space(s, 100) for s in [str1, str2, str3, str4]]
[2100, 2100, 2000, 2200]
What works in this solution:
- it allowed a single typo ('a' -> 'z' in
str1
,str2
)
What didn't work in this solution:
- it did not allow for omissions (
str3
was not grouped withstr1
andstr2
) - it did not create sufficient distance between
str4
and the others
Welcoming any ideas, thank you!!
Clarification String-to-string comparison such as jaro-winkler or levenstein is not an option, as we have a very large number of strings, and pair-wise comparison squares the number of operations