String Matching Classification and Scoring

Question

I've been playing with a problem around matching product names. I've trained a model based on a variety of different features (numbers only, Levenshtein distance, pulling out package sizes, brands, etc.). The model is spitting out predictions of match vs. no match (1 or 0). Ultimately, I understand that this is giving me what I want... it either matches or doesn't match.

What is the conventional wisdom around getting a match score? Is it simply adding up all of the feature scores and dividing, essentially an average? At the end of the day, I want to send in a product to an API and receive a list of the highest probable matches if the algorithm really can't get me to a "match".

Answer 1

If the model is a soft classifier (i.e. it predicts a probability before converting it to a class), then the simple option is to use the underlying probability (e.g. with a predict_proba function).

Another option is to directly train a regression model: in the training data, any instance with class "match" is represented as 1 and any instance with class "no match" is represented as 0. This way the model directly predicts a numeric value which can be used as a score.