I'm trying to classify same kind of text labels in to one category. For example, if I have labels like qty, quantity, qty_no all of them should direct to Quantity. Since I'm new to data science, what's the best way to start this kind of thing?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
One way to do it could be with fuzzy string search. Levenshtein distance algorithm is what you may use for it.
... the Levenshtein distance is a string metric for measuring the difference between two sequences. Informally, the Levenshtein distance between two words is the minimum number of single-character edits (insertions, deletions or substitutions) required to change one word into the other.
From wikipedia
More on Levenshtein distance -
$\begingroup$
$\endgroup$
It would have been good to see some example of your trainings data in order to get a better understanding of the question you want to get answered. :)
Anyway, as far as I understood it, you got several labels which some of them refer to the exact same category. In your case called quantity.
From my current point of view that is purely related to data preparation, rename the labels (in your traings data file e.g. CSV) to what you want them to be and then you are good to go.
