I have a data set of 700+ mil records with a feature that should yield good predictive power. The problem is that it has far more unique values than it should. The 10k+ unique values should map to about 150. I have that list of 150 values I want them to map to. Thinking about using a distance algorithm (levenshtein?) to map unique values from data to the desired set of values. What are some other ways to think about this problem?
Ex. 'Table', 'tab', 'tbl' should all map to 'table'. I'm not about to manually build a lookup table for this process given the volume of unique values. The unique values in the data are all derived from the desired values - they are acronyms or abbreviations.