1
$\begingroup$

Good afternoon!

I have a dataset with thousands of shop names written in English. Several shop names might belong to one business entity, for instance, shops with names "KFC 001", "WWW.KFC.COM" and "KFC LITTLE STORE" might belong to KFC.

I want to make a clustering model to group specific shops by their names similarity into business entities as in the example described above. So I want to encode shop names someway, each shop name to some vector. Shop names might be rather long (30-40 letters), the names might contain uppercase English letters, numbers and special symbols.

My question is which pre-trained model would you recommend to generate vector embeddings for my purpose from shop names? Important features the modell shall have:

  1. The model shall someway save the info about order of the symbols in the words
  2. The model shall save the info about the symbols themselves

So what would be your advice?

$\endgroup$

1 Answer 1

0
$\begingroup$

The most common approach is to write custom preprocessing steps to standardize the names, examples include tokenizing, stemming, and lower casing.

After extensive preprocessing, the resulting tokens can be mapped to existing vector embeddings. One useful example is a model trained on Common Crawl, such as FastText.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.