-1
$\begingroup$

I have a list of words in this format:

chem, chemistry
chemi, chemistry
chm, chemistry
chmstry, chemistry

Here, the first column represents the actual word which is in the second column. I need to apply NLP (in python3) so that when the model is trained using this dataset and I give 'chmty' as input, it will give 'chemistry' as output. I don't want string similarity techniques, I want to build an NLP model.

$\endgroup$
1
$\begingroup$

1- use LSTM with character n-gram x = chem, chemi, chmstry y = chemistry

  1. use fuzzy match with ratio to match the word with base word.

3- use regex, matach the list and get the result

$\endgroup$
1
  • $\begingroup$ I think the proposed solutions are quite inefficient. Eg fuzzy matching is irrelevant as the words and their variations are unique and distinct $\endgroup$
    – Nikos M.
    Oct 29 '21 at 16:30
1
$\begingroup$

In NLP parlance what you ask for can be framed as a case of Lemmatisation

There are NLP tools like spaCy for Python which support such tasks.

Beware that a very simple NLP algorithm for Lemmatisation which can be applied directly to your case is Dictionary Lookup (which is equivalent to a trivial rule-based system). Yes this is an NLP algorithm.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.