# NLP: robust ways to handle morphological variations in words (e.g. plurals, verb conjugations, hyphens, etc.)?

I need to process natural language sentences in which words can appear with morphological variations: car -> cars; play -> playing, played; etc. There might be hyphens also, e.g. "dog-friendly hotel", "load-bearing walls", "rock-hard cake", etc. Are there any SOTA techniques in NLP for handling morphological variations in words in a robust way?

I was thinking that maybe working at the character level could help to learn morphological patterns in words (e.g. "ing" suffix at the end of a verb means that it's a gerund, etc.), but that would require using some sort of character-level embeddings instead of the typical word-level embeddings which are commonly used. So I'm not sure if this is the best approach, and I'd rather be informed of the current state-of-the-art before proceeding.

Motivating context: I'm working on VQA (visual question answering), and I'm still trying to figure out how to best implement the NLP branch that will process the questions. I want to implement a robust model, that's why ideally I would like it to be robust to morphological variations in words, which happen all the time in natural language.

Note: although my motivating context is VQA, answers to this post need not be restricted to the VQA domain, so feel free to forget about VQA when answering if you want.

• I don't completely understand the question but the traditional method is to extract the lemma with a lemmatizer (also called stemming). Many Part-of-speech (POS) taggers can output both the POS tag and the lemma. – Erwan Jul 19 '19 at 15:15
• @Erwan by using stemming wouldn't you be losing the morphological information in words? for instance, verb conjugation (past, present, future, third-person, first-person), singular vs plural, etc.? – Pablo Messina Jul 21 '19 at 4:22
• You can keep both for every token: POS tag + lemma. – Erwan Jul 21 '19 at 11:16

from nltk.corpus import wordnet as wn