1
$\begingroup$

What were the traditional/earlier methods in which NLP researchers captured the word sequence information through feature engineering?

I know the current methods which rely on deep learning models like roBERT and BERT and work well with capturing sequence information. I also know about embeddings like word2vec, but they fail to capture the sequence information.

For example, I would like a feature which can differentiate between "cat ran after the dog." and "dog ran after the cat."

$\endgroup$

1 Answer 1

1
$\begingroup$

There are different methods, it depends what kind of sequential task.

  • Conditional Random Fields are typically used for sequence labeling tasks like POS tagging or NER.
  • n-gram models can be used for standard language models, but n-grams can also be used as features in various tasks where order matters. However the larger $n$ the more data is required, so it's rare to go beyond $n=5$ and this is a limitation for tasks which require long distance relations to be represented.
  • In order to reasonably capture semantics, the traditional approach is to deploy a chain of components: POS tagging, syntactic parsing (e.g. dependency parsing), then semantic role labeling. This would result in an explicit semantic representation of the sentence.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.