3
$\begingroup$

The text and output to be extracted are similar to the following :

"Check it every two weeks" - two weeks

"Check it on day 1 and day 14" - day 1 and day 14

"day 19 and day fourteen are important" - day 19, day fourteen

"The game is in 6 wks" - 6 weeks

"Check it in 6mo" - 6 months

"days 1 and d14 are important" - day 1, day 14

"Check it on days 11 and 14" - day 11, day 14

"check it on day one and twelve" - day one, day twelve

I have tried using SUTime library to extract the necessary information, but it just works in the case of first example and fails to properly extract information from the most of the other sentences. Considering the many ways in which the same text can be written, it is not very feasible to use ReGex.

I was wondering if there are any feasible NLP/ML solutions to extract this information. If NLP/ML is the best way to solve this, I have two questions: 1. I am not sure if I can use classification models and my first preference was to use a regression model, is it right to do that? 2. In either of the cases, I don't have a lot of labeled data to train. So, it'd be helpful if anyone can let me know if there's any open source labeled data for training+.

$\endgroup$
1
  • $\begingroup$ recognising the plausible patterns you most probably will be used in your application a simple list of regexes for each pattern should easily solve the problem, via exhasustive enumeration, nothing fancy is needed $\endgroup$
    – Nikos M.
    Jan 15, 2021 at 16:21

1 Answer 1

1
$\begingroup$

What you are trying to do here is named-entity recognition. Namely, the task consists of classifying substrings into a set of named entities (i.e. person, location, etc.). From a more formal perspective, this is a sequence labeling task that classifies parts of a sequence.

This task can be approached in different ways:

  • gazetteers/string matching
  • regular expressions
  • machine learning

I would highly recommend using SpaCy. They allow you to customize and retrain a model with your own data and labels, and typically, for such a use case, the model will perform well without requiring massive amounts of data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.