I am trying to generate a table with values parsed from unstructured text. Below are a couple of examples of possibly thousands of entries. For each entry, I would like to identify the title, assign the movie to a category (e.g. "SciFi"), and so on.

"The Lord of the Rings Trilogy, Blu-Ray, Extended Edition",
"Lord of the Rings, DVD",
"Blu-Ray LOTR: The Two Towers",
"Star Wars: Episode 4"

Results should look like this:

Title                       Format    Category   Franchise
=====                       ======    ========   =========
Lord of the Rings Trilogy   DVD       Fantasy    LOTR
LOTR: The Two Towers        Blu-Ray   Fantasy    LOTR
Star Wars: Episode 4        NaN       SciFi      Star Wars

A naive approach would be to search for patterns such as "Lord of the Rings" or "Star Wars". Pattern search could be fuzzy as well. This, however, would require some kind of pre-defined mapping.

However, I wondered whether there is a smarter approach from data mining. I know text classification algorithms (e.g. spam classifiers), but they have only two classes.

I do not know much about natural language processing, and this does not really seem to be "natural language" in the stricter sense.

Can a parsing/mapping of such unstructured text be learned in any way (with sufficient training data)?

I am looking forward to get a bit of general inspiration on how to approach the problem.

  • 2
    $\begingroup$ I think your problem is similar to semi-supervised named-entity recognition systems. That should be a basis for research. $\endgroup$ – Mephy Aug 15 '18 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.