3
$\begingroup$

I need to design a system which can identify movie and production company names in a sentence.

The approach that comes to my mind is to train a NER Named-entity recognition system on labeled data so that it identifies the corresponding entities. But what about new entities (movie or production company name) which trained system hasn't seen, how can we tag them. Re-training the model every time with new released movies won't be feasible.

Labeled data: Sentences with the position of words that corresponds to movie or production company name

I am a beginner in NLP any help would be appreciated

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

But what about new entities (movie or production company name) that trained system hasn't seen how can we tag them. Re-training the model every time with new released movies won't be feasible.

A NER model should not have to be retrained to tag a new text it has not seen before. If trained successfully it will use information it learned from the labeled data and be able to apply it on new data. For your use case this could be information such as:

  • Capitalization - common for movie titles and production company names
  • Words used - frequently used words for either movie titles or company names
  • Where in text - movie titles perhaps appear early in texts

spaCy is a good library to get started with NER. Here is an example on how to train one using your own data.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your views, I have performed preliminary experiments with spaCy it doesn't recognise most of the movie name, need to train it on custom data $\endgroup$
    – Atinesh
    Apr 24, 2019 at 11:54
1
$\begingroup$

Since we don't have a lot of info, this is how I would have opted to work on it.

  1. Get a huge amount of text related to movies
  2. Use a movie ontology, for example this one
  3. Train a classifier to identify the movie title and the production company in the data I already have
  4. Then on another dataset, apply POS-tagging (for details here)
  5. Use the POS-tagging (maybe with extra features) and the labels from (3) to train another classifier (probably a neural network) and identify if any word is a movie or a production company
  6. Whenever you feel your model underperform, do the same from start.
Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.