I have a dataset with two columns. First column has some text (news article) and the second column contains names of people (not exactly English names) present in those news articles (first column). I've tried to build a custom named entity recognizer using Spacy but it isn't working. Can I use deep learning approach to identify the names in unseen news articles ? (Test data)


Named Entity Recognition (NER) is about identifying the position of the NEs in a text. This means that each instance must represent a particular position in a text, and the NER will predict whether this position corresponds to a NE or not. Currently your data is not formatted in this way so it's not surprising that it doesn't work: in a vector representation of the whole text the model cannot find the kind of indication it needs to identify NEs.

| improve this answer | |
  • $\begingroup$ Yes, but is there any deep Learning aprroach like sequential modeling ? $\endgroup$ – Amresh Giri Aug 23 '19 at 11:43
  • $\begingroup$ I don't know about recent DL approaches, hopefully somebody else will give you an answer. The only approach I know with DL is hybrid: train a RNN on the sequences, then use the resulting features in a CRF model (CRF is the traditional approach for sequence labeling like NER). $\endgroup$ – Erwan Aug 23 '19 at 13:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.