I have a dataset with two columns. First column has some text (news article) and the second column contains names of people (not exactly English names) present in those news articles (first column). I've tried to build a custom named entity recognizer using Spacy but it isn't working. Can I use deep learning approach to identify the names in unseen news articles ? (Test data)
Named Entity Recognition (NER) is about identifying the position of the NEs in a text. This means that each instance must represent a particular position in a text, and the NER will predict whether this position corresponds to a NE or not. Currently your data is not formatted in this way so it's not surprising that it doesn't work: in a vector representation of the whole text the model cannot find the kind of indication it needs to identify NEs.