1
$\begingroup$

I have a dataset with two columns. First column has some text (news article) and the second column contains names of people (not exactly English names) present in those news articles (first column). I've tried to build a custom named entity recognizer using Spacy but it isn't working. Can I use deep learning approach to identify the names in unseen news articles ? (Test data)

$\endgroup$

3 Answers 3

1
$\begingroup$

The CNN and RNN are quite popular for custom NER. I will suggest you try to implement CNN first and then RNN. Also, when I did custom NER, I found out that for my dataset stacking is giving really good results. I used Random Forest, XGBoost and Linear Regression.

$\endgroup$
0
$\begingroup$

Named Entity Recognition (NER) is about identifying the position of the NEs in a text. This means that each instance must represent a particular position in a text, and the NER will predict whether this position corresponds to a NE or not. Currently your data is not formatted in this way so it's not surprising that it doesn't work: in a vector representation of the whole text the model cannot find the kind of indication it needs to identify NEs.

$\endgroup$
2
  • $\begingroup$ Yes, but is there any deep Learning aprroach like sequential modeling ? $\endgroup$ Aug 23, 2019 at 11:43
  • $\begingroup$ I don't know about recent DL approaches, hopefully somebody else will give you an answer. The only approach I know with DL is hybrid: train a RNN on the sequences, then use the resulting features in a CRF model (CRF is the traditional approach for sequence labeling like NER). $\endgroup$
    – Erwan
    Aug 23, 2019 at 13:31
0
$\begingroup$

I suggest you to take a look at the performance of mainstream taggers first. Namely: spacy, nltk, stanfordnlp. Check how good they are on finding the right set of names on your Test set.

Alternatively, you must make your own tagger. Any NER tagger is fundamentally a classifier with RNN or CNN layers that process input text. The output, Dense layer has as many nodes as classes avilable. Another option is to download some pretrained model. For example, Huggingface's transformers library lets you download lots of Transformer-based architectures (like BERT), that you can then fine tune for your specific task.

However, building your own model is a lot of work. If you find that mainstream classifiers do a pretty good job, and you don't have too much time, then go with those tools. Good luck!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.