1
$\begingroup$

I was working on Named Entity Identification (not recognition) task. In this NLP task, given a sentence, model has to predict whether each word (aka token) is named entity or not. The dataset used were CONLL2003 dataset.

Initially, I included a feature first-letter-capital which was 1 if a token has its first letter capitalized. The model learnt to predict first word of each sentence as named entity.

So I removed this feature and added a feature first-letter-capital-for-non-sentence-start-word, which was 1 if a word is not a first word of the sentence and has first letter a capital. This made model to classify first word of each sentence a non named entity.

When I kept neither, the model predicted no word as named entity. Why this might have happened? Can someone share their insight?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

If you're using a regular classification algorithm like SVM, it's not very surprising that the model fails to find any indication in the features if the features consist only of the word and this boolean feature.

This kind of task is usually done with a sequence labelling model like CRF: such models learn indications from the context of the sentences in order to detect entities. For example in the sentence "X said that ..." the model can use the next word "said" to detect "X" as an entity.

In case you want to keep using SVM, you could try to add some context features. This would probably help a bit but it wouldn't work as well as a CRF model.

Improve this answer
$\endgroup$
2
  • $\begingroup$ But then why SVM behaves the way it behaves for features I described? (Also, I know about CRF etc, but I have to utilize SVM only.) $\endgroup$
    – Rnj
    Commented Oct 14, 2021 at 20:29
  • $\begingroup$ @Rnj Because the model cannot distinguish: with only the first letter capital it cannot differentiate between first word capital vs entity capital and since there are more entities than first word it considers everything as entity. In the second case it cannot differentiate when a first word is an entity or not because it doesn't have any feature to do so. In general I would suggest at least adding POS tags to give some indications to the model, but also giving the context words and context POS tags as features. This might not be as good as CRF but it should improve performance. $\endgroup$
    – Erwan
    Commented Oct 14, 2021 at 22:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.