I was working on Named Entity Identification (not recognition) task. In this NLP task, given a sentence, model has to predict whether each word (aka token) is named entity or not. The dataset used were CONLL2003 dataset.
Initially, I included a feature
first-letter-capital which was
1 if a token has its first letter capitalized. The model learnt to predict first word of each sentence as named entity.
So I removed this feature and added a feature
first-letter-capital-for-non-sentence-start-word, which was
1 if a word is not a first word of the sentence and has first letter a capital. This made model to classify first word of each sentence a non named entity.
When I kept neither, the model predicted no word as named entity. Why this might have happened? Can someone share their insight?