0
$\begingroup$

I am training a spacy model from scratch by creating a dataset of my own with format spacy needs it to be in, the model is an NER model and the entity i am trying to recognize is Food items. I have created a dataset with 263 rows and after training the spacy model from scratch on this dataset, my model is performing great (I am getting around 80% accuracy) and this accuracy may not look a lot but it is better and I am being able to do my task a lot better now.

Now I want to improve my model even further by increasing train data. For increasing the train data I am thinking of using the rows I am sending as test and checking manually if spacy recognised each entity correctly or not and if all entities in a sentence are recognised correctly use this sentence in my training set.

My question is will this method improve my model in any way ? Or in general if we add data which is correctly classified by our model in training set, will the performance of model improve to some extent?

$\endgroup$
1
$\begingroup$

More gold data of any sort (especially at this small scale) usually leads to improvement because the model improves after seeing more examples in different contexts, but you especially want to add cases that the model got wrong (after correcting the annotation, of course) to see improvement.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.