I want to extract entities such as Organization(companies,brands,etc), Religion, Occupations, Nationality and hobbies. I will be working with millions of data and spacy is not very efficient. I have tried fine tuning BERT for NER, but it is missing out on a lot of important entities in my data. I have given it a training data of about 1,000,000 rows. Any suggestions or tips on the best way to approach this?
1 Answer
$\begingroup$
$\endgroup$
2
Since you mentioned that you have training dataset also, then you can try several approaches like
- You should try newer LLMs like llama3, GPT or Claude
- If they are expensive then train a smaller LM like T5, or llama3-8B model to give the NER values as structured JSON output.
Here The output label will be JSON, example
{"Organization": "xxx", "Religion" : "xxx", "Occupations" : "xxx", "Nationality" : "xxx", "hobbies": "xxx"}
Edit: I have used BERT/T5/LLMs for NER extraction. LLM > T5 >> BERT models
-
$\begingroup$ T5 and llama3 8b are more suited to text classification. Not sure how they will perform for NER. I was wondering if some non contextual models would be a good idea like KNN classification. $\endgroup$ Commented Aug 30 at 8:55
-
$\begingroup$ No, they are generalized models and can adapt these tasks easily $\endgroup$ Commented Aug 30 at 9:44