I have a training (gold) labelled dataset than consists of 10000 sentences. The task is to create a model that classifies correctly unseen data with B-I-O tags.
I have used a BERT and a GRU RNN model.
Now, I also have millions of sentences from the same field (law) but these have no label. I have tried to produce silver labels and re train the models on the full dataset but the results were not great. How can I use these unclassified data to improve a) a BERT model and b) the GRU model? Any ideas? Many thanks.