What you are describing is called auto-adaptive learning. This is what most recommendation systems use to adapt to ever changing data and feedback. It is also known as autoML.
Article does a good job of explaining it. Based on what your data looks ...
One Solution is "Human in the Loop" with Sentence Encoder. You can use hybrid approach using cosine similarity + Topic modelling + fuzzywuzzy + Bert. I totally understand the NLP world and the kind of problem you are asking. There is no single straight through solutions. And then use voting mechanism to filter out the best resolution.
If you are open to using huggingface transformer for fine tuning which is really popular, here is a code sample:
self.Bert = transformers.CamemBertModel.from_pretrained('camembert-base')
self.fc0 = nn.Linear(768,1)
No, approaches 1 and 2 are not the same:
In approach 1 (feature extraction), you not only take BERT's output, but normally take the internal representation of all or some of BERT's layers.
In approach 2, you train not only the classification layers but all BERT's layers also. Normally, you choose a very low learning rate and a triangular learning rate ...
In approach 2, after you have fixed all the Bert layers and trained the last classification model, you have the option to un-fix the Bert layers and further train your model, and get (hopefully ) more optimized parameters for your current task.
In Pytorch this can be done by:
first fix layers
for param in model.parameters():
param.requires_grad = False
They are meant for different purposes and they are hardly comparable.
RoBERTa is meant for text classification and tagging tasks. The idea is that you take a pretrained RoBERTa model and finetune it on your (potentially small) classification or tagging dataset. Some examples of tasks where RoBERTa is useful are sentiment classification, part-of-speech (POS) ...
The point of setting class weights is to manipulate the loss function to put more focus on the minor label. In fact, each of the data point passed to your learning algorithm will contribute information to help your loss function. By making the weight of a minor instance bigger, you say to your loss function that it should put more focus on that particular (...
I am not sure about the model you are using but I might explain what the procedure is for ML in general. You have three "vanilla" solutions for coping with unbalanced supervised dataset.
Reweighing class label so that there is the same number (calculated as sum of weights for given label) of samples per label. For example if a label with maximum ...