1

I think you should treat this problem as a binary classification problem. For each word in the changed sentence, you will have a binary label: correct or incorrect. I would recommend relabeling so that "correct" words will have a label of 0 and "incorrect" words will have a label of 1. In your example you would have: correct_sentence = ...


1

The sentence "During pre-training, the model is trained on unlabeled data over different pre-training tasks." means that BERT was pre-trained on normal textual data on two tasks: masked language model (MLM) and next sentence prediction (NSP). There were no other classification/tagging labels present in the data, as the MLM predicts the text itself ...


1

What shepan6 is suggesting is basically to manually search for the best "transformation choice hyperparameters" by trying them all and seeing what performs best. This is a good idea (I upvoted), but if you want to go further, you can use a package like hyperopt and manually define an "objective" function that accepts a parameter that ...


Only top voted, non community-wiki answers of a minimum length are eligible