BERT can be fine-tuned on a dataset for a specific task. Is it possible to fine-tune it on all these datasets for different tasks and then be utilized for these tasks instead of fine-tuning a BERT model specific to each task?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
This is possible but the BERT model will lose its purpose. Each NLP task will have its optimal loss value. If many tasks are fine-tuned on the same model, the optimal loss function for all the tasks will not be reached.
-
$\begingroup$ Is this guaranteed? Transfer learning might help generalization instead of hindering it $\endgroup$– Ggjj11Commented Jan 20 at 18:15