1
$\begingroup$

I used the XLNET for a sentiment classifier in determining whether a comment is positive or negative. I was able to get good results

enter image description here

But when I plotted the validation and training losses I saw this

enter image description here

I think this means that the model is overfitting? But I am not exactly sure. If there is any suggestions I would really appreciate it.

$\endgroup$
1
$\begingroup$

This kind of overfitting is typical when finetuning large LMs.

The usual approaches to "avoid" it are:

  • Early stopping: select the checkpoint with the best validation loss.
  • Random restarts: train multiple times from scratch, and select the model with the best validation performance.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.