Overfitting while fine-tuning pre-trained transformer

Question

Pretrained transformers (GPT2, Bert, XLNET) are popular and useful because of their transfer learning capabilities.

Just as a reminder: The goal of Transfer learning is is to transfer knowledge gained from one domain/task and use that transfer/use that knowledge to solve some related tasks. This is done by training a model on a huge amount of labelled data (which we already have and is probably easy to get), then remove the last few layers and fine-tune the model for the new related task with task-related dataset.

I took a recent pretrained transformer published by Google, called XLNET, and just added the classification layer from the top of it and fine-tuned the whole network. (Which is the main intention of this kind of model, correct me if I am wrong).

The problem is that the model is largely overfitting. I have 1200 examples to train and each has 350 words on average.

To overcome the overfitting, I set the dropout of each layer of the transformer from 0.1 to 0.5. This did not work. So I decreased the number of trainable parameters (since the transformer has a huge number of parameters), by freezing first 10 layers (11 layers + 1 classification layer in total). Even that does not work. So I counted the number of trainable parameters in the last layer. There are 7680000 parameters which are very high compared to my dataset (around 1200*350= 420000 words). So, this high number of tunable parameters is the most possible reason for overfitting.

Here is the loss graph:

My questions are: Do you see any flaw in my analysis? Is there anything I can do to decrease overfitting? (tried with low learning rate and large batch size) If my analysis is correct, then the claim that "fine-tune pre-trained transformers with small dataset" is bit misleading and datasets should not be so small. Am I correct?

Answer 1

What makes you think your model is overfitting? Are you concerned about the difference between the training loss and validation loss?

If so, this is not overfitting. Overfitting is when the weights learned from training fail to generalize to data unseen during model training.

In the case of the plot shown here, your validation loss continues to go down, so your model continues to improve its ability to generalize to unseen data.

Once your validation loss starts creeping upward, then you have begun to overfit.

See chapter 5 of Jeremy Howard's Deep Learning for Coders with fastai and PyTorch for more details. https://www.amazon.com/Deep-Learning-Coders-fastai-PyTorch/dp/1492045527

Answer 2

Your question is valid. There are couple of known issues when trying to fit BERT-large version on small datasets (small implies a couple of 1000 training data points). The number of parameters itself is not a primary source of concern. The issues chiefly are - the use of a non-standard optimizer introduces bias in the gradient estimation; the top layers of the pre-trained BERT model provide a bad initialization point for finetuning; and the use of a pre-determined , but commonly adopted number of training iterations hurts convergence.

This causes identical learning processes with different random seeds result in significantly different models for such scenarios. This CAN be fixed!

The following paper has good suggestions to fix all of these: https://openreview.net/pdf?id=cO1IH43yUF

I quote the author's concluion here - "we show that the debiasing omission in BERTADAM is the main cause of degenerate models on small datasets commonly observed in previous work... ...Second, we observe the top layers of the pre-trained BERT provide a detrimental initialization for fine-tuning and delay learning. Simply re-initializing these layers not only speeds up learning but also leads to better model performance. Third, we demonstrate that the common one-size-fits-all three-epochs practice for BERT fine-tuning is sub-optimal and allocating more training time can stabilize fine-tuning."

Another recent SOTA paper on similar lines is: https://openreview.net/pdf?id=nzpLWnVAyah

Answer 3

Please be specific on what basis The problem is that the model is largely overfitting has been concluded. The validation loss decreased and did not show any evidence of overfitting.

How to Fine-Tune BERT for Text Classification? demonstrated the Further Pre-training as the fine-tuning method and the diagrams of the training exhibit the similar diagram for the successful fine-tunings (a), (b), and (c) except the (d) which is catastrophic forgetting.

We find that a lower learning rate, such as 2e-5, is necessary to make BERT overcome the catastrophic forgetting problem. With an aggressive learn rate of 4e-4, the training set fails to converge.

Hence your diagrams of training and validation loss would not be the basis to conclude overfitting.

Please provide the evidences that the model has too adapted to the training data hence lost the generalized capability, by inferior accuracy on the test data set, etc.