# Difference between using BERT as a 'feature extractor' and fine tuning BERT with its layers fixed

I understand that there are two ways of leveraging BERT for some NLP classification task:

1. BERT might perform ‘feature extraction’ and its output is input further to another (classification) model
2. The other way is fine-tuning BERT on some text classification task by adding an output layer or layers to pretrained BERT and retraining the whole (with varying number of BERT layers fixed

However, if in the second case, we fix all the layers and add ALL the layers from the classification model will be added, 1st and 2nd approaches are effectively the same, am I right?

In approach 2, after you have fixed all the Bert layers and trained the last classification model, you have the option to un-fix the Bert layers and further train your model, and get (hopefully ) more optimized parameters for your current task.

In Pytorch this can be done by:

first fix layers

for param in model.parameters():


then unfix layers

for param in model.parameters():