# Can we fine-tune a model on the same dataset which it is pretrained on?

So I was reading this paper (about a use case of pretraining then self-training) which got me thinking - suppose I pre-train a model on a particular dataset, then fine-tune it again on the same dataset.

Theoretically, if we pre-train it as a masked LM and fine-tune, it might lead to overfitting - but I am not sure. Maybe it won't be able to generalize well, but would still give an accuracy boost?

Does anyone know some research or other credible sources to explain why or why not this should be done?

Also, if this is indeed possible/advised do we have to take some certain extra steps in Tensorflow, Keras, or HuggingFace (basically any Deep-Learning Framework or library) for doing so?

EDIT:- A simple example, I am saying like if we have a specific supervised task, and pre-train on the same features as we would fine-tune on - would that increase accuracy by any bit?

But, generally, if you have trained on dataset X for N epochs, and then you fine-tune one more epoch using the whole of X, it is just another way of saying you trained for N+1 epochs. Nothing wrong with it, except as you note in the question, if you were starting to overfit after N epochs, you are even more over-fitted now.