You are mixing to unrelated concepts. Pretraining/finetuning and overfitting are not related.
First, let's clarify some concepts:
Overfitting: from wikipedia:
"the production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit to additional data or predict future observations reliably"
This means that overfitting is when our model learns "too well" the training data and, when faced with data at inference, it performs poorly.
Test set: it is a set of data that is not a subset of the training data but that is draw from the same distribution as the training data. It helps us assess the model's performance after training. It is important that there is no overlap between the test and the training data, to ensure that the evaluation is faithful; otherwise, when faced with real inference data has not actually been seen during training, the model performance would probably not match its performance on the test set.
Pretraining: it is when we train a model on a large training data, so that we can do fine tuning over a not-so-large data from our downstream task. The distributions of pretraining data and the training data are by definition different, although similar to some degree. Of course, the more similar they are, the more useful would be the pretrained model. Some examples of pretraining and finetuning data are in textual data:
- General domain text → Domain-specific text
- Multiple language text → Single language text
- Text in one language (e.g. Spanish) → text in a similar language (e.g. Portuguese)
Now, to answer your question: having an overlap between the pretraining data and the finetuning data is not related to overfitting, because overfitting refers to the trained model having different behaviour when applied to the training data (performs good) and the test data (performs bad). As long as the downstream task test data is not leaked to the pretraining or the finetuning training sets, there should be no problem.
Using a pretrained model is just a means to having a "starting point" in the model optimization that is assumed to be better than random initialization. The more similar the pretraining data to the finetuning data, the better such an "starting point" would be and, presumably the better the end performance on the downstream task.