May the training set and validation set overlap?
Similarly, may the testing set and validation set overlap?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
They should not.
The validation set is used to know when to stop training your model. The idea is that you check your model performance often and when there seems to be no more improvement, you stop.
Take a look at the plot below. It is plotting the loss of the model. If the loss is still decreasing means that you can keep on improving your model, but if the loss stops decreasing you stop training.
Notice how the valid loss stops decreasing before the train loss. That is because the model could keep improving to improve the performance on the training set, but if you do that, you will get overfitting. So, by having an unseen validation set, you will stop training earlier and not overfit your mode, otherwise, if you keep on training, the accuracy on the validation set will start to decrease.
That means that if the two sets overlap, the validation loss will become more similar to the training loss, so you model will keep on training and you will overfit your model.
They should not.
You already used your validation set to stop training your model. That means that you already know the performance of your model again the validation set.
Now your model is trained and you want to test your model with unseen data points, i.e. with the testing set. If your sets overlap, you are biasing your test results towards the performance which you already know your model has.