In the "deep learning crash course" given by Leo Isikdogan in lecture 4 https://www.youtube.com/watch?v=ms-Ooh9mjiE&list=PLWKotBjTDoLj3rXBL-nEIPRN9V3a9Cx07&index=4 Overfitting, Underfitting, and Model Capacity , He suggests that the data should be split in train, validation and test sets. The Train set is used to train the model, the validation set to optimize the hyperparameters and the test set to give an unbiased estimation of the generalization error. When I look at how people implement the design, they usually use gridseachCV to evaluate the deep learning neural network to configure some of the hyperparameters such as the number of neurons, learning rate, optimizer, etc. After that, they used the validation set to perform the early stopping.
I think that it is possible to obtain a hyperparameter configuration from gridsearchCV that gave the best performance because it is already overfitted. Is it correct?
Is there another option to tune the hyperparameters with a validation set. Why they do not try different configurations in the validation set?