I'd like to cite a paragraph from the book Hands On Machine Learning with Scikit Learn and TensorFlow by Aurelien Geron regarding evaluating on a final test set after hyperparameter tuning on the training set using k-fold cross validation:
"The performance will usually be slightly worse than what you measured using cross validation if you did a lot of hyperparameter tuning (because your system ends up fine-tuned to perform well on the validation data, and will likely not perform as well on unknown datasets). It is not the case in this example, but when this happens you must resist the temptation to tweak the hyperparameters to make the numbers look good on the test set; the improvements would be unlikely to generalize to new data."
-Chapter 2: End-to-End Machine Learning Project
I am confused because he said that when the test score is WORSE the cross validation score (on the training set), you should not tweak the hyperparameters to make the testing score better. But isn't that the purpose of having a final test set? What's the use of evaluating a final test set if you can't tweak your hyperparameters if the test score is worse?