Your hyperparameters are choosen on the whole set of examples, and thus there is a leakage of information of the test set into the model. The validation will therefore be too optimistic (as you suspected). What could be fixed is instead of:

clf.fit(X, y)

just use

clf.fit(x_train,y_train)

as you suggested yourself already. After that line, nothing needs to be changed. You are fitting the model on the training data, and then testing the performance on the (as of yet unseen) test data.

edit: actually, reading the documentation of GridSearchCV it is even a bit worse (depending on how your pipeline.fit method works under the hood): not only are the model hyperparameters chosen on the whole set, by using model = clf.best_estimator_, also your model is now trained using the training + test set, which is a bad thing of course, and will lead to the roc score on the test data to be much higher than on actual unseen data.

Your hyperparameters are chosen based on the whole set of examples, and thus there is a leakage of information of the test set into the model. The validation will, therefore, be too optimistic (as you suspected).

What could be fixed is instead of:

clf.fit(X, y)

just use

clf.fit(x_train, y_train)

As you suggested yourself already. After that line, nothing needs to be changed. You are fitting the model on the training data, and then testing the performance on the (as of yet unseen) test data.

edit: actually, reading the documentation of GridSearchCV it is even a bit worse (depending on how your pipeline.fit method works under the hood): not only are the model hyperparameters chosen on the whole set, by using model = clf.best_estimator_, also your model is now trained using the training + test set, which is a bad thing of course, and will lead to the roc score on the test data to be much higher than on actual unseen data.

Your hyperparameters are choosen on the whole set of examples, and thus there is a leakage of information of the test set into the model. The validation will therefore be too optimistic (as you suspected). What could be fixed is instead of:

clf.fit(X, y)

just use

clf.fit(x_train,y_train)

as you suggested yourself already. After that line, nothing needs to be changed. You are fitting the model on the training data, and then testing the performance on the (as of yet unseen) test data.

Your hyperparameters are choosen on the whole set of examples, and thus there is a leakage of information of the test set into the model. The validation will therefore be too optimistic (as you suspected). What could be fixed is instead of:

clf.fit(X, y)

just use

clf.fit(x_train,y_train)

as you suggested yourself already. After that line, nothing needs to be changed. You are fitting the model on the training data, and then testing the performance on the (as of yet unseen) test data.

edit: actually, reading the documentation of GridSearchCV it is even a bit worse (depending on how your pipeline.fit method works under the hood): not only are the model hyperparameters chosen on the whole set, by using model = clf.best_estimator_, also your model is now trained using the training + test set, which is a bad thing of course, and will lead to the roc score on the test data to be much higher than on actual unseen data.