import pandas as pd
from xgboost import XGBRegressor
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import KFold, cross_val_score, GridSearchCV
from sklearn.pipeline import Pipeline
import pickle

df = pd.read_pickle('dataset')

Y = df.iloc[:, 2:5]
X = df.iloc[:, 6:21]

for i in range(2,5):
    y = df.iloc[:, i]
    steps = list()
    steps.append(('scaler', MinMaxScaler()))
    steps.append(('grid search', GridSearchCV(estimator=XGBRegressor(),
            param_grid={"learning_rate": (0.05, 0.10, 0.15),
                        "max_depth": [ 3, 4, 5, 6, 8],
                        "min_child_weight": [ 1, 3, 5, 7],
                        "gamma":[ 0.0, 0.1, 0.2],
                        "colsample_bytree":[ 0.3, 0.4, 0.6],},
            cv=cv, scoring='neg_mean_squared_error', verbose=0, n_jobs=-1)))
    steps.append(('model', XGBRegressor()))
    pipeline = Pipeline(steps=steps)
    scores = cross_val_score(pipeline, X, y, scoring='neg_root_mean_squared_error', cv=cv)

As you may guess, I'm fairly inexperienced. What I'm trying to do is to estimate the expected prediction error, that is, $$ Err = E[Err_T] $$ (as in Hastie, Tibshirani, Friedman, Elements of Statistical Learning, pag 228, formula 7.16) by means of a Cross validation procedure, for a model trained on the whole dataset T (which is quite small and thus can not be partioned in train, validation and test sets).

The problem is that, to avoid data leakage, I need to include the Grid Search in the pipeline, but the attempt I posted above doesn't work, and I can't seem to find anyone with the same problem. How can I do this?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.