Problem in imputation inside pipeline in Sklearn

Question

I was trying to solve the 2024 Kaggle Playground Series and I had to handle some columns with missing values. To do this tried to perform an imputation inside the pipeline. But the imputation isn't working, I have checked it multiple times but cannot figure out why it is happening. Can you please help me in this?

import pandas as pd
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OrdinalEncoder, StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split

data = pd.read_csv('/kaggle/input/playground-series-s4e8/train.csv')
data.drop([
    'stem-root',
    'veil-type',
    'veil-color',
    'spore-print-color',
], axis = 1, inplace = True) #more than 80% null
length = data.shape[0]

acceptable_cols = [col for col in data.columns
            if data[col].isnull().sum()/data.shape[0]*100 <= 1]
trainSet = data[acceptable_cols].copy().sample(frac = 0.1, random_state = 0)
X = trainSet.drop(['class', 'id'], axis = 1)
y = trainSet['class']

cat_cols = X.select_dtypes('object').columns
num_cols = list(set(X.columns) - set(cat_cols))  ###################Selection of number columns
transformations = ColumnTransformer(transformers = [
    ('oe', OrdinalEncoder(), cat_cols),
    ('impute & scale', Pipeline([
        ('impute', SimpleImputer()),   ############################# The imputer
        ('scaler', StandardScaler())
    ]), num_cols)
])

pipeline = Pipeline([
    ('transformations', transformations),
    ('gbr', GradientBoostingClassifier(verbose = True))
])

X_train, X_val, y_train, y_val = train_test_split(X, y)
pipeline.fit(X_train, y_train) ###########################the line where it shows the error
pipeline.score(X_val, y_val)

It always shows the error:

Input X contains NaN.
GradientBoostingClassifier does not accept missing values encoded as NaN natively. For supervised learning, you might want to consider sklearn.ensemble.HistGradientBoostingClassifier and Regressor which accept missing values encoded as NaNs natively. Alternatively, it is possible to preprocess the data, for instance by using an imputer transformer in a pipeline or drop samples with missing values. See https://scikit-learn.org/stable/modules/impute.html You can find a list of all estimators that handle NaN values at the following page: https://scikit-learn.org/stable/modules/impute.html#estimators-that-handle-nan-values

Thanks in advance for helping me.

Answer 1

I think that you have made it kind of overly complex. A way to solve this issue could be doing everything as separate steps in the pipeline:

ohe = ColumnTransformer([('categorical', OrdinalEncoder(), cat_cols)], remainder='passthrough')
simple_imputer = SimpleImputer()
std_scaler = ColumnTransformer([('numerical', StandardScaler(), num_cols)], remainder='passthrough')
clf = GradientBoostingClassifier(verbose=True)
pipe = Pipeline(steps = [('encoder', ohe),
                         ('imputer', simple_imputer),
                         ('scaler', std_scaler),
                         ('model', clf)])

However, if you want it to "completely" work, you would need to specify the columns for the encoder and the scaler as their indexes, instead of the names as you are doing it currently. For example,

std_scaler = ColumnTransformer([('numerical', StandardScaler(), [1, 7, 8])], remainder='passthrough')

where you are scaling the columns with the index 1, 7, and 8. I do not know the specific indexes of the columns you want to encode or scale, so you would just need to plug them in.