KNN Imputation utilize mean or mode?

Question

In my current project, I am doing KNN imputation with K = 5 and I am using sklearn.impute.KNNImputer. I have a mix of continuous and nominal variables(encoded as 0/1 or ordinal ones that have been encoded as 0/0.25/0.5/0.75/1 etc). However, the docs say "Each sample’s missing values are imputed using the mean value from n_neighbors nearest neighbors found in the training set." Because of this, I am getting in-between values like 0.4 for nominal attributes. Is there any way to override this to change from mean to mode for nominal columns?

Also, I looked at missingpy and fancyimpute but they both seem to be using mean as well~

Answer 1

By default scikit-learn's KNNImputer uses Euclidean distance metric for searching neighbors and mean for imputing values.

If you have a combination of continuous and nominal variables, you should pass in a different distance metric.

If you want to use another imputation function than mean, you'll have to implement that yourself.

Answer 2

The best way of solve this is using pipelines as follows:

Working example:

import pandas as pd
import numpy as np

from sklearn.pipeline import Pipeline
from sklearn.compose import make_column_selector, make_column_transformer
from sklearn.linear_model import LogisticRegression
from sklearn.impute import SimpleImputer, KNNImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import plot_roc_curve
from sklearn.datasets import fetch_openml

# Load the data
X, y = fetch_openml("titanic", version=1, as_frame=True, return_X_y=True)
X.replace({None:np.nan}, inplace = True)
# Some preprocessing to correct data types and replace None with nans for pipeline imputer
X.drop(["name","home.dest"], axis = 1, inplace = True)
X["embarked"] = X["embarked"].astype("object")
X["sex"] = X["sex"].astype("object")

X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, test_size=0.2)

cat_prepro = Pipeline([("imputer",SimpleImputer(strategy="most_frequent")),
                        ("encoder",OneHotEncoder(handle_unknown = "ignore"))])

cont_prepro = Pipeline([("imputer", KNNImputer()), ("scaler",StandardScaler())])

preprocessor = make_column_transformer(
    (cat_prepro,make_column_selector(dtype_include= "object")),
     (cont_prepro,make_column_selector(dtype_exclude="object"))
     )

model = Pipeline([("preprocessor",preprocessor),
                  ("classifier",LogisticRegression(random_state= 1990))]).fit(X_train, y_train)


model.score(X_test,y_test)

plot_roc_curve(model,X_test,y_test);