1
$\begingroup$

In my current project, I am doing KNN imputation with K = 5 and I am using sklearn.impute.KNNImputer. I have a mix of continuous and nominal variables(encoded as 0/1 or ordinal ones that have been encoded as 0/0.25/0.5/0.75/1 etc). However, the docs say "Each sample’s missing values are imputed using the mean value from n_neighbors nearest neighbors found in the training set." Because of this, I am getting in-between values like 0.4 for nominal attributes. Is there any way to override this to change from mean to mode for nominal columns?

Also, I looked at missingpy and fancyimpute but they both seem to be using mean as well~

$\endgroup$
4
  • $\begingroup$ Nearest neighbor imputation uses a distance metric to determine the neighbours and if all of your variables are either binary variables or transformed ordinal variables you will likely get very bad results. Standard distance metrics do not handle categorical variables well. Imagine I have a categorical variable for country of residence and my variable of interest is language spoken. Well standard metrics will say that Canada is as far from USA as it is from Japan, but clearly that is not the case when predicting language. Also transforming your ordinal variable like that is probably a bad idea $\endgroup$
    – astel
    Jul 9, 2020 at 22:21
  • $\begingroup$ @astel, would it make sense to do separate imputations then? KNN for continuous variables and mode for nominal columns separately and then combine all the columns together or sth. $\endgroup$
    – moii789
    Jul 10, 2020 at 0:38
  • $\begingroup$ I’m saying knn imputation probably isn’t appropriate for your dataset if you have primarily categorical and ordinal data $\endgroup$
    – astel
    Jul 10, 2020 at 2:22
  • $\begingroup$ In your place, I would use separate imputer for nominal, ordinal and continuous variables. Say simple imputer for categorical and ordinal filling with the most common or creating a new category filling with the value of MISSING and only for continuous KNN $\endgroup$
    – Multivac
    Sep 7, 2020 at 14:37

2 Answers 2

1
$\begingroup$

By default scikit-learn's KNNImputer uses Euclidean distance metric for searching neighbors and mean for imputing values.

If you have a combination of continuous and nominal variables, you should pass in a different distance metric.

If you want to use another imputation function than mean, you'll have to implement that yourself.

$\endgroup$
1
$\begingroup$

The best way of solve this is using pipelines as follows:

Working example:

import pandas as pd
import numpy as np

from sklearn.pipeline import Pipeline
from sklearn.compose import make_column_selector, make_column_transformer
from sklearn.linear_model import LogisticRegression
from sklearn.impute import SimpleImputer, KNNImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import plot_roc_curve
from sklearn.datasets import fetch_openml

# Load the data
X, y = fetch_openml("titanic", version=1, as_frame=True, return_X_y=True)
X.replace({None:np.nan}, inplace = True)
# Some preprocessing to correct data types and replace None with nans for pipeline imputer
X.drop(["name","home.dest"], axis = 1, inplace = True)
X["embarked"] = X["embarked"].astype("object")
X["sex"] = X["sex"].astype("object")

X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, test_size=0.2)

cat_prepro = Pipeline([("imputer",SimpleImputer(strategy="most_frequent")),
                        ("encoder",OneHotEncoder(handle_unknown = "ignore"))])

cont_prepro = Pipeline([("imputer", KNNImputer()), ("scaler",StandardScaler())])

preprocessor = make_column_transformer(
    (cat_prepro,make_column_selector(dtype_include= "object")),
     (cont_prepro,make_column_selector(dtype_exclude="object"))
     )

model = Pipeline([("preprocessor",preprocessor),
                  ("classifier",LogisticRegression(random_state= 1990))]).fit(X_train, y_train)


model.score(X_test,y_test)

plot_roc_curve(model,X_test,y_test);

enter image description here

$\endgroup$
1
  • $\begingroup$ I don't think this does what OP is looking for: you impute categorical variables by their global mode, rather than each sample's nearest neighbors' mode. $\endgroup$
    – Ben Reiniger
    May 7, 2021 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.