What is the benefit of using imputation to handle missing values in training data?

Question

This is a bit of a theoretical question. Does imputation to handle missing values in training data add that much benefit to the final prediction accuracy? It seems to me that the imputation would be good when there is very little missing data. However, if there is very little missing data, then we could probably train a good model even without imputation. The reverse argument is when there are a lot of missing values - the imputation wouldn't be good and then the final model wouldn't be either.

Answer 1

We implement imputation for models that do not support NaN values. For example linear Regressors. Here's an example of the output you'll get if you try to do so :

ValueError: Input X contains NaN.
LinearRegression does not accept missing values encoded as NaN natively. 

For supervised learning, you might want to consider:

• sklearn.ensemble.HistGradientBoostingClassifier or
• sklearn.ensemble.HistGradientBoostingRegressor

In which accepts missing values encoded as NaNs natively.

Alternatively, it is possible to preprocess the data, for instance, by using an imputer transformer in a pipeline or dropping samples with missing values. Ref

You can find a list of all estimators that handle NaN values on this page

Imagine you have 3 features: ['feature A', 'feature B', 'feature C'].

• ['feature A', 'feature B'] contain no NaN values, and 'feature C' contains 90% NaN value. In this case, you'll be tempted just to drop 'feature C' and work with features ['feature A', 'feature B'].

However, let's give a quick example of why that's not always the right choice. We'll suppose that:

• for the model using just 'feature C' on the 10% available data, $R2_{score}=0.9$,
• for the model using features ['feature A', 'feature B'] on the 90% available data, $R2_{score}=0.2$

Then, we can see that 'feature C' can explain the model's output variance really well in this case. And that both features ['feature A', 'feature B'] can explain some of the variances. So, all 3 variables are important.