I've been reading about how to approach missing categorical features in test data, and the most common approach is to use imputation - for example using the last known value or getting the majority feature in given row/column.

Is there a better way to approach missing data? Why can't the classifier just ignore the missing feature, and rely on the known features? Why is imputation necessary?

I'm using scikit learn, and am trying to feed in NaN to a classification model (naive bayes, logistic regression, decision tree, random forest), to see what happens.

  • $\begingroup$ Usually the best way to handle NaN is just as a separate categorical value. No imputation needed. $\endgroup$ – Paul Dec 8 '16 at 16:49

Certain models are able to deal with missing values 'naturally', like certain tree based models. Most models however are just a mathematical function which is shaped after the training data. A very easy example would be:

$$f(x) = \alpha x_1 + log(2x_2)$$

What do you do if one of them is NaN? The function is undefined and no prediction can be made. By imputing the values you make a reasonable guess where this sample lies on the data manifold.

  • $\begingroup$ Ok that makes it clearer, thanks :) I've read about using combined evidence with bayesian classifiers as a technique for missing data, but don't understand it too well - do you know anything about this? $\endgroup$ – gbhrea Aug 10 '16 at 15:09
  • $\begingroup$ XGBoost is usually good at handling missing data, no need for manual imputation when using this model. The XGBoost will impute the data internally for you based on loss reduction. Otherwise, for most cases, it is better to use one of these well established methods for imputation: k-means clustering imputation, statistical (mean, median, etc) imputation, interpolation and extrapolation, etc. Each of these will have some pros and cons, and all of them should be used carefully. $\endgroup$ – Donald S Jun 17 '20 at 0:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.