3
$\begingroup$

I've been reading about how to approach missing categorical features in test data, and the most common approach is to use imputation - for example using the last known value or getting the majority feature in given row/column.

Is there a better way to approach missing data? Why can't the classifier just ignore the missing feature, and rely on the known features? Why is imputation necessary?

I'm using scikit learn, and am trying to feed in NaN to a classification model (naive bayes, logistic regression, decision tree, random forest), to see what happens.

Improve this question
$\endgroup$
1
  • $\begingroup$ Usually the best way to handle NaN is just as a separate categorical value. No imputation needed. $\endgroup$
    – Paul
    Dec 8, 2016 at 16:49

1 Answer 1

Reset to default
2
$\begingroup$

Certain models are able to deal with missing values 'naturally', like certain tree based models. Most models however are just a mathematical function which is shaped after the training data. A very easy example would be:

$$f(x) = \alpha x_1 + log(2x_2)$$

What do you do if one of them is NaN? The function is undefined and no prediction can be made. By imputing the values you make a reasonable guess where this sample lies on the data manifold.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Ok that makes it clearer, thanks :) I've read about using combined evidence with bayesian classifiers as a technique for missing data, but don't understand it too well - do you know anything about this? $\endgroup$
    – gbhrea
    Aug 10, 2016 at 15:09
  • $\begingroup$ XGBoost is usually good at handling missing data, no need for manual imputation when using this model. The XGBoost will impute the data internally for you based on loss reduction. Otherwise, for most cases, it is better to use one of these well established methods for imputation: k-means clustering imputation, statistical (mean, median, etc) imputation, interpolation and extrapolation, etc. Each of these will have some pros and cons, and all of them should be used carefully. $\endgroup$
    – Donald S
    Jun 17, 2020 at 0:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.