I've been reading about how to approach missing categorical features in test data, and the most common approach is to use imputation - for example using the last known value or getting the majority feature in given row/column.

Is there a better way to approach missing data? Why can't the classifier just ignore the missing feature, and rely on the known features? Why is imputation necessary?

I'm using scikit learn, and am trying to feed in NaN to a classification model (naive bayes, logistic regression, decision tree, random forest), to see what happens.

  • $\begingroup$ Usually the best way to handle NaN is just as a separate categorical value. No imputation needed. $\endgroup$ – Paul Dec 8 '16 at 16:49

Certain models are able to deal with missing values 'naturally', like certain tree based models. Most models however are just a mathematical function which is shaped after the training data. A very easy example would be:

$$f(x) = \alpha x_1 + log(2x_2)$$

What do you do if one of them is NaN? The function is undefined and no prediction can be made. By imputing the values you make a reasonable guess where this sample lies on the data manifold.

  • $\begingroup$ Ok that makes it clearer, thanks :) I've read about using combined evidence with bayesian classifiers as a technique for missing data, but don't understand it too well - do you know anything about this? $\endgroup$ – gbhrea Aug 10 '16 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.