2
$\begingroup$

Many discussions of missing data in supervised (and unsupervised) learning deal with various methods of imputation, like mean values or EM. But in some cases the data will be missing as a necessary consequence of the data generation process.

For instance, let's say I'm trying to predict students' grades, and one of the inputs I want to analyze is the average grades of the student's siblings. If a particular student is an only child, then that value will be missing, not because we failed to collect the data, but because logically there is no data to collect. This is distinct from cases where the student has siblings, but we can't find their grades.

Other examples abound: say we're in college admissions and we want to include students' AP exam results, but not all students took AP exams. Or we're looking at social network data, but not all subjects have facebook and/or twitter accounts.

These data are missing, but they're certainly not missing at random. And many algorithms, such as all supervised learning packages in scikit-learn, simply demand that there be no missing values at all in the data set.

How have people dealt with this in the past, and what off-the-shelf solutions are there? For instance, I believe the gradient boosting algorithm in R uses trees with three possible branches: left, right, and missing. Any other alternatives out there?

Improve this question
$\endgroup$
1
  • $\begingroup$ Simple way to deal with missing data is to use population's average for this variable. Intuition here is that if we don't know anything about it, then we would want to use something "neutral" - not better and not worse than an average. From statistical point of view, we may say that if that value existed, its prior would most probably be distributed normally with mean in population's expectation. $\endgroup$
    – ffriend
    Commented Oct 2, 2014 at 22:40

1 Answer 1

Reset to default
1
$\begingroup$

I solved a similar problem not so long ago. Let X be a numerical variable that has missing values. First, we assign the value 0 to those instances in which the value is missing. Then, we add a new categorical variable to our model, X_missing, which domain is {True,False}. We obtain a data model with mixed numerical/categorical variables. You can then apply gradient descent to train a regression model from these variables.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the input. This means greatly expanding the number of rhs variables, since you now need two variables for each one that has missing values. Also, I'd think you have to be careful about making 0 the default value for lack of information. But this is a good starting point, and I'll think more about it. $\endgroup$
    – David Pepper
    Commented Oct 3, 2014 at 20:11
  • $\begingroup$ Well, in my particular case the number of new variables was not a problem, since I did not have so many numerical features. And regarding the value 0 for the missing values, it made sense in the specific context of my problem. But maybe these ideas may be adapted somehow to your particular problem. $\endgroup$
    – Pablo Suau
    Commented Oct 10, 2014 at 8:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.