2
$\begingroup$

I don't know much about how to deal with missing data. When the data is categorical it doesn't seem too bad, if I one-hot encode it without dropping any of the actual categories, the missing data is accounted for in the model because all the associated one-hot columns are zero. But for numeric data, I don't like the options as I understand them: omit the entire observation, or impute a value. Are there any other choices?

I'd lean toward omitting the data (in the data I am working on it would lead to ~2% data loss), but this doesn't help if you are trying to make predictions when data is missing, and I I can't find much useful discussion addressing this aspect of the problem (mostly it is around training the model).

I guess you could train multiple models that deal with all the permutations of missing features, and select the model matching the available predictive data? But that seems likely to be too unwieldy and time consuming to be practical. (The data set I'm working on now is artificial; it has 100 features, 96 of which are numeric, and all of which [both training and prediction sets] have from 2-20 missing values.)

Fwiw, my plan if I have to impute data is to regress using the 98% of complete training/prediction data, predict mean for each missing value, then randomize it based on the variance/CI.

I guess this boils down two questions/requests:

1) As in bold above re omission/imputation: are there any other choices?

2) General advice on how to proceed, given the predictions must be based on missing data?

-- Edit/Add 15-Mar-20 -- I would particularly like to know what other options are, and what is considered best practice, when implementing models such as regression that do not cope with missing values?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

2 Questions.

  1. Yes there are certain algos that deal with it internally. For example, lightGBM will ignore missing values during a split, then allocate them to whichever side reduces the loss the most. Check section 3.2 here Or Catboost for example also

  2. Try to save/impute information as much as possible, if there is just too much missing values (30%+) than just drop the column. You will do more harm injecting the noise while imputing. (NOTE 30% is approximation based on experience, some datasets require less or more)

Improve this answer
$\endgroup$
2
  • $\begingroup$ I should have noted that methods that split data would probably suit; I guess what I'm looking for is how to handle missing numerical data with regression, SVM, and other methods that don't handle missing data. I share your concern about imputing data. I guess this is an inherent advantage in decision tree based models? $\endgroup$
    – James
    Dec 21, 2019 at 23:04
  • $\begingroup$ not all. RF does not do this, you have to do it manually $\endgroup$
    – Noah Weber
    Dec 22, 2019 at 8:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.