I don't know much about how to deal with missing data. When the data is categorical it doesn't seem too bad, if I one-hot encode it without dropping any of the actual categories, the missing data is accounted for in the model because all the associated one-hot columns are zero. But for numeric data, I don't like the options as I understand them: omit the entire observation, or impute a value. Are there any other choices?
I'd lean toward omitting the data (in the data I am working on it would lead to ~2% data loss), but this doesn't help if you are trying to make predictions when data is missing, and I I can't find much useful discussion addressing this aspect of the problem (mostly it is around training the model).
I guess you could train multiple models that deal with all the permutations of missing features, and select the model matching the available predictive data? But that seems likely to be too unwieldy and time consuming to be practical. (The data set I'm working on now is artificial; it has 100 features, 96 of which are numeric, and all of which [both training and prediction sets] have from 2-20 missing values.)
Fwiw, my plan if I have to impute data is to regress using the 98% of complete training/prediction data, predict mean for each missing value, then randomize it based on the variance/CI.
I guess this boils down two questions/requests:
1) As in bold above re omission/imputation: are there any other choices?
2) General advice on how to proceed, given the predictions must be based on missing data?