So I'm quickly learning that dealing with missing values for feature(s) in some of your observations is a part of every day life in data. I get the gist of imputation, when/how it's appropriate and when it's not, and I'll read up on it in the near future. But, what about this:
Suppose you have predictors $X_1, \dots, X_p$, and you want to model, say, a binary response $Y$ via, say, logistic regression. Instead of just imputing values to missing predictor values, couldn't you have a separate model built for every possible subset of the predictors, and apply that model for prediction when those predictors happen to be present? Each of those models would naturally be trained on just the data for which those predictors (and also others) are present. This seems to me a more reasonable approach than just making up values, but I have no theoretical justification for this.
I do realize that this involves building $2^p$ different models, with $2^p$ different model matrices, etc., but for a moderate $p$ and $n$ it could be feasible, and especially if only a few of your features tend to be missing more often than others.
Is this ever done? And if so, is there a standard way to implement this in R? In the case of logistic regression, you can specify to R's glm function how you want it to handle NA values, but your only options seem to be either tossing out observations altogether, or some kind of imputation scheme.