Mean imputation is generally bad practice because it doesn’t take into account feature correlation. Imagine we have a table showing age and fitness score and imagine that an eighty-year-old has a missing fitness score. If we took the average fitness score from an age range of 15 to 80, then the eighty-year-old will appear to have a much higher fitness score that he actually should. Thus, I wonder if we have any way to use mean imputation without violating feature correlation.
For me, a straightforward solution is to replace mean of the whole population in data by mean of a group of similar subjects. For example, in the above example, we can fill missing score of a person by mean score of other persons of the same age, or for more flexibility, in the same range of age. This approach still works if we have more than 2 features and still want to take into account correlation among all features. We just need to define a similarity metrics between subjects based on all features, e.g. cosine similarity. Of course, similarity computation is expensive, that is a limitation of this approach. So I wonder if there is any better alternative.
