Replacing missing values with mean of feature calculated from previously replaced values

Question

I am not sure how to ask this, but I will try my best.

I have replaced some missing values in a feature with the mean of the feature conditional on a second categorical feature. However, not all missing values were replaced because some categories of the second feature did not have any values in the first feature, so its mean cannot be calculated. Then, I tried again by replacing the leftover missing values with mean conditional on a third feature. The third feature also contains values that were just replaced with the mean of the first feature. So, is it alright to replace missing values with the mean calculated from values that include some that have once replaced other missing values?

Answer 1

So, is it alright to replace missing values with the mean calculated from values that include some that have once replaced other missing values?

There's no law against it :)

But by replacing missing values everywhere in many different ways you're probably damaging the reliability of your dataset at some point, and that could in turn mess up your experiment.

You don't give any detail about the task or the data so I assume you chose this approach on purpose, but just in case let me remind you of other options:

• you could discard features for which most values are missing, unless they happen to be very important for the task
• you could discard instances which have many/some missing values, especially if you have a very large dataset anyway
• you could use a ML method able to deal with missing values by itself