Suppose we have a dataframe df in python, with numerical and categorical variables.
when do we replace by mean and when by median.
is there an approach which needs to be followed.
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
In my experience, there is no one correct answer to this question. It depends mostly on your problem itself. I'll base my answer on most general cases though.
For most Numerical data, a parametric method is the best way to go. If you have some sample data which you can use as a baseline. Try using a parametric method such as Normal distribution, Exponential Distribution or so depending on your model.
For categorical data, you can try various oversampling techniques or undersampling techniques. Methods such as SMOTE, ADASYN etc can be used to fill in some missing values.
However, these methods are just general cases. But as aforementioned it depends more on your model and the type of data you are working with. It could be that maybe you are using scientific data and you don't need to generate synthetic data so based on how much data you have available you can discard the missing rows.
All such nuances come into play before making the most optimal decision for your data model.