StandardScaler and MinMaxScaler vs RobustScaler

Question

I've recently read that Standard Scaler functions best in situations where the distribution of the features are approximately normal.

MinMaxScaler works in a way that it preserves the features' original shape.

Both of them are sensitive to outliers as sklearn itself states.

But I can't seem to get RobustScaler. I've read people saying that it reduces the effect of outliers in the distribution, so if one considered the outliers shouldn't have an effect on the data, one should use RobustScaler. But I don't think that makes much sense because if one would think outliers shouldn't impact the data, then it would make more sense to remove then before performing the scaling.

I've also read people saying that it doesn't reduce the effect of the outlier, but it doesn't let the distribution get distorted like MinMax and Standard Scaler do.

Therefore, I'm having a hard time understanding situations in which it would make sense to use different types of scalers, specially when it comes to RobustScaler, should I use it when I have outliers or when I want to desconsider the effect of those outliers on the data?

Answer 1

Extreme values will usually cause problems with many methods/models, but that does not mean you should remove them. If your model does not fit your data, change your model, not your data.

Extreme values will impact the mean/SD or min/max of the data which will then have an effect on data normalization. An extremely high value will cause your variable to be in [0,1] after min/max normalization, but most of your data will be close to 0, so it will have an asymmetric distribution compared to other similar variables without extremes. Likewise an extremely high value will cause the SD to be inflated and this variable will be much more narrowly distributed around 0 as a result of standardization, even though the total SD will be 1.

Robust normalization, which uses the median and the median absolute deviation, will make sure that your variables are much more comparable with each other, that is that most data will have mass at roughly the same place, but the extreme values will still be extreme. Hopefully these extreme values won't affect the normalization parameters and change the distribution of all the data, and this is what robust normalization tries to achieve.