0
$\begingroup$

Does normalization (ex: log or sqrt) of skewed features help to reduce model's bias? or it is better to leave the skewed distribution of predictors as it is?

$\endgroup$

2 Answers 2

0
$\begingroup$

The most common use of the word normalization is to rescale the feature set such that all features have a common range of values. This is the kind of normalization supported by something like sklearn's MinMaxScaler().

If this is what you mean by normalization then weither or not you should do it depends on the algorithm you're using. In general it's a good prepocessing step but models using metrics which aren't normalized (e.g. K-Means clustering) will be negatively impacted by normalization.

But from the context of your question what you seem to mean is something like:

"If I have a feature X with a lognormal distribution, should I replace it with log(X) since log(X) will be normally distributed?"

I can't recall ever seeing an example of someone engineering features to enforce a normal distribution on the feature set. And I think there are some examples of why you wouldn't want to do this.

For example, say your training a linear regression model between a target variable Y and a single feature X, where Y is normally distributed and X has a lognormal distribution. If X and Y appear to have a linear relationship then you wouldn't replace X with log(X) since log(X) wouldn't have a linear relationship with Y and thus the performance of your linear model would deteriorate.

Hope this helps

$\endgroup$
0
$\begingroup$

If you mean preprocessing methods involving BoxCox/YJ transforms, they do reduce the outlier effects and thus can (potentially) reduce bias.

They do much more than that however (like distorting linear relationships) so whether it's actually helpful depends on many factors.

In practice, if you use gridsearch-like routines for cross-validation of your models, you can search for the optimal preprocessing function in a pipeline as well. Sklearn PowerTransformer() and QuantileTransfomer() (including the uniform transformation for the latter) are able to yield better overall metrics compared to StandardScaler()/MinMaxScaler()/RobustScaler() in many cases, however there doesn't seem to be any rule of thumb here, except maybe for neural networks which tend to prefer quantile transformations.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.