1- Which algorithms would benefit from data that has been transformed, so that distributions of continuous variables resemble that of a normal distribution ?
2- What would be the benefits of transforming variables in such a way ?
$\begingroup$
$\endgroup$
0
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
- It's not necessary for all features, but most algorithms would benefit from making highly-skewed or heavy-tailed features more Gaussian-like.
- Transforming to Gaussian makes the data symmetric and removes both heavy and long tails. That does a similar job of normalizing to $[0,1]$, without aggressively smashing the data into a bounded interval.
Side note: Transforming to standard Gaussian means that the center scale of every feature is the same. This is very helpful in making sure that your model learns "fairly", rather than putting undue influence on the feature with greater variance. See here for an example of how un-standardized data can harm K-means results. See also the comments in the other answer about regularization.
$\begingroup$
$\endgroup$
1
Models with direct regularization on their weights benefit from this. These regularizations add a prior to the model and punishes high weights. If your variables are not in the same range then the regularization is not the same for each input. Search for weight regularization.
Another form is where in neural networks the weights are initialized in a way that they expect the inputs to be normally distributed. If they are far from that, the impact in the later layers can be fairly big, greatly impacting convergence and numerical stability.
On the other hand, tree based methods don't care at all usually.
answered Mar 7, 2018 at 18:19
-
$\begingroup$ The OP is about transformations towards Gaussian, so not yet a (+1). $\endgroup$ Mar 7, 2018 at 18:40