while working on some regression problems I have found that if the target variable is skewed, making it normally distributed(using transformations) almost always helps. Why is that?
Should we also transform independent variables to have near normal distributions?
In some cases, it may actually help getting better results (depending on the model type), but it is also likely that the improvement comes from the fact that the performance metric is computed differently. For instance, a skewed distribution will lead to high MSE values due to cases located on the other side of the distribution, while the MSE is limited if the data is transformed to a normal distribution. So when comparing the cases, make sure you evaluate the performance on the back-transformed target.
Cases where the model will actually perform better with a normally distributed target include, among others, Gaussian process regression, because of the underlying assumption of a Gaussian random variable. There should be quite a few other model types which somehow have similar assumptions, and thus perform better with transformed data.
To avoid writting things like making sure assumptions are met, for example linear regression and residuals I also like to think about it the following way (in Analogy to balance of classes in binary classification for example):
If you bin you data in a way that the more skewed part is one bin and the other one, les skewed is the other bin, than you have unbalanced-target-problem, that should be addressed. (for starters there is not enough samples of class 1 for algo to learn something)
