I am trying to understand transformations but this question seems to be in my and some people's mind. If we have a numeric variable in EVERY data science case. Transforming data(Log, power transforms) into normal distribution will help the model to learn better? And stationarity. Stationarity is a different thing than transforming data to make it have a normal distribution. Is Transforming EVERY numeric data to stationery will make EVERY model learn better too?
1
-
$\begingroup$ In general the answer is definitely no: one should only transform the data for some specific reason related to the learning algorithm or the dataset. Doing this systematically is likely to be counter-productive in most cases. $\endgroup$– ErwanMar 23, 2022 at 11:38
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
7
The short answer is no, you don't always need to transform your data to a normal distribution.
This depends a lot on the learning algorithm you're using. Additionally, you should treat continuous and categorical variables differently.
Continuous variables:
Tree-based models such as Decision Trees, Random Forest, Gradient Boosting, XGBoost, and others, are not affected by the distribution of your data.
However, algorithms like Linear Regression, Logistic Regression, KNN or Neural Nets can be highly affected by both the distribution and scale of your data. You will likely both get better results and finish training the model faster if you transform the data for these algorithms.
Categorical variables:
Independently of what algorithm you're using, you should one-hot-encode nominal categorical variables (this is the most common way, but there are other approaches such as Feature Hashing and Bin-counting that might work better if you have many categories). If they're ordinal, you should keep them as they are (given that they are integers, and if not, convert them to integers while maintaining the implied order).
Extra side note:
Also, make sure to not scale the entire dataset at once to prevent data leakage. Instead, scale your train set, then apply the same scaler on the test set, as explained in this SO answer.
-
$\begingroup$ If linear regression needs normal features, how do you explain ANOVA using categorical features? // That last comment about data leakage is spot on, however. $\endgroup$– DaveMar 23, 2022 at 15:01
-
$\begingroup$ Perhaps I wasn't clear. You can use categorical features with linear regression (and with the other non tree-based algorithms I mentioned) as well, but they should not be transformed to normal. Instead, use one-hot encoding if they're nominal or keep as they are if ordinal. I assumed the question referred to continuous variables. I've updated my answer to clarify this. $\endgroup$– JakobMar 23, 2022 at 17:34
-
$\begingroup$ But then if categorical variables need not be normal, why does any other variable in a linear regression have to be normal? Consider, for instance, a designed experiment that intentionally has uniform features. // You might be interested in reading about common myths in linear regression, one of which is a requirement for normal features. $\endgroup$– DaveMar 23, 2022 at 17:40
-
$\begingroup$ @Jakob thank you for your beautiful and detailed answer. However, there is an infinite number of models in deep learning-machine learning. How will I know which one requires transformations? Because google search might not help in some cases. For example googling: "Does xx architecture need transformation?" may not help. $\endgroup$– canPMar 24, 2022 at 3:47
-
$\begingroup$ @Dave It's not that they necessarily have to be normal. You can still train your model if the data is not normal. However, the results will likely be worse and it will probably take longer time to train the model if not. It's common practice to train and test many models with different transformation techniques to see which one performs best. As long as you are aware of data leakage and treat the train and test sets separately, the results on your test set should be indicative of your model's general performance, and thus, what transformation that works best for that particular case. $\endgroup$– JakobMar 24, 2022 at 12:00