I recently ran an elastic net model on my data. My predictors are mostly skewed. I found my model perform slightly better when I standardize on log-transformed data than standardizing on original data.

I have some general questions about how standardizing and/or log transform predictors affect prediction result in machine learning.

  1. Standardizing (center&scale): I understand many models require standardizing predictors. For those models that do NOT require standardization, theoretically how does standardization affect the prediction result? I know it changes model interpretation. But my primary goal here is prediction. If it does not matter, then can I always scale my predictor before modeling, at least making training data more consistent when exploring many different models?

  2. log transformation: When predictors are skewed, we often do log transformation to make them normal. How do log-transformation affect the prediction result? For models like random forest, I thought it should not matter. But I did see some difference in prediction performance to my prior experience. Again, prediction is the goal. in addition, even for simple linear regression, the assumption is residual is normally distributed. It does not require normality of predictors. Do log-transform (normalizing) predictors help make residual normal?

  3. standardization on log-transformed predictors: For models like elastic net, when predictors are skewed, should I do standardization on log-transformed predictors? As I mentioned earlier, I did see some difference in prediction performance.

I think these are some pretty common data preprocess questions. Some online sources are pretty confusing to me. I look forward to some experts to clarify these importance questions. I believe a clear answer will benefit a lot of people in this community. Thanks a lot.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.