For some machine learning methods it is recommended to use feature normalization to use features that are on the same scale, especially for distance based methods like k-means or when using regularization. However, in my experience, boosting tree regression works less well when I use normalized features, for some strange reason. How is your experience using feature normalization with boosted trees does it in general improve our models?


3 Answers 3


Boosting trees is about building multiple decision trees. Decision tree doesn't require feature normalization, that's because the model only needs the absolute values for branching.

Wikipedia for decision tree:

Requires little data preparation. Other techniques often require data normalization....

However, it's always a good idea to normalize your features because:

  • It's easier to visualize and interpret your model
  • It's easier to compare another model (e.g. SVM) with the same data set
  • 2
    $\begingroup$ I thought the same. But doing feature normalization with boosted trees sometimes leads to worse models (in my experience). Therefore, it does not seem to be "always a good idea". I am asking more for experience than for general directions. $\endgroup$
    – Soerendip
    Commented Jan 13, 2017 at 6:00

How is your experience using feature normalization with boosted trees does it in general improve our models?

My rather limited experience with scaling of features suggests that it has virtually no impact on xgboost results.

I suppose by normalisation you mean subtracting the mean and then dividing by standard deviation. If you calculated the statistics based on entire dataset (including holdout) you would get data leakage, which might indeed, at least theoretically, degrade the performance on holdout.

According to my understanding of xgboost, the correctly performed scaling should have no impact on the performance.

I suggest you double check your implementation or provide more details on how you do it, preferably with including a reproducible example.


Sometimes feature scaling/normalization is usefully even for tree-based model. For example, assume formula, where formula denotes the mean value of feature x in a time period T, if you don't do feature scaling/normalization, your model may perform poorly if the distribution of x changes in the future formula, and time-series normalization will help in this case.

Besides, if you are using Trees with linear models at leaves for lightgbm, it's important to scale the data.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.