34
$\begingroup$

MinMaxScaler() in scikit-learn is used for data normalization (a.k.a feature scaling). Data normalization is not necessary for decision trees. Since XGBoost is based on decision trees, is it necessary to do data normalization using MinMaxScaler() for data to be fed to XGBoost machine learning models?

$\endgroup$

3 Answers 3

37
$\begingroup$

Your rationale is indeed correct: decision trees do not require normalization of their inputs; and since XGBoost is essentially an ensemble algorithm comprised of decision trees, it does not require normalization for the inputs either.

For corroboration, see also the thread Is Normalization necessary? at the XGBoost Github repo, where the answer by the lead XGBoost developer is a clear:

no you do not have to normalize the features

$\endgroup$
2
  • $\begingroup$ doesn't reg_alpha and reg_lambda, the coefficients of regularization on weights, mean that weights should be on the same scale? Or does this happen naturally in XGBoost? $\endgroup$ Feb 24, 2022 at 7:08
  • $\begingroup$ @ItamarMushkin not sure I'm following; maybe better to open a new question with the details? And weights are not data. $\endgroup$
    – desertnaut
    May 9, 2022 at 10:30
1
$\begingroup$

While decision trees have a natural resistance to outliers, boosted trees are susceptible, since new trees are built off the residual. Normalization, or even just a log transform, will give you better protection from outliers.

For an XGB model that is planned to go into production, I recommend doing it. It's not like one can interpret the output anyway. I think of XGBoost as being on the black box side.

$\endgroup$
1
  • $\begingroup$ Normalization is not about removing outliers, which will be present anyway even after normalization; and the residuals have to do with the output, not with the features, which the question is about. $\endgroup$
    – desertnaut
    Apr 21, 2021 at 20:23
0
$\begingroup$

It's not a bad idea so much as it's unnecessary. So, if you don't do it, you leave your features on the scale they are already and thus in prediction of new data, you don't have to worry about scaling said data exactly the same.

It's unnecessary since the base learners are trees, and any monotonic function of any feature variable will have no effect on how the trees are formed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.