# Standardizing giving worse results

I am training a Decision tree regressor on the famous Boston House Price dataset. I read that tree based models are fairly immune to scaling so I tried to see practically. Before scaling I was getting MAE = 2.61. After scaling using StandardScaler, I got worse results which is quite surprising as scaling the data should result in an improvement. Also if not an improvement, I was hoping to see the same result as I got without scaling since I am using a tree based model. Also there are no categorical variables in this dataset and all the variables included in the scale_list a large difference in their min and max values hence have been included. I have not included and binary (1's and 0's)variables.

 train_x, test_x, train_y, test_y = train_test_split(data2, y, test_size = 0.2, random_state =
69)

#STANDARDIZING THE DATA
scale_list = ['CRIM', 'NOX', 'RM', 'AGE', 'DIS', 'TAX', 'PTRATIO', 'B', 'LSTAT']

train_x_stand = train_x.copy()
test_x_stand = test_x.copy()

for i in scale_list:

scale = StandardScaler().fit(train_x_stand[[i]])
train_x_stand[i] = scale.transform(train_x_stand[[i]])
test_x_stand[i] = scale.transform(test_x_stand[[i]])


Any help would be appreciated!

As you well mentioned, tree-based models are not sensitive to feature scaling, but on the contrary it might help with the convergency of finding the minimum in the optimization on boosted models

I replicated your code and I found pretty much the same metrics in both scaling and no scaling versions of the model.

from sklearn.datasets import load_boston
from sklearn.tree import DecisionTreeRegressor
from sklearn.pipeline import Pipeline
from sklearn.compose import make_column_transformer
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_absolute_error

X, y = pd.DataFrame(boston.data, columns=boston.feature_names), pd.DataFrame(boston.target)

train_x, test_x, train_y, test_y = train_test_split(X, y, test_size = 0.2, random_state = 69)

scale_list = ['CRIM', 'NOX', 'RM', 'AGE', 'DIS', 'TAX', 'PTRATIO', 'B', 'LSTAT']

preprocessor = make_column_transformer((StandardScaler(),scale_list), remainder="passthrough")

smodel = Pipeline([("scaler",preprocessor),
("model", DecisionTreeRegressor(random_state=42))]).fit(train_x, train_y)

tmodel = Pipeline([("model", DecisionTreeRegressor(random_state=42))]).fit(train_x, train_y)

print(f"metric with scale features: {mean_absolute_error(test_y, smodel.predict(test_x))}\nmetric with no scale features:{mean_absolute_error(test_y, tmodel.predict(test_x))}")

metric with scale features: 2.6941176470588237
metric with no scale features:2.662745098039216


I dare to think that something is wrong with the way you are applying the scaling. I recommend you to check it or use pipelines.

Also I was unsure if you used Ensemble or a simple DT, but in both cases remember to set the random_state in order to have reproducible results.

Hope it helps!