0
$\begingroup$

I have a dataset that contains 7 features. Values are not too large. I trained scikit-learn's RandomForestRegressor for predicting the target variable. The $R^2$ score turns out to be something aroung 0.2, but when I first scale x_train and x_test using scitkit-learn's MinMaxScaler, the $R^2$ score drops to 0.11. I heard decision trees are not affected by scaling so it is not necessary to scale the data. My question is why would the score drop so much if trees are not affected by scaling?

This is without scaling

regressor = RandomForestRegressor()
predictions = fit_predict(X_train, y_train, X_test)
evaluate(y_test, predictions)

This is with scaling

scaler = preprocessing.MinMaxScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.fit_transform(X_test)
regressor = RandomForestRegressor()
predictions = fit_predict(X_train_scaled, y_train, X_test_scaled)
evaluate(y_test, predictions)
$\endgroup$
7
  • $\begingroup$ Can you include details how you applied the scaler (e.g. code)? $\endgroup$
    – Jonathan
    Jul 1, 2021 at 11:14
  • $\begingroup$ If you use the same min max scaler on the whole data set, you might have such problem. But using scalers for each column should be ok. $\endgroup$ Jul 1, 2021 at 12:58
  • $\begingroup$ I have added some code @Sammy. $\endgroup$ Jul 1, 2021 at 16:49
  • $\begingroup$ Well this code is ok I guess? @Nicolas M $\endgroup$ Jul 1, 2021 at 16:49
  • 1
    $\begingroup$ Check what the result is if you use transform to scale your test data instead of fit_transform $\endgroup$
    – Jonathan
    Jul 1, 2021 at 17:27

1 Answer 1

3
$\begingroup$

Your error is that you train your model to work on the training data after a scaling operation that you defined (fit) on the training. But then to evaluate using your test data you refit the scaler on the test data, meaning you are going to apply a different scaling to the test set, than you did on the training data to train the model.

You need to not refit the scaler on the test set and just apply the transformation:

X_test_scaled = scaler.transform(X_test)

$\endgroup$
1
  • $\begingroup$ Yes simple mistake, I fixed it now score with scaling is 0.00015 lower instead of 0.1. I guess it is due to some rounding errors? $\endgroup$ Jul 5, 2021 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.