I have a dataset that contains 7 features. Values are not too large. I trained scikit-learn's RandomForestRegressor for predicting the target variable. The $R^2$ score turns out to be something aroung 0.2, but when I first scale x_train and x_test using scitkit-learn's MinMaxScaler, the $R^2$ score drops to 0.11. I heard decision trees are not affected by scaling so it is not necessary to scale the data. My question is why would the score drop so much if trees are not affected by scaling?

This is without scaling

regressor = RandomForestRegressor()
predictions = fit_predict(X_train, y_train, X_test)
evaluate(y_test, predictions)

This is with scaling

scaler = preprocessing.MinMaxScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.fit_transform(X_test)
regressor = RandomForestRegressor()
predictions = fit_predict(X_train_scaled, y_train, X_test_scaled)
evaluate(y_test, predictions)
  • $\begingroup$ Can you include details how you applied the scaler (e.g. code)? $\endgroup$
    – Sammy
    Jul 1 at 11:14
  • $\begingroup$ If you use the same min max scaler on the whole data set, you might have such problem. But using scalers for each column should be ok. $\endgroup$ Jul 1 at 12:58
  • $\begingroup$ I have added some code @Sammy. $\endgroup$ Jul 1 at 16:49
  • $\begingroup$ Well this code is ok I guess? @Nicolas M $\endgroup$ Jul 1 at 16:49
  • 1
    $\begingroup$ Check what the result is if you use transform to scale your test data instead of fit_transform $\endgroup$
    – Sammy
    Jul 1 at 17:27

Your error is that you train your model to work on the training data after a scaling operation that you defined (fit) on the training. But then to evaluate using your test data you refit the scaler on the test data, meaning you are going to apply a different scaling to the test set, than you did on the training data to train the model.

You need to not refit the scaler on the test set and just apply the transformation:

X_test_scaled = scaler.transform(X_test)

  • $\begingroup$ Yes simple mistake, I fixed it now score with scaling is 0.00015 lower instead of 0.1. I guess it is due to some rounding errors? $\endgroup$ Jul 5 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.