Everything that I've read about random forests has indicated that they do not require scaling of inputs and that scaling should not affect the construction of the model. Here's a quote from another SE question (https://stats.stackexchange.com/questions/255765/does-random-forest-need-input-variables-to-be-scaled-or-centered):
Random Forests are based on tree partitioning algorithms.
As such, there's no analogue to a coefficient one obtain in general regression strategies, which would depend on the units of the independent variables. Instead, one obtain a collection of partition rules, basically a decision given a threshold, and this shouldn't change with scaling. In other words, the trees only see ranks in the features.
Basically, any monotonic transformation of your data shouldn't change the forest at all (in the most common implementations).
Here's what I'm using currently. If I remove the weights multiplier, I get a different model (i.e., different value from
model.score and different tree depths) despite setting
random_state=0 in either case.
model = RandomForestRegressor(n_estimators=10, criterion='mse', random_state=0) weights = np.arange(1,self.x_train.shape+1)[None,:] # weights = [[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.]] model.fit(self.x_train * weights, self.y_train)
By comparison, I noticed that if I use
XGBRegressor rather than
RandomForestRegressor, scaling does not change the model.
Is there an obvious mistake that I'm making or is the explanation above not correct?
get_depthson the individual trees and they are both different. $\endgroup$