Everything that I've read about random forests has indicated that they do not require scaling of inputs and that scaling should not affect the construction of the model. Here's a quote from another SE question (https://stats.stackexchange.com/questions/255765/does-random-forest-need-input-variables-to-be-scaled-or-centered):
Random Forests are based on tree partitioning algorithms.
As such, there's no analogue to a coefficient one obtain in general regression strategies, which would depend on the units of the independent variables. Instead, one obtain a collection of partition rules, basically a decision given a threshold, and this shouldn't change with scaling. In other words, the trees only see ranks in the features.
Basically, any monotonic transformation of your data shouldn't change the forest at all (in the most common implementations).
Here's what I'm using currently. If I remove the weights multiplier, I get a different model (i.e., different value from model.score
and different tree depths) despite setting random_state=0
in either case.
model = RandomForestRegressor(n_estimators=10, criterion='mse', random_state=0)
weights = np.arange(1,self.x_train.shape[1]+1)[None,:]
# weights = [[ 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.]]
model.fit(self.x_train * weights, self.y_train)
By comparison, I noticed that if I use XGBRegressor
rather than RandomForestRegressor
, scaling does not change the model.
Is there an obvious mistake that I'm making or is the explanation above not correct?
model.score
andget_depths
on the individual trees and they are both different. $\endgroup$