I am reading a lot about the Random Forest regressor. I read about bagging (Bootstrap and aggregation) and random subspace. But I am not sure if the random forest regressor is just using bagging or bagging and random subspace method. Because in some articles, the random subspace method seems like an alternative to the bagging method.

  • $\begingroup$ I recently followed this videos youtube.com/… If you follow them it will give you a better understanding of RF $\endgroup$ – Carlos Mougan Jan 24 '20 at 9:54
  • $\begingroup$ Thank you for sharing this video, but i dont understand why he is create a bootstrapp function and just use the implementation for the random forest? $\endgroup$ – ml_learner Jan 24 '20 at 11:28
  • $\begingroup$ Check the video of how to build a decision tree. If you start by there and follow the videos you should have a high knowledge of random forest and decision trees algorithms $\endgroup$ – Carlos Mougan Jan 24 '20 at 11:41
  • $\begingroup$ Own answer in the SO thread Why is Random Forest with a single tree much better than a Decision Tree classifier? may be of help here. $\endgroup$ – desertnaut Sep 21 '20 at 12:05

Most modern implementations do both, at least optionally.

sklearn has max_features and bootstrap.
ranger has mtry and replace/sample.fraction.
xgboost's random forest has colsample_bynode and subsample.
h2o has mtries/col_sample_rate_per_tree and sample_rate (and a couple modifiers).

  • $\begingroup$ Thank you for the answer. If a want to use a random forest regressor with bagging i just have to select ´´´bootstrap=true´´´ and ´´´max_feature=auto´´´? What if i want to use both methods? $\endgroup$ – ml_learner Jan 24 '20 at 7:08
  • $\begingroup$ Correct (I might specify max_features=None just to be more explicit). If you want both, still bootstrap=True, now max_features= basically anything else, depending on how small you want your subspace. sqrt is the most common that I've seen, or a float if you want to do a hyperparameter search on different subspace sizes. $\endgroup$ – Ben Reiniger Jan 24 '20 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.