I am reading a lot about the random forest regressor. I reading about bagging (bootstrap and aggregation) and random subspace. But I am not sure if the random forest regressor just using bagging or bagging and random subspace method. Because in some article the random subspace method seems like an alternative to the bagging method. I hope you understand my question and thank you for reading my question.

  • $\begingroup$ I recently followed this videos youtube.com/… If you follow them it will give you a better understanding of RF $\endgroup$ – Carlos Mougan Jan 24 at 9:54
  • $\begingroup$ Thank you for sharing this video, but i dont understand why he is create a bootstrapp function and just use the implementation for the random forest? $\endgroup$ – ml_learner Jan 24 at 11:28
  • $\begingroup$ Check the video of how to build a decision tree. If you start by there and follow the videos you should have a high knowledge of random forest and decision trees algorithms $\endgroup$ – Carlos Mougan Jan 24 at 11:41

Most modern implementations do both, at least optionally.

sklearn has max_features and bootstrap.
ranger has mtry and replace/sample.fraction.
xgboost's random forest has colsample_bynode and subsample.
h2o has mtries/col_sample_rate_per_tree and sample_rate (and a couple modifiers).

  • $\begingroup$ Thank you for the answer. If a want to use a random forest regressor with bagging i just have to select ´´´bootstrap=true´´´ and ´´´max_feature=auto´´´? What if i want to use both methods? $\endgroup$ – ml_learner Jan 24 at 7:08
  • $\begingroup$ Correct (I might specify max_features=None just to be more explicit). If you want both, still bootstrap=True, now max_features= basically anything else, depending on how small you want your subspace. sqrt is the most common that I've seen, or a float if you want to do a hyperparameter search on different subspace sizes. $\endgroup$ – Ben Reiniger Jan 24 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.