I understood random forest is building a model with multiple decision trees,

  1. Row sampling is based on bootstraping

My question is how feature selection is happening for each tree ?

Any help would be appreciated.


A random subset of features than using the best split logic as done in normal Tree

The Random Forest algorithm introduces extra randomness when growing trees; instead of searching for the very best feature when splitting a node, it searches for the best feature among a random subset of features. The algorithm results in greater tree diversity, which (again) trades a higher bias for a lower variance, generally yielding an overall better model.

enter image description here
$\hspace{3cm}$Image credit - SE( https://stats.stackexchange.com/a/438384/256691) - Ashish Anand

Extra-trees -
More randomness by getting the feature using a random Threshold instead of searching the best split.

When you are growing a tree in a Random Forest, at each node only a random subset of the features is considered for splitting. It is possible to make trees even more random by also using random thresholds for each feature rather than searching for the best possible thresholds (like regular Decision Trees do)

I used Extra-tree to explain the point to have a contrast. It helps in catching the point.
Please also read the SE link of the image

[Blockquote ref] - Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.