I understood random forest is building a model with multiple decision trees,
- Row sampling is based on bootstraping
My question is how feature selection is happening for each tree ?
Any help would be appreciated.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ I'll start reading from here: scikit-learn.org/stable/auto_examples/inspection/… $\endgroup$– Itamar MushkinJul 5, 2020 at 7:57
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
A random subset of features than using the best split logic as done in normal Tree
The Random Forest algorithm introduces extra randomness when growing trees; instead of searching for the very best feature when splitting a node, it searches for the best feature among a random subset of features. The algorithm results in greater tree diversity, which (again) trades a higher bias for a lower variance, generally yielding an overall better model.
$\hspace{3cm}$Image credit - SE( https://stats.stackexchange.com/a/438384/256691) - Ashish Anand
Extra-trees -
More randomness by getting the feature using a random Threshold instead of searching the best split.
When you are growing a tree in a Random Forest, at each node only a random subset of the features is considered for splitting. It is possible to make trees even more random by also using random thresholds for each feature rather than searching for the best possible thresholds (like regular Decision Trees do)
I used Extra-tree to explain the point to have a contrast. It helps in catching the point.
Please also read the SE link of the image
[Blockquote ref] - Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition
