Why do we pick random features in random forest

Question

I understand that random forest is a stylized version of bagging of trees. We choose randomly data points as well as random features for constructing random forest.

But if we use just plain version of bagging by choosing only data points randomly then we have trees which have trained on more number of features unlike the random forest in the stylized version. Since learning with more features every individual tree has more information about the data points and so are more 'intelligent' in some sense than the individual trees in the random forest.

So why does the random forest using stylized version of bagging performs better than the random forest using plain implementation of bagging?

I understand that the random forest using the stylized version gives a model lower variance but since each of its trees is trained on some of the features shouldn't it make the model a bit high biased?

Answer 1

The idea of random forests is basically to build many decision trees (or other weak learners) that are decorrelated, so that their average is less prone to overfitting (reducing the variance). One way is subsampling of the training set. The reason why subsampling features can further decorrelate trees is, that if there are few dominating features, these features will be selected in many trees even for different subsamples, making the trees in the forest similar (correlated) again.

The lower the number of sampled features, the higher the decorrelation effect. On the other hand, the bias of a random forest is the same as the bias of any of the sampled trees (see for example Elements of Statistical Learning), but the randomization of random forests restrict the model, so that the bias is usually higher than a fully-grown (unpruned) tree. You are correct in that you can expect a higher bias if you sample fewer features. So, "feature bagging" really gives you a classical trade-off in bias and variance.

Answer 2

My intuitions is that training each tree on a subset of all variables helps the less useful variable to be used at all. Since often there are some features highly correlated to the target and then all trees will just use these very good features and never ever use the weak ones. By working on random subsets the weak features are used sometimes and contribute to the result a little.