How does a random forest algorithm deal with a few irrelevant input variables

Question

I have a list of variables from which I would like to train a Random Forest Algorithm. I suspect that a few of my input variables, which have noisy distributions, won't be able to predict much. Can I use them anyway, knowing the algorithm will eliminate them in the process, or should I beware that these variables may biais my model?

Answer 1

Random Forest tends to be not too sensitive to features with low predictive power. The reason is that RF looks for a "best split" given a subset of features (columns) and observations (rows) at each node. So "weak" features will likely be ignored in most cases (splits).

However, removing the $x$ percent weakest features may increase the model's performance. In case you use sklearn, there are convenience functions to do this, e.g. SelectFromModel(). See the docs for more details.

>>> from sklearn.ensemble import ExtraTreesClassifier
>>> from sklearn.datasets import load_iris
>>> from sklearn.feature_selection import SelectFromModel
>>> X, y = load_iris(return_X_y=True)
>>> X.shape
(150, 4)
>>> clf = ExtraTreesClassifier(n_estimators=50)
>>> clf = clf.fit(X, y)
>>> clf.feature_importances_  
array([ 0.04...,  0.05...,  0.4...,  0.4...])
>>> model = SelectFromModel(clf, prefit=True)
>>> X_new = model.transform(X)
>>> X_new.shape               
(150, 2)