When to use Random Forest

Question

I understand Random Forest models can be used both for classification and regression situations.

Is there a more specific criteria to determine where a random forest model would perform better than common regressions (Linear, Lasso, etc) to estimate values or Logistic Regression for classification?

Answer 1

Adding some extra general points to the previous answer:

• As a decision tree algorithm, Random Forests are less influenced by outliers than other algorithms. A good discussion about it is here.
• They also do not make any assumptions about the underlying distribution of your data, and can implicitly handle collinearity in features, because if you have two highly similar features, the information gain from splitting on one of the features will also use up the predictive power of the other feature. Read about it here.
• Random Forests can be used for feature selection because if you fit the algorithm with features that are not useful, the algorithm simply won't use them to split on the data. It's possible to extract the 'best' features (which could be the total number of times a feature was used to split on the data, or the mean decrease in impurity etc). However, as with my point above, you cannot read too much into the relative importance of the features, especially if you have ones that are highly correlated.
• I think the biggest deciding factor of whether to use a RF or another algorithm is probably if you want to understand more about the relationship that features have with the target and the degree of influence they have. If this is important for your use case (for example, you want to know if a feature has a positive or negative relationship with the target and the degree to which it affects the outcome) then others like Logistic Regression and Lasso are better choices.
• Also, if you want your model to extrapolate to predictions for data that is outside of the bounds of your original training data, a Random Forest will not be a good choice.

Hope this helps!

Answer 2

This is only a general answer but in case it helps:

• In general decision trees tend to be "robust" in the sense that they can work with practically any kind of data, in particular in cases where other methods such as linear or logistic regression might struggle. For example they have no problem in the case of heterogeneous features, e.g. when mixing categorical and numerical features, or mixing completely different ranges of values, etc.

• Random forests add ensemble learning to the mix, making decision trees even more robust and especially well equipped to deal with noisy data, whereas standard regression methods can get easily confused by noise.

Intuitively I see decision trees (random forests included) as the "swiss army knife" of supervised learning: efficient, versatile, easy to use.