When to use Random Forest

Question

I understand Random Forest models can be used both for classification and regression situations.

Is there a more specific criteria to determine where a random forest model would perform better than common regressions (Linear, Lasso, etc) to estimate values or Logistic Regression for classification?

Answer 1

Adding some extra general points to the previous answer:

• As a decision tree algorithm, Random Forests are less influenced by outliers than other algorithms. A good discussion about it is here.
• They also do not make any assumptions about the underlying distribution of your data, and can implicitly handle collinearity in features, because if you have two highly similar features, the information gain from splitting on one of the features will also use up the predictive power of the other feature. Read about it here.
• Random Forests can be used for feature selection because if you fit the algorithm with features that are not useful, the algorithm simply won't use them to split on the data. It's possible to extract the 'best' features (which could be the total number of times a feature was used to split on the data, or the mean decrease in impurity etc). However, as with my point above, you cannot read too much into the relative importance of the features, especially if you have ones that are highly correlated.
• I think the biggest deciding factor of whether to use a RF or another algorithm is probably if you want to understand more about the relationship that features have with the target and the degree of influence they have. If this is important for your use case (for example, you want to know if a feature has a positive or negative relationship with the target and the degree to which it affects the outcome) then others like Logistic Regression and Lasso are better choices.
• Also, if you want your model to extrapolate to predictions for data that is outside of the bounds of your original training data, a Random Forest will not be a good choice.

Hope this helps!

Answer 2

Please consider the following:

1) Random forest algorithm can be used for both classifications and regression task.
2) It typically provides very high accuracy.
3) Random forest classifier will handle the missing values and maintain the accuracy of a large proportion of data.
4) If there are more trees, it usually won’t allow overfitting trees in the model.
5) It has the power to handle a large data set with high dimensionality

Ultimately, what algo you choose to work with is up to you. you definitely want the predictive capabilities of the algo to be pretty high (over 90%). Sometimes other algos beat the RF algo, but I have found that often the RF is quite good! Usually, I start with RF, and if I see decent performance, I am done. I believe, in around at least 80% of the time, I'm done. If you are not getting good results from the RF algo, test some others.

This gives a nice comparison of a few different algos.

import pandas
import matplotlib.pyplot as plt
from sklearn import model_selection
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB
from sklearn.svm import SVC
# load dataset
url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv"
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
dataframe = pandas.read_csv(url, names=names)
array = dataframe.values
X = array[:,0:8]
Y = array[:,8]
# prepare configuration for cross validation test harness
seed = 7
# prepare models
models = []
models.append(('LR', LogisticRegression()))
models.append(('LDA', LinearDiscriminantAnalysis()))
models.append(('KNN', KNeighborsClassifier()))
models.append(('CART', DecisionTreeClassifier()))
models.append(('NB', GaussianNB()))
models.append(('SVM', SVC()))
# evaluate each model in turn
results = []
names = []
scoring = 'accuracy'
for name, model in models:
    kfold = model_selection.KFold(n_splits=10, random_state=seed)
    cv_results = model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring)
    results.append(cv_results)
    names.append(name)
    msg = "%s: %f (%f)" % (name, cv_results.mean(), cv_results.std())
    print(msg)
# boxplot algorithm comparison
fig = plt.figure()
fig.suptitle('Algorithm Comparison')
ax = fig.add_subplot(111)
plt.boxplot(results)
ax.set_xticklabels(names)
plt.show()

Reference: How To Compare Machine Learning Algorithms in Python with scikit-learn

Answer 3

This is only a general answer but in case it helps:

• In general decision trees tend to be "robust" in the sense that they can work with practically any kind of data, in particular in cases where other methods such as linear or logistic regression might struggle. For example they have no problem in the case of heterogeneous features, e.g. when mixing categorical and numerical features, or mixing completely different ranges of values, etc.

• Random forests add ensemble learning to the mix, making decision trees even more robust and especially well equipped to deal with noisy data, whereas standard regression methods can get easily confused by noise.

Intuitively I see decision trees (random forests included) as the "swiss army knife" of supervised learning: efficient, versatile, easy to use.