0
$\begingroup$

I fit a dataset with a binary target class by the random forest. In python, I can do it either by randomforestclassifier or randomforestregressor.

I can get the classification directly from randomforestclassifier or I could run randomforestregressor first and get back a set of estimated probabilities. Then I can find a cutoff value to derive the predicted classes out of the set of probabilities. Both methods can achieve the same goal (i.e. predict the classes for the test data).

Also I can observe that

randomforestclassifier.predict_proba(X_test)[:,1])

is different from

randomforestregressor.predict(X_test)

So I just wanna confirm that both methods are valid and then which one is better in random forest application?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Binary target class already says it all: classification. Therefore: Random Forest Classifier $\endgroup$
    – Nikolas Rieble
    Jan 5, 2017 at 8:44

2 Answers 2

Reset to default
2
$\begingroup$

If you are doing a classification task, use random forest classifier. If you want probabilities use predict_proba or you can use predict directly to get classes. You might be getting correct results but understand that the random forest regressor works on a different cost function, and is not constrained to give outputs between 0 and 1, so what you are getting out of the regressor are not really probabilities. If you run it on enough datasets it might start giving you outputs greater than 1. Hence, bottom line stick to classifier. Also I will recommend reading up a bit on the differences of regression and classification tasks.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Thanks for that. In the scope of my dataset, I didn't see the output larger than 1. So you mean the underlying regression was not the logistic regression but rather a linear regression? If so, how can it take the y_train of 0,1 to fit the linear regression? $\endgroup$
    – LUSAQX
    Jan 5, 2017 at 8:01
  • 1
    $\begingroup$ You can fit a linear regression on an output of 1,0. Try it. Also, I am not saying its linear regression, random forests are used for fitting non-linear hypotheses. The only thing is that the regressor's cost function works for continuous output and not discrete values. Hence, it is ideal to use the classifier because it is actually churning out probabilities. $\endgroup$
    – Himanshu Rai
    Jan 5, 2017 at 8:06
  • $\begingroup$ Thanks. Yes I also think the regressor was non-linear one. But you mentioned a key concept here, i.e. cost function. I will do some research. On the other hand, even though the output of the regressor was not probability (0-1 metric), I can also find a cutoff value to get the predictive classes and the output can be treated as a general test score rather than probability. So this is not the key, and I will examine the cost function. Thanks for help. $\endgroup$
    – LUSAQX
    Jan 5, 2017 at 8:14
  • $\begingroup$ But how about the methodology to use logistic regression for the classification, i.e. do the classification out of an inner regression model? In that way, it is also to get the discrete class out of a continuous value. $\endgroup$
    – LUSAQX
    Jan 5, 2017 at 8:20
  • $\begingroup$ Cheers. Yes Random Forest Classifier uses cross-entropy function. I dont really know, what regressor uses . Look into it. $\endgroup$
    – Himanshu Rai
    Jan 5, 2017 at 8:20
0
$\begingroup$

On the one hand, as Himanshu Rai has mentioned, if you are using RandomForestRegressor for classification tasks, the "probability" you get might greater than 1, or less than 0. That's not what we expected.

On the other hand, RandomForestClassifier uses accuracy_score as loss function, while RandomForestRegressor uses r2_score. That's a big difference.

And some advice

References

http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier.score

http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html#sklearn.ensemble.RandomForestRegressor.score

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.