1
$\begingroup$

I am trying to understand - when would someone choose Random Forest over XGBoost and vice versa. All the articles out there highlights on the differences between both. I understand them. But when actually given a real world data set, how should we approach the problem to choose between these?

For eg: Is there a set of statistical tests for variance check, and then choose? Or is it simply like you have a number of features, and cannot really choose to do parameter tuning, so you apply Random Forest to get results?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Let´s say that the best way to choose is empirical. You run both algorithms in the dataset and check which one has better performance.

It's true that you can do a lot of theoretical analysis but at the end you have to try no matter what. They both use decision trees ensemble so the results should not be too different. By experience gradient boosting tends to achieve better results. Also, it is more mathematically complicated to understand.

Normally decision trees don´t require too much parameter turning, or at least less than other models.

There is no classical statistic test that will tell you will perform better. There are some heuristics but I find them overly complicated.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the answer. I was also trying to understand the intuition when someone has good experience working on a set of data sets in a specific domain and always come across conversations like "Because the data is like this, or features are "n", I would apply a certain algorithm. I was curious if there was a certain approach as starters for Ensemble Models. $\endgroup$
    – dshero
    Mar 17, 2020 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.