From reading articles I've read that random forests are suited for representing both linear and non linear data quite well, but I can't find an explanation as to how and why it has his flexibility?
Any information to clarify this would be great!
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
Decision Trees are able to create both linear and nonlinear boundaries and so are Random Forests.
This is because of how they cluster the data based on nested "if-else" statements. These statements draw vertical/horizontal lines between the samples and cluster them in rectangles. Consecutively, rectangles of the same class can be far away from each other (with other class rectangles in between) but still they belong to the same class. This is how nonlinear relations are modeled.
-
$\begingroup$ Using a nonlinear staircase to fit linear is not ideal though. It needs a much bigger model to have reasonable accuracy, though for many problems it does not really matter $\endgroup$ – jonnor Jun 29 '18 at 8:44
-
$\begingroup$ OP asked to explain "how and why it has this flexibility", not if it's ideal. "Ideal" solutions pretty much depend on the nature of the data. $\endgroup$ – pcko1 Jun 29 '18 at 8:49
-
$\begingroup$ Agreed! Just wanted to point out that there are disadvantages also. No silver bullets and all that $\endgroup$ – jonnor Jun 29 '18 at 9:45
