I've read a question comparing Linear Regression and Random Forest Regression. I was supposed to choose between then and solve a problem (=predict prices). The question mentioned that "Random Forest can also handle missing values and cardinality well compared to linear regression". Why is that? How does that happen?
Generally, random forests are a much more sophisticated method than linear regression: it's an ensemble method with multiple decision trees, and a single decision tree is already a much more flexible method than linear regression.
- With linear regression, every instance considered must have all its values defined. There are several ways in which decision tree can handle missing values during training.
- I assume that cardinality refers to the number of features. Linear regression must include all the features in the linear model, and this can cause various problems (correlated or spurious features, overfitting). By contrast decision trees can select their features, and this is one of the features which make random forests a good ensemble method.