Let's assume that we have a binary classification problem, and we built a decision tree on our data set.
Assuming that we have 5 features, then the decision tree, in the first step, will choose the best feature of the 5, and on this feature it will choose the best threshold in order to split the data set, and then continue to make the tree deeper etc. The definition of
best is the lowest classification error.
My question is: Since the decision tree, on each step, chooses the best feature to split on, and the best threshold to split on, then why random forest (which is many decision trees), is an improvement of decision trees? Shouldn't a decision tree be sufficient?
I more mean it like: If you have a
decision tree classifier, and a
random forest classifier with the same parameters, when possible, (
number of children etc), will the
decision tree classifier score the same on the
training set, with the
random forest classifier?