Every Blog and Youtube video talks about the same steps:

  1. Choose that you have to build N number of tree and do the task 2-5 below for all of those N trees
  2. Choose random samples with replacement
  3. Choose f features randomly from total F
  4. For each tree, at each split find the node with Minimum Gini entropy (or Max Info gain) and split
  5. Run the test sample to get a result
  6. Aggregate the results

No what I'm trying to understand intuitively is that how the next iteration is done?

For example, in Linear Regression (assuming two variables), we calculate the difference between predicted and actual values and move the line with theta degrees

Or in Neural Networks, we use Gradient Descent and Chain rule so that weights of the matrix at each layer are updated according to what they contributed to the actual prediction in the next iteration.

How is it done in Random Forest? What is the learning? How the loss (objective function) is propagated back to the Nodes? The closest I could Find was this slide: enter image description here


2 Answers 2


In a random forest classifier, there is no backpropagated loss. Instead, the N trees are grown independently from each other and then, for a new prediction, a majority vote is performed among all N outcomes.

The only function that is used at each split is the Entropy / Information Gain, but this function uses the entire training subset available for the growing of each tree and does not have any learning component.


If you understand decision trees and the intuition behind the decision tree then the random forest is simple.

To understand the decision tree, we can intuitively consider what we want it to do. We want to find the best possible way for us to split up our data according to the input features. The way of deciding what is best is the loss function, entropy/gini or any other loss function you want. The model is built directly from the data itself, and the model is the data itself. A random forest is an extension of decision trees by creating a bunch of them with different data features.

A major difference between the intuition behind a tree and a linear model is that a decision tree does not have "parameters". The tree model performs direct manipulations on the data itself. Whereas the linear models need to create and find the optimal parameters. Thus the need to perform back propagation to "learn" the parameters.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.