0
$\begingroup$

Is there an algorithm out there that creates a random forest but then prunes all the leaves that have an impurity measure above a certain threshold that I would determine?

In other words, if I set min samples per leaf to be 500 and leaves have to have at least a 90% purity for example, the algorithm would only keep leaves that respect these parameters.

My dataset is extremely noisy so most leaves have a gini impurity around 0.5 but some leaves are almost around 0. I care only for the latter in my use case. Is there an algorithm that does something like what I described?

$\endgroup$

1 Answer 1

1
$\begingroup$

If some leaves are pruned then it means that the model cannot predict the instances which would normally have fallen into these leaves. The logic of a decision tree is to represent every possible instance: like any supervised learning method, it must be able to predict for every possible instance as input. In some cases where all the leaves have low purity, this condition would even make the model unable to predict anything at all. This is why there is no such algorithm, at least not as a regular supervised learning algorithm.

However you can rely on the probability predicted for the instance instead, this gives you exactly the same information in a way consistent with standard ML.

$\endgroup$
1
  • $\begingroup$ Thank you for your answer. For my use case, it's ok if for most inputs it returns "nothing" as I only care for precision. In other words, the algorithm will return all leaves that have a low impurity as the conditions that lead to these aforementioned leaves will tell me for example : 550 cases are covered by this leaf and 90% are positive. That is much more meaningful to me than having high impurity nodes voting in the final decision or contributing in someway to the final probability of the model for an input X. $\endgroup$ May 10, 2021 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.