I was wondering whether one could adjust a random forest to estimate a range of values instead of receiving one exact estimate. What I mean by that: my current rf predicts a value of e.g. 5 based on different variables. What I would be interested in, is the range around that value, e.g. 4.2-5.3.

Thank you!


1 Answer 1


Yes, you can.

Let $\mathbb{X}$ be the input space and $\mathbb{Y}$ the output space. In your scenario, $\mathbb{Y} = \mathbb{R}$, the set of real numbers (i.e., a regression setting).

A decision tree model (DT) is a function $t : \mathbb{X} \to \mathbb{Y}$ that outputs $y \in \mathbb{Y}$ for input $x \in \mathbb{X}$, namely, $t(x) = y$.

A random forest model (RF) is a collection $\mathcal{T} = \{t_1,t_2,\ldots,t_n\}$ of $n$ (base) DTs. Moreover, an RF is equipped with a voting aggregation function $v : \mathbb{Y}^n \to \mathbb{Y}$ of all unit votes of each DT. Therefore, an RF is also a function $\mathcal{T} : \mathbb{X} \to \mathbb{Y}$ that outputs $y \in \mathbb{Y}$ for input $x \in \mathbb{X}$ by exploiting the function $v$, namely, $\mathcal{T}(x) = v(t_1(x),t_2(x),\ldots,t_n(x)) = v(y_1,y_2,\ldots,y_n) = y$.

We can investigate the mathematical properties of $v$. In your setting, we can generalize it to predict confidence intervals (CIs) instead, that is, $v : \mathbb{Y}^k \to \mathbb{Y} \times \mathbb{Y}$ implying $\mathcal{T} : \mathbb X \to \mathbb Y \times \mathbb Y$. For example, instead of the prediction $\mathcal T(x) = 5$ it can now predict $\mathcal T (x) = (4.5,5.5)$.

How to achieve this desired property?

Let $\mathcal Y_x = \{t_i(x) \mid t_i \in \mathcal T \}$ be the unit votes of the $n$ DTs in the RF $\mathcal T$ for input $x$, which are independent by definition of RF. We obtain an interval estimate by injecting the calculation of CIs over $\mathcal Y_x$. Informally, the latter is a "range" of values as wanted.

To conclude, $v$ can be any function (e.g., a linear regression, a DT, a neural network, etc.).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.