I was wondering whether one could adjust a random forest to estimate a range of values instead of receiving one exact estimate. What I mean by that: my current rf predicts a value of e.g. 5 based on different variables. What I would be interested in, is the range around that value, e.g. 4.2-5.3.
Thank you!
Yes, you can.
Let $\mathbb{X}$ be the input space and $\mathbb{Y}$ the output space. In your scenario, $\mathbb{Y} = \mathbb{R}$, the set of real numbers (i.e., a regression setting).
A decision tree model (DT) is a function $t : \mathbb{X} \to \mathbb{Y}$ that outputs $y \in \mathbb{Y}$ for input $x \in \mathbb{X}$, namely, $t(x) = y$.
A random forest model (RF) is a collection $\mathcal{T} = \{t_1,t_2,\ldots,t_n\}$ of $n$ (base) DTs. Moreover, an RF is equipped with a voting aggregation function $v : \mathbb{Y}^n \to \mathbb{Y}$ of all unit votes of each DT. Therefore, an RF is also a function $\mathcal{T} : \mathbb{X} \to \mathbb{Y}$ that outputs $y \in \mathbb{Y}$ for input $x \in \mathbb{X}$ by exploiting the function $v$, namely, $\mathcal{T}(x) = v(t_1(x),t_2(x),\ldots,t_n(x)) = v(y_1,y_2,\ldots,y_n) = y$.
We can investigate the mathematical properties of $v$. In your setting, we can generalize it to predict confidence intervals (CIs) instead, that is, $v : \mathbb{Y}^k \to \mathbb{Y} \times \mathbb{Y}$ implying $\mathcal{T} : \mathbb X \to \mathbb Y \times \mathbb Y$. For example, instead of the prediction $\mathcal T(x) = 5$ it can now predict $\mathcal T (x) = (4.5,5.5)$.
How to achieve this desired property?
Let $\mathcal Y_x = \{t_i(x) \mid t_i \in \mathcal T \}$ be the unit votes of the $n$ DTs in the RF $\mathcal T$ for input $x$, which are independent by definition of RF. We obtain an interval estimate by injecting the calculation of CIs over $\mathcal Y_x$. Informally, the latter is a "range" of values as wanted.
To conclude, $v$ can be any function (e.g., a linear regression, a DT, a neural network, etc.).
