I'm interested in interaction effects in Random Forest. I try to implement Friedman H'statistic basically developped for Gradient boosting Friendman, 2001. But I have no idea how calculate (44) on the real data.

Part 8.1

If anybody has some examples or explanations?


In your article $\hat F$ is defined by Eq. (39). Although it's a bit abstract, it says that the partial dependency function $\hat F_j$ is obtained by averaging over all variables but a subset $s$.

As an example, the one-variable dependency function for the variable $1$ is given by $$\hat F_1(x_1) = \frac{1}{n}\sum_{i=1}^n \hat f \left (x_{1}, x_{i2}, \dots, x_{im} \right ) ,$$ where $\hat f$ is your model prediction function (Random Forest in your case), $n$ is the number of samples, $m$ is the number of features (I've included $\hat f$ instead of $\hat F$ for the predictor to avoid confusion). Here $x_1$ is a variable, and the $x_{i2}, \dots, x_{im}$ values are taken from your samples.

For example, if you have two variables and two samples $(x_1=0,x_2=0)$ and $(x_1=1, x_2=1$) in your dataset (this is a very dummy, pedagogical example), the partial dependence function for the first variable reads: $$ \hat F_1 (x_1) = \frac{1}{2}\left [ \hat f(x_1, x_2=0) + \hat f(x_1, x_2=1) \right ]. $$

Similarly, you can compute the two-variables partial dependence function as $$\hat F_{12}(x_1, x_2) = \frac{1}{n}\sum_{i=1}^n \hat f \left (x_{1}, x_{2}, \dots, x_{im} \right )$$ (same as before, but note that the $i$ index is not under $x_2$ anymore).

The two equation I've just made explicit are the only two terms that appear in the equation (44), thus you should be able to compute the $H$-statistic with them.

Let me know if it is still unclear, and check out this for more exegesis on the $H$-statistic.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.