1
$\begingroup$

I've been looking for a paper where the Gini importance was first proposed, but I am not sure if this is actually how it came to be.

Here's the formula I am familiar with and am looking to find in a paper:

$$\frac{N_s}{N_t} * \left(i - \frac{{N_s}_r}{N_s} i_r - \frac{{N_s}_l}{N_s} i_l \right)$$

where

$N_s$ = number of samples at a particular node

$N_t$ = number of total samples

$i$ = Impurity

$*_r$ = measure of the right child node

$*_l$ = measure of the left child node

If I understand properly, this is the formula sklearn's random forests also use in model.feature_importances_

Tags (since I don't have the rep to create new ones yet): Feature Importance Score, Gini Importance

$\endgroup$
1
$\begingroup$

There is a paper that covers "The origins of the Gini index". Gini index was detailed by Leo Breiman et al. in "Classification and regression trees" book in 1984. Leo Breiman also wrote a seminal paper on Random Forests in 2001 which includes the notion of feature importance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.