XGboost build the boosted tree in the following way:
- Each level of each tree (the phase of selecting the next feature with conditional value) selected according to the max gain (less impurity).
If we check the feature importance (based on
gain) we sould see that the more the features have higher gain (same feature may contribute several times at each tree level) the more they will be important.
XGboosttree, there is no way that feature with less gain will be selected before feature with higher gain.
Shapmethod runs multiples instances of
XGboostwith different subset of features and calcuated the difference between the accuracy of results with or without the selected/ unselected features.
- So is it possible that shap feature importance result will be more accurate than gain ?
- If we run feature selection (wrapper model) and used the selected features to get the importance, is it possible that
shapfeature importance result will be more accurate than gain ?