0
$\begingroup$

XGboost build the boosted tree in the following way:

  • Each level of each tree (the phase of selecting the next feature with conditional value) selected according to the max gain (less impurity).
  • If we check the feature importance (based on gain) we sould see that the more the features have higher gain (same feature may contribute several times at each tree level) the more they will be important.

  • When building XGboost tree, there is no way that feature with less gain will be selected before feature with higher gain.

When using SHap:

  • The Shap method runs multiples instances of XGboost with different subset of features and calcuated the difference between the accuracy of results with or without the selected/ unselected features.
  1. So is it possible that shap feature importance result will be more accurate than gain ?
  2. If we run feature selection (wrapper model) and used the selected features to get the importance, is it possible that shap feature importance result will be more accurate than gain ?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.