I recently came across SHAP while looking for feature-importance methods. To use SHAP, first a model needs to be created, and then based on the predictions made by the model, SHAP values are calculated.

If I use a random forest model and then calculate the SHAP values, how much is the role of the model accuracy? If the model accuracy is not very good (50-60%), can SHAP values be considered reliable? Or if the model accuracy is good (>90%), why SHAP is used if random forest can provide me the feature importance?

Could someone explain the role of model accuracy while considering SHAP for determining feature importance?

  • $\begingroup$ Note that it is always a crappy idea to ask a crappy model for its feature importance. Now the other question is: what is a good model - the answer might be that you need to consider cost sensitivity ... But that depends on your business case $\endgroup$
    – Ggjj11
    Jan 9 at 21:59
  • $\begingroup$ @Ggjj11 makes sense. So my point was if I have a good model, why shouldn't I believe in RF's feature importance? SHAP algorithm may be superior to RF but still, it's built on RF predictions. $\endgroup$
    – lsr729
    Jan 9 at 22:02
  • 1
    $\begingroup$ Well there is 2 main ways of RF feature importances 1) default in sklearn is the Split-Feature importance which was shown to be bad, also due to interaction with the cardinality of a feature 2) the other one is the OOB feature importance which was reported to be much better. Also see stackoverflow.com/a/15821880/12229416 $\endgroup$
    – Ggjj11
    Jan 9 at 22:06


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.