I need some help understanding my partial dependence plots for binary features passed to a GradientBoostClassifier when comparing them to the feature importances. For some background, my goal here is to investigate user churn. My classifications are: 0 = not churned, 1 = churned.

enter image description here First of all when I plot the feature importances I get some interesting results which do not agree with the feature importances from my random forest. I put this down to algorithmic differences since many do align or are similar, but my specific concern is the binary feature 'registeredEmail', which is clearly very important in the random forest model, but not in the other (gradient boost, as shown). This is not my main concern, just adding in case relevant to my question.

Secondly, and this is the part I am most confused by. Why does my feature importance chart show that 'playerInAlliance' (another binary feature) is significantly more important than 'registeredEmail', since when I check the partial dependencies there is a much steeper slope for 'registeredEmail'. I would interpret this as the churned prediction being highly influenced by a player not having a registered email, compared to being less likely to predict churn if the player does have their email registered. Is this correct?

Please note these plots are in descending order of feature importance, from left to right. Also ignore the title.

Comparing this to 'playerInAlliance' I can see that no matter if the player is in an alliance or not, this feature is influential in predicting a churned player. I don't understand how to interpret this, as if there's no real difference between players churning when in an alliance compared to having no alliance, then why is my GradientBoostClassifier considering it to be a highly important and highly dependent feature?

enter image description here

Finally, if I plot both of these binary features against each other on one plot, we can see that dependence on a player not having a registered email is massively higher than if the player is in an alliance or not. So why is the registeredEmail feature less significant in terms of importance?

TL;DR: How do I interpret Binary features in partial dependence plots?

Any help appreciated.

  • $\begingroup$ just wondering... in sklearn feature importances usually add up to 1.0. did you renormalize them? $\endgroup$ – oW_ Jun 30 '17 at 15:29
  • $\begingroup$ hi oW,I made the importances relative to the 'most' important feature. So the most important feature is 1.0, the rest are relative to this. Hope that helps! $\endgroup$ – Dan Carter Jul 3 '17 at 9:17
  • $\begingroup$ how many % of players have registered with their emails? $\endgroup$ – oW_ Jul 7 '17 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.