3
$\begingroup$

Forgive me if this is a duplicate question, I haven't found anything that answers my question specifically after searching for a while.

I have a dataset which I'm using to predict mobile app user retention, using the RandomForestClassifier in the SciKit Learn package. I'm pleased with the accuracy I'm getting and I'm planning on including a number of other metrics including precision, recall, Matthew's Corr Coefficient etc. I'm pretty sure the model is good.

The key thing that I'm interested in here are the features themselves. I want to know what is contributing to my user churn. I have extracted the feature importances and plotted a nice looking graph, but now I'm stuck. I'd ideally like to know how each variable influences the churned/not churned outcome. The problem with GINI feature importances is that I can see which ones are most influential, but for example with continuous variables I want to know at which value the RF found best to split on. I don't need to see this for every feature as I have 70+ only the most 'important' ones.

I saw a very nice decision tree plot here, but cannot find any way of reproducing something similar using scikit learn. I'm open to other suggestions. Thanks in advance :)

Improve this question
$\endgroup$
3
  • $\begingroup$ stackoverflow.com/questions/40155128/… $\endgroup$
    – Hobbes
    Commented Jun 20, 2017 at 22:48
  • 1
    $\begingroup$ It's also in the documentation. If your forest is big, you're probably better off with partial dependence plots instead of looking at every node in every tree. $\endgroup$
    – oW_
    Commented Jun 20, 2017 at 23:17
  • $\begingroup$ @oW_ That's absolutely perfect! Just what I'm looking for! The doc even says it should be used with ideally 1 to 2 features selected from Variable Importance metrics. So this is an ideal next step. If you'd write up an answer, I will happily mark it as accepted :) $\endgroup$
    – Dee Carter
    Commented Jun 22, 2017 at 10:22

1 Answer 1

Reset to default
5
$\begingroup$

The documentation offers a couple options. To plot the individual trees in your forest, one can access them like model.estimators_[n].tree_ and then plot them with export_graphviz as explained in the documentation, or you can follow this example that directly prints the structure in text format.

However, I would say this is not the best idea, because a feature can occur in different trees and nodes with different split points. You probably get a better intuition about your features from partial dependence plots that try to isolate the effect of one variable on your response variable.

As a bonus, here is a good article about more alternatives to gain insight into your model (not all applicable to random forests).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.