I am currently working on a pancreatic cancer dataset which has numerous features including smoking, family history, age, etc. I have used the feature_importances_ method in sklearn to find and plot the individual feature importances, but wanted to know how to find the importance of combined variables (e.g. are those who smoke and have a family history of pancreatic cancer much more likely than others?). I know it is not as simple as adding the individual feature contributions for the combined features, and wanted to know if there is a specific method/function that can be used to do that. I have also used treeinterpreter to find the feature contributions.
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
My recommendation would be to use Partial Dependence Plots, which show the marginal effect one or two features have on the predicted outcome. They plot the average predictions over a range of values of the predictor(s) you specify, with the influence of all other predictors "averaged out". If you are familiar with linear regression, it is like plotting the predicted value of the target for different values of a predictor multiplied by the estimated weight/coefficient.
In your question, you are interested in two predictors (smoking and family history), so you would plot a two-way partial dependence plot showing the dependence of the target variable on joint values of smoking and family history.
There is a PDPBox package you can install that produces a variety of very nice plots and has some tutorials: https://github.com/SauceCat/PDPbox. It claims to support all Scikit-Learn algorithms.
There is also a Cross Validated Q&A on interpreting PDP's for Random Forests (produced with R): https://stats.stackexchange.com/questions/121383/interpreting-y-axis-of-a-partial-dependence-plots
