I have a sklearn Random Forest classifier with 59 features as input. I'd like to plot the decision boundaries of only two features at indices i1 i2.
If I use the average/median values for the remaining features, the classifier ends up in a path that ignores the features i1/i2.
I'd like to plot the decision boundary of i1/i2 while ignoring the remaining features. Is this possible? or useful at all?
For decision trees, this can be both possible and useful. For the random forest (RF), each decision tree has their own boundaries for these features. And, since RF trains each tree with a subset of original features, some trees don't have any these features and so boundaries. This might also occur due to pruning. So, you have a set of boundaries, which can be plotted at the same time to get a feeling of how the trees in the forest learnt your dataset. However, these boundaries also depend on other features. If some more explanatory feature is selected in a higher node in some trees, the low level thresholds might mislead you.
