I have a dataset with 20 features(columns that is). I create a few models pairs with a subset of these parameters.
For example: If I have 6 columns (named A, B, C, D, E, F) with 10k lines of data, one of model in the pair will have (A, B, C) with all 10k lines of data and the other (A, B, C) with maybe say 6k lines of data. Yet another pair will have the parameters (B, C, E), however, the number of lines is always fixed at 10k and 6k.
I am limited to using a sci-kit learn's decision tree and this cannot be changed. I use the following function to calculate my features:
model = clf_gini.best_estimator_ print(model.feature_importances_)
My question is: **Is there any formal, programmatic way to compare feature importance across model pairs using sci-kit, other than intuition? Also, does it make sense to compare to models trained on some similar some different parameters? **
Please let me know if the question is ambiguous, I will clarify.