Does it make sense to use feature importances based in gini index for other classifiers?

Question

I would like to know if makes sense running yellowbrick.features.FeatureImportances with a RandomForestClassifier model in order to find most influent features, and then fit a different model (e.g. MLPClassifier) with them.

Unfortunately, FeatureImportances doesn't support MLPClassifier to find features. Docs can be checked here

Answer 1

Feature Importance Or Correlation is dependent on the approach used.
It's like the model is saying, "When I used my approach I find the particular feature very important."

When another model uses the same approach or an approach that is a superset of that approach, then it will be able to find that Importance too.

Few examples -

• If RF finds a Feature very important and able to achieve a good score. It means the feature is a good predictor for the scenario but LinearRegression might not able to exploit the predictor
• Neural Network can exploit that since it can learn any function
• DecisionTree may struggle to recognize a good feature but easily seen by LinearRegression. See image below