I use the MLPClassifier from scikit learn. I have about 20 features. Is there a scikit method to get the feature importance? I found
but it seems that it only exists for decision trees.
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
The short answer is that there is not a method in scikit-learn to obtain MLP feature importance - you're coming up against the classic problem of interpreting how model weights contribute towards classification decisions.
However, there are a couple of great python libraries out there that aim to address this problem - LIME, ELI5 and Yellowbrick:
LIME (or Local Interpretable Model-agnostic Explanations, blog post here, arxiv paper here) which "explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally around the prediction", or basically to explain model outputs by approximating the results of a classifier with a model that can be explained. It's on pypi and they have examples on their github page - very straightforward to get started.
ELI5 (or Explain Like I'm 5) is a "Python library which allows to visualize and debug various Machine Learning models using unified API". Although not all scikit-learn integration is present when using ELI5 on an MLP, Permutation Importance is a method that "...provides a way to compute feature importances for any black-box estimator by measuring how score decreases when a feature is not available", which saves you from trying to implement it yourself.
Yellowbrick is "a suite of visual diagnostic tools called “Visualizers” that extend the Scikit-Learn API to allow human steering of the model selection process" and it's designed to feel familiar to scikit-learn users. Compared to the other two libraries here it doesn't offer as much in the way for diagnosing feature importance, but it's still worth mentioning for more general use cases.
The Multi-Layer Perceptron does not have an intrinsic feature importance, such as Decision Trees and Random Forests do. Neural Networks rely on complex co-adaptations of weights during the training phase instead of measuring and comparing quality of splits.
A simpler approach for getting feature importance within Scikit can be easily achieved with the Perceptron, which is a 1-layer-only Neural Network.
from sklearn.datasets import load_breast_cancer from sklearn.linear_model import Perceptron X, y = load_breast_cancer(return_X_y=True) clf = Perceptron(tol=1e-3, random_state=0) clf.fit(X, y) coeffs = clf.coef_
The coefficients will return an array for binary classification or a matrix [n_classes, n_features] for a multi-class classification.