4
$\begingroup$

I use the MLPClassifier from scikit learn. I have about 20 features. Is there a scikit method to get the feature importance? I found

clf.feature_importances_

but it seems that it only exists for decision trees.

$\endgroup$
6
$\begingroup$

The short answer is that there is not a method in scikit-learn to obtain MLP feature importance - you're coming up against the classic problem of interpreting how model weights contribute towards classification decisions.

However, there are a couple of great python libraries out there that aim to address this problem - LIME, ELI5 and Yellowbrick:

LIME (or Local Interpretable Model-agnostic Explanations, blog post here, arxiv paper here) which "explains the predictions of any classifier in an interpretable and faithful manner, by learning an interpretable model locally around the prediction", or basically to explain model outputs by approximating the results of a classifier with a model that can be explained. It's on pypi and they have examples on their github page - very straightforward to get started.

ELI5 (or Explain Like I'm 5) is a "Python library which allows to visualize and debug various Machine Learning models using unified API". Although not all scikit-learn integration is present when using ELI5 on an MLP, Permutation Importance is a method that "...provides a way to compute feature importances for any black-box estimator by measuring how score decreases when a feature is not available", which saves you from trying to implement it yourself.

Yellowbrick is "a suite of visual diagnostic tools called “Visualizers” that extend the Scikit-Learn API to allow human steering of the model selection process" and it's designed to feel familiar to scikit-learn users. Compared to the other two libraries here it doesn't offer as much in the way for diagnosing feature importance, but it's still worth mentioning for more general use cases.

| improve this answer | |
$\endgroup$
0
$\begingroup$

The Multi-Layer Perceptron does not have an intrinsic feature importance, such as Decision Trees and Random Forests do. Neural Networks rely on complex co-adaptations of weights during the training phase instead of measuring and comparing quality of splits.

A simpler approach for getting feature importance within Scikit can be easily achieved with the Perceptron, which is a 1-layer-only Neural Network.

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import Perceptron
X, y = load_breast_cancer(return_X_y=True)
clf = Perceptron(tol=1e-3, random_state=0)
clf.fit(X, y)

coeffs = clf.coef_

The coefficients will return an array for binary classification or a matrix [n_classes, n_features] for a multi-class classification.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.