# Python library that can compute the confusion matrix for multi-label classification

I'm looking for a Python library that can compute the confusion matrix for multi-label classification.

FYI:

• What did you end up doing? – Morteza Shahriari Nia Oct 13 '16 at 22:27
• @MortezaShahriariNia I stayed monolabeled. – Franck Dernoncourt Oct 13 '16 at 22:33
• what would a theoretical multilabel confusion matrix look like? I do not think it applies, does it? – user798719 May 25 '17 at 4:05
• very funny that all 3 answers to this question are of such remarkably low quality. – Monica Heddneck Aug 11 '17 at 6:41
• Not aware of any packages, but you could perhaps consider all possible multi-label combination as a separate class and use some of the already-available packages for multi-class. Then, from that confusion, build your multi-label matrix – Valentin Calomme Jan 30 '18 at 22:20

Also take a look at scikit-multilearn. It is a very good library that extends sklearn for multi-label learning. However, I'm not sure how the confusion matrix works for multi-label problems...

This guy claims he has solved it.

There are many different parameters which can evaluate the performance of your method by comparing the real and predicted labels. I suggest PyCM module which can give a vast variety these parameters which are suitable for multi-class classification.

Although this question is old, I am writing this answer for new audience.
scikit-learn now supports confusion matrix for multi-label classification.

https://scikit-learn.org/stable/modules/generated/sklearn.metrics.multilabel_confusion_matrix.html

Try mlxtend. Here's an example of multi-class case.

• how about multi-label multi-class? Is it supported? – Franck Dernoncourt Feb 15 '17 at 17:49

Sklearn has a method for it using which you can compute confusion matrix for multi class.

from sklearn import cross_validation
confusion_matrix(original, Predicted)


Scikit-learn does support multi-label confusion matrix. See the links below for documentation and user guide:

http://scikit-learn.org/stable/modules/model_evaluation.html#confusion-matrix

• Thank you, that's why I am looking for an alternative option :-) – Franck Dernoncourt Feb 15 '17 at 17:48

Look at sed_eval library. It is developed for evaluating event detection in audio which is a multi-label problem (as in each audio, multiple events exist). They have many evaluation options, which might fit to your needs. You can get the true-positive rate, ... and from there computing the confusion matrix is not that hard.