0
$\begingroup$

I am performing multi label classification in python using sklearn. Here is the classification report

          precision    recall  f1-score   support

       0       0.77      0.67      0.71      7536
       1       0.76      0.77      0.76      6811
       2       0.84      0.84      0.84      5948
       3       0.78      0.75      0.77      4006
       4       0.96      0.94      0.95      3956
       5       0.70      0.60      0.65      3282
       6       0.85      0.70      0.77      3199
       7       0.74      0.68      0.71      3023
       8       0.64      0.57      0.60      2729
       9       0.92      0.85      0.88      1970
      10       0.75      0.56      0.64      1952
      11       0.98      0.93      0.95      1952
      12       0.88      0.81      0.84      1683
      13       0.79      0.75      0.77      1592
      14       0.75      0.64      0.69      1581
      15       0.75      0.68      0.71      1549
      16       0.84      0.69      0.76      1429
      17       0.70      0.63      0.66      1293
      18       0.63      0.51      0.56      1226
      19       0.71      0.50      0.59       993
      20       0.81      0.54      0.65       941
      21       0.61      0.35      0.45       815
      22       0.77      0.57      0.66       747
      23       0.83      0.57      0.68       752
      24       0.79      0.15      0.25       661
      25       0.73      0.63      0.68       526
      26       0.54      0.31      0.39       459
      27       0.66      0.44      0.53       450
      28       0.70      0.62      0.66       398
      29       0.78      0.09      0.16       229
      30       0.75      0.57      0.65       141
      31       0.75      0.22      0.34       108
      32       0.60      0.11      0.19       106

micro avg 0.79 0.70 0.74 64043
macro avg 0.76 0.58 0.64 64043 weighted avg 0.79 0.70 0.73 64043
samples avg 0.82 0.76 0.76 64043

/usr/local/lib/python3.6/dist-packages/sklearn/metrics/classification.py:1437: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in samples with no predicted labels. 'precision', 'predicted', average, warn_for)

I don't understand the above warning . If the precision and F-score was ill-defined then the precision should be 0 for some class. But for all the values, precision and f1-score are values greater than 0. Why is this warning taking place? Am I missing something here?

Note:- If you want more information you can reply

$\endgroup$

1 Answer 1

1
$\begingroup$

I don't know the framework but it seems that's what happens:

Since this is multi-label classification, it's possible that an instance is assigned no label at all by the model. Apparently at least one of the samples contains only instances which are predicted with no label. This means that there are no instances predicted positive for any label in this sample. This causes an NaN value for the precision of this particular sample, which in turn makes the average precision over all samples undefined (hence the error).

I assume that this is not related to the fact that all the classes have a positive precision and recall, since these values would be calculated over the whole dataset, not sample by sample.

$\endgroup$
6
  • $\begingroup$ But when I predict the model, It should predict the output labels of all the samples in dataset. Right? So you mean that one of the data was predicted with no label. $\endgroup$
    – Pratik.S
    Commented Jun 15, 2019 at 14:33
  • $\begingroup$ Yes but it's multi-label classification: it can predict zero, one, two or more labels. it's normal that it can predict zero labels, making the instance negative for every class. $\endgroup$
    – Erwan
    Commented Jun 15, 2019 at 14:40
  • $\begingroup$ So for this instance in my case a newsarticle (since my dataset is newsarticles), there was no any label prediction. Right? $\endgroup$
    – Pratik.S
    Commented Jun 15, 2019 at 14:45
  • $\begingroup$ Yes, and according to the error message it happens even in a full "sample" but i don't know what is called a "sample" in this framework? if a sample contains only a few instances it's reasonably likely that this would happen (depending on the data of course). $\endgroup$
    – Erwan
    Commented Jun 15, 2019 at 14:49
  • 2
    $\begingroup$ No, it does predict but in the multi label setting it's normal to predict 0 labels by definition. basically the questions you ask the model are: is it label A? is it label B? is it label C? The model answers each question by yes or no, and if the answer happens to be no for each label then there's zero label predicted $\endgroup$
    – Erwan
    Commented Jun 15, 2019 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.