0
$\begingroup$

Does it make sense to calculate the recall for each sample in a multilabel classification problem?

Suppose I have 3 data samples, each having its own true set of labels and predicted set of labels.

Sample output

I want to see the match between the true set of labels and the predicted set of labels. I do not care for the true negatives or false positives in each prediction, so this translates to recall score for me. Programmatically, I would do an AND operation between y_predicted and y_true to get the number of true positives and divide it by the total number of true labels for each sample. (in other words, true positives/(true positives+false negatives))

My question is -

Is calculating recall per sample (not per label), usually done?

Is my thought process correct?

I've seen articles where a single recall is calculated for the whole matrix of y_true and y_predicted or recall is calculated for a single label.

$\endgroup$
1
$\begingroup$

This metric is usually referred to as a sample-based or example-based score and can be applied in multi-label cases (to recall and other scores too). You can find a brief explanation here.

Scikit learn has an implementation for it (see here for recall) when setting average='samples' for recall_score:

'samples': Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your answer! The links were really helpful too. Question - I did look at the sklearn implementation but it says "find their average". Is there a way I can extract just the sample-wise recall without calculating the average? $\endgroup$ – Aishwarya A R Feb 22 at 12:41
  • 1
    $\begingroup$ @AishwaryaAR Unfortunately, as far as I know, there is no implementation of that in scikit learn (unless you apply recall_score on each example but I'd rather implement it myself then). $\endgroup$ – Sammy Feb 22 at 13:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.