Micro and macro averaging are metrics for multi-class classification. However, for binary classification when data are imbalanced, it seems that micro and macro precision have different results. My question is that: does it make sense to use micro and macro precision in binary classification problems when classes are imbalanced?
does it make sense to use micro and macro precision in binary classification problems when classes are imbalanced?
In general micro- and macro-average performance are not relevant in binary classification, whether the classes are balanced or not. Their value can be especially misleading if there is a strong imbalance, because it takes into account both the minority class (harder for the classifier) and the majority class (easier):
- By definition micro-average gives more weight to the majority class, so the micro-average performance can be high even if the classifier does a terrible job at distinguishing the two classes.
- The macro-average is not biased towards any of the two classes, still it's uselessly complex, it makes it harder to understand what's going on than simple performance on the positive class, which is normally the minority one (because that's the challenging one).
Of course there can be cases where it makes sense not to follow this standard evaluation setting, it's always a matter of choosing the appropriate way to evaluate a particular task.
The below example illustrates why micro- and macro-average are confusing in a standard case of imbalance:
true A true B predicted A 90 9 predicted B 0 1
- For A: precision = 0.91, recall = 1, f1-score = 0.95
- For B: precision = 1, recall = 0.1, f1-score = 0.18
- micro-average: precision = 0.91, recall = 0.91, f1-score = 0.91
- macro-average: precision = 0.95, recall = 0.55, f1-score = 0.70
Assuming we don't know anything else than the selected performance measure, this classifier:
- performs almost perfectly according to the performance of the majority class A,
- performs very well according to micro-average,
- performs decently according to macro-average,
- performs terribly according to the performance of the minority class B.
Looking at the confusion table, it's clear that the classifier doesn't do a good job at distinguishing the two classes. So the most "honest" performance measure is the last one, i.e. the non-averaged performance on the minority class.