8
$\begingroup$

Why is the F-measure usually used for (supervised) classification tasks, whereas the G-measure (or Fowlkes–Mallows index) is generally used for (unsupervised) clustering tasks?

The F-measure is the harmonic mean of the precision and recall.

The G-measure (or Fowlkes–Mallows index) is the geometric mean of the precision and recall.

Below is a plot of the different means.

enter image description here

F1 (harmonic) $= 2\cdot\frac{precision\cdot recall}{precision + recall}$

Geometric $= \sqrt{precision\cdot recall}$

Arithmetic $= \frac{precision + recall}{2}$

The reason I ask is that I need to decide which average to use in a NLG task, where I measured BLEU and ROUGE ( where BLEU is equivalent to precision and ROUGE to recall). How should I calculate the mean of these scores?

$\endgroup$
  • $\begingroup$ Maybe it's just how the defination goes! $\endgroup$ – Aditya Aug 12 '18 at 9:42
  • 1
    $\begingroup$ @Aditya, you are right, it was just a badly formulated questions about definition. I edited it reformulating into something more concrete. $\endgroup$ – BrunoGL Aug 12 '18 at 9:57
3
$\begingroup$

The Fı-score is preferred to simple classification accuracy in order to counter the problem of imbalanced datasets; if the thing you are looking for occurs only rarely anyway then a naive classifier can always say no and appear to be working very well! A variant on Fı is Fß, where

Fß = (1+ß²) × [ (P × R) ÷ ( (ß² × P) + R ) ]

Vary ß to balance precision and recall. As to the why F or G, I believe it to be empirical - you don't say if you are classifying or clustering in your own application?

$\endgroup$
  • 1
    $\begingroup$ Thanks for the answer, but I think you miss understood my question. I don't mean to compare F1 vs. simple accuracy., instead, I mean to compare the Harmonic vs Geometric vs Arithmetic means. I am not doing traditional classification or clustering, I have a NLG task, which is measured in BLEU and ROUGE which could be averaged with one of the means, but I am not sure which to pick. $\endgroup$ – BrunoGL Aug 12 '18 at 14:56
-1
$\begingroup$

If Precision and Recall are similar, F1 is a good single measure to compare different models.

Short and sweet :)

$\endgroup$
  • $\begingroup$ I don't see how you even attempted to answer my question... $\endgroup$ – BrunoGL Feb 23 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.