4
$\begingroup$

The F beta formula according to the wikipedia is "The weighted harmonic mean of precision and recall". I can not understand why in the left part of equation there is beta and in the right one is beta^2: enter image description here

To my mind if I claim that Precision is 5 times important than Recall: F beta = (1+beta)/(beta/P+1/R)=(1+beta)PR/(beta*R+P), where beta=0.2.

Is this right?

$\endgroup$

1 Answer 1

3
$\begingroup$

That's a great question, because on its face it seems like the weight should be $\beta$ alone, and, it should be in front of recall. The answer is in the text from which that reference is taken, on page 133: http://www.dcs.gla.ac.uk/Keith/pdf/Chapter7.pdf

The definition is designed to make the metric indifferent to a change in precision or recall when $P/R = \beta$. That is, $F_\beta$ increases by the same amount when either precision or recall increases, at the point where precision is already $\beta$ times bigger than recall.

The definition does indeed weight recall more highly as you can verify. Honestly on re-reading the text above, I was confused, because I don't see how it makes sense to think of "equilibrium" as the point where precision is much bigger, if recall matters more.

I plugged in the formula to Wolfram Alpha, and: enter image description here

Hm. These are only equal if $R/P = \beta$! I think the paper may have misstated this then, or else I've really missed something. It's a formula whose value changes at the same rate with respect to precision or recall, when recall is already $\beta$ times larger, and in that sense it corresponds to treating recall as $\beta$ time more important.

$\endgroup$
7
  • $\begingroup$ I have found two papers about F_beta: 1.cs.odu.edu/~mukka/cs795sum10dm/Lecturenotes/Day3/… 2. qwone.com/~jason/writing/fmeasure.pdf So from the the first link I figured out why in the F_beta formula it is beta^2, also that F_beta, technically speaking, presents the harmonic mean with weights of beta^2 for Recall and 1 for Precision. BUT in the second paper I see a bit different definition of F_beta (formula 5) This confused me. Which one is appropriate and which one to use? $\endgroup$ Aug 17, 2017 at 8:37
  • $\begingroup$ Second one is the same, with a = 1/(1+B^2) $\endgroup$
    – Sean Owen
    Aug 18, 2017 at 6:23
  • $\begingroup$ F_a = (a+1)RP / (R+aP), where a = 1 / (1+ Beta^2)?? $\endgroup$ Aug 18, 2017 at 10:00
  • $\begingroup$ Oops, mixing up the two papers. In the second, imagine a = B^2. In the first paper there's a similar derivation with alpha, but there a = 1/(1+B^2) $\endgroup$
    – Sean Owen
    Aug 18, 2017 at 17:54
  • $\begingroup$ Right. And now we have F_b^2 = (b^2+1)RP / (R+b^2P), which says that R b^2 times important than P. The left part of the formula is my main concern, because according to the first paper the left part should be F_b not F_b^2. It is kind of ambiguity to me. $\endgroup$ Aug 18, 2017 at 18:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.