F - measure derivation (harmonic mean of precision and recall)

Question

We can define the F - measure as follows:

$F_{\alpha}=\frac{1}{\alpha \frac{1}{P}+(1-\alpha)\frac{1}{R}} $

Now we might be interested in choosen a good $\alpha$. In the article The truth of the F-measure the author states that one can choose the conditions:

$\beta=R/P$, where $\frac{\partial F_{\alpha}}{\partial P}=\frac{\partial F_{\alpha}}{\partial R}$

and then we obtain $\alpha=1/(\beta^2+1)$ and

$F_{\beta}=\frac{(1+\beta^2)PR}{\beta^2 P+R} $

It is said that The motivation behind this condition is that at the point where the gradients of E w.r.t. P and R are equal, the ratio of R against P should be a desired ratio $\beta$. I understand that the condition will garantify that the user is willing to trade an increment in precision for an equal loss in recall. But I do not get why the equality of both partial derivatives correspond to these hypothesis. I would rather understand when one partial derivative equals the other partial derivative multiplied by minus one. Could anyone explain me why the desired condition (condition in words) correspond to this equality (condition in math terms)?

EDIT:

Well, we could do the following:

$\partial F=\frac{\partial F_{\alpha}}{\partial P}\partial P+\frac{\partial F_{\alpha}}{\partial R}\partial R$.

And since we want for $\partial P=-\partial R$ that $\partial F=0$, we obtain easily the condition.

But I have one problem with this: Since $\frac{\partial F_{\alpha}}{\partial P}/\frac{\partial F_{\alpha}}{\partial R}=1$, and the fact that the gradient is perpendicular to each level curve (https://ocw.mit.edu/courses/mathematics/18-02sc-multivariable-calculus-fall-2010/2.-partial-derivatives/part-b-chain-rule-gradient-and-directional-derivatives/session-36-proof/MIT18_02SC_pb_32_comb.pdf) we would have that the level curve must have $m=-1$. Nonetheless, when I calculate the level curve for some constant $c$ I get the result,

$R(P)=\frac{c(1-\alpha ) P}{P-c\alpha}$,

which clearly is not a linear funcion with $m=-1$. What am I missing?

Answer 1

why the equality of both partial derivatives correspond to these hypothesis. I would rather understand when one partial derivative equals the other partial derivative multiplied by minus one.

Your intuition of "trading off" by "subtracting" a value is correct when you speak in terms of $\Delta R$ and $\Delta P$ (as you yourself noticed in the edit), but is not valid if you speak in terms of derivatives.

Think of it this way: a partial derivative with respect to a parameter tells you by how much your metric changes if you improve that variable. By the very design, $F$ will always increase if you improve either $R$ or $P$ - its partial derivatives are positive everywhere.

They are not equal everywhere, however. At some configurations $(R,P)$ you would gain more by improving one parameter rather than the other. Say, if your $R$ is much lower than $P$, your $F$-score would increase more if you improved the $R$ a bit rather than $P$.

In the case of an $F_1$ metric, $R$ and $P$ enter the formula symmetrically and it is kind-of easy to see that whenever $R=P$, either of the components contributes equally to the increase of $F$. In fact, the set of points where $R$ and $P$ are "equal in importance" is exactly the line $R=P$ (which can also be written as $R/P=1$, if you prefer).

Now consider the equation for $F_\beta$:

$$\frac{\text{constant}}{\beta^2\frac{1}{R} + \frac{1}{P}}.$$

Let us see how much it changes in response to increasing $R$ or $P$. As both parameters are in the denominator, the change in value of $F_\beta$ is proportional to the change in the value of its denominator, hence we may simplify a bit and only consider the change in

$$\beta^2\frac{1}{R} + \frac{1}{P}$$

You can now compute that a $dx$ increase in $R$ would decrease the value of the expression above by $\beta^2\frac{dx}{R^2}$. A similar increase in $P$ would decrease the value of the above expression by $\frac{dx}{P^2}$. There is a set of points $(R,P)$ where these effects are exactly equal. These are the points where $$\beta^2\frac{dx}{R^2} = \frac{dx}{P^2} \quad\Rightarrow\quad \frac{R}{P}=\beta$$

When $\beta > 1$ you are effectively putting more marginal importance on recall, in the sense that until your recall is at least $\beta$ times the precision, it is always beneficial to work on it, rather than improving precision.

What am I missing?

You are missing the fact that the condition $\frac{\partial F_\alpha}{\partial P} = \frac{\partial F_\alpha}{\partial R}$ does not hold everywhere, but just on the set of points $R=\beta P = \sqrt{\frac{1-\alpha}{\alpha}}P$. This line intersects the level curve at just a single point.

The line with slope -1 would be the tangent to the level line at that point - the green dashed line on the illustration below.