# Is it possible to make F1_Score differentiable and use it directly as a Loss function?

One of the metrics that is widely used in binary classification is the F1 score:

$$F_1 = 2\cdot \frac{recall \cdot precision}{recall+precision}$$

The problem of the F1-score is that it is not differentiable and so we cannot use it as a loss function to compute gradients and update the weights when training the model. The F1-score needs binary predictions (0/1) to be measured.

I am seeing it a lot. Let's say I am using per example a Linear regression or a gradient boosting.

Is there any way that it can be minimized directly?

Yes there is, let's take $$F_1$$ score base definition, with : $$F_1 = 2 \times \frac{precision \times recall} {precision + recall} \\ F_1 = \frac{2 \times TP} {2 \times TP + FP + FN}$$ And this is the same as the Sørensen-Dice coefficient, also known as Dice coefficient or Bray-Curtis distance. This is a statistical indicator that measures the similarity of two samples :

$$Dice(X,Y) = \frac{2|X \cap Y|}{|X| + |Y|}$$

Concerning the implementation of this loss, we can approximate $$|X \cap Y|$$ as the sum of the matrix obtained using Hadamard product ($$\odot$$, also known as the element-wise product) between the ground truth ($$y$$) and the prediction ($$\hat{y}$$). We can then define $$L_{Dice}$$ as follows:

\begin{align*} L_{Dice} &= 1 - Dice \\ L_{Dice}\left(y, \hat{y}\right) &= 1 - \frac{ 2\sum y \odot \hat{y}} {\sum y + \sum \hat{y}} \end{align*}

You will often find this loss in the context of segmentation problems, as well as others quite close, such as the Jaccard index (IoU).

• Isn't $\sum y \odot \hat{y}$ the same as the dot product of $y$ and $\hat{y}$, since both are one-dimensional vectors? Jun 13, 2021 at 18:36
• I like it. What are the gradient and hessian? We take derivative w.r.t. $\hat{y}$, right? Jul 14, 2021 at 21:33
• I'm trying to implement this as an objective for XGBClassifier. xgboost.readthedocs.io/en/latest/python/python_api.html Jul 14, 2021 at 21:43
• If I consider each $y_i$, $\hat{y_i}$ to be independent, then my loss becomes $L_i = 1 - \frac{2 y_i \hat{y_i}}{y_i + \hat{y_i}}$. If I use a sigmoid for $\hat{y_i}$ to let the model predict a logit, which we squash to a probability, then $L_i = 1 - \frac{2 y_i \sigma(\hat{y_i})}{y_i + \sigma(\hat{y_i})}$. Then first and second derivative w.r.t. $\hat{y_i}$ can be found: wolframalpha.com/input/…, wolframalpha.com/input/… Jul 16, 2021 at 22:01
• I get better results using the cross entropy loss, though: $L(y,\hat{y}) = -\frac{1}{N} \sum_{i} y_i log(\sigma(\hat{y_i})) + (1-y_i) log (1-\sigma(\hat{y_i}))$, where $\sigma(x) = \frac{1}{1 + e^{-x}}$ -> $L_i = - (y_i log(\sigma(\hat{y_i})) - (1-y_i) log(1-\sigma(\hat{y_i})))$ -> $\frac{\partial}{\partial \hat{y_i}} L_i = -y_i + \sigma(\hat{y_i})$ and $\frac{\partial^2}{\partial \hat{y_i}^2} L_i = \sigma(\hat{y_i})(1 - \sigma(\hat{y_i}))$ Jul 16, 2021 at 22:06

Following on Thomas, on the relation between the Bray-Curtis distance and the F1 score and the calculation of the first and second-order derivatives: If one defines the Bray Curtis distance between vector X and Vector Y as: $$\sum |X_i-Y_i| \over {\sum (X_i+Y_i)}$$, than the first derivative to $$x$$ is $$d \over (dx)$$ $$|x - y| \over {(x + y)}$$ = $$2y(x - y) \over{\big((x + y)^2(|x - y|)\big)}$$ and the second derivative is $${d^2 \over{dx^2}} {|x - y| \over(x + y)} = {-4y*(x - y) \over\big((x + y)^3*(|x - y|)\big)}$$

• How can I use this? As $x$, or as I might call it $\hat{y}$ approaches $y$, the gradient and hessian explode, because the denominator goes to 0. Seems like the optimum must be an unstable one. Jul 16, 2021 at 22:34