I have two loss functions $\mathcal{L}_1$ and $\mathcal{L}_2$ to train my model. The model is predominantly a classification model. Both $\mathcal{L}_1$ and $\mathcal{L}_2$ takes are two variants of the focal losses. $\mathcal{L}_1$ and $\mathcal{L}_2$ takes as input the same class probabilities and a hyper-parameter $\gamma$. The formulation of $\mathcal{L}_1$ and $\mathcal{L}_2$ are distinct. It is possible to mathematically show that

\begin{equation} \mathcal{L}_1\geq\mathcal{L}_2. \end{equation}

Based on this information can we comment on the useful-ness of $\mathcal{L}_1$ and $\mathcal{L}_2$? i.e. which of $\mathcal{L}_1$ and $\mathcal{L}_2$ is more useful to train the model?


1 Answer 1


A loss being less than another loss has nothing to do with its usefulness. You can simply subtract a constant from a loss to obtain another loss (i.e. $\mathcal{L}_2 = \mathcal{L}_1 - \alpha$) that is always less than the former but nonetheless, it is exactly as useful as the other one was.

Also, a loss is as good as the model it can train so you normally judge the usefulness of the loss by comparing the performance of the models themselves.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.