I have read in a book where it was described that Loss and Accuracy are only conditionally suitable for a statement strength. Other metrics are used like Precision, Recall, ... .

Does anyone know a good scientific paper that describes in more detail why you should not use Loss and Accuracy, but rather focus on other metrics for example Precision and Recall, ... .

So far I have not been able to find any scientific work on the fact that loss and accuracy are not particularly good and should therefore be used on other metrics.

  • 1
    $\begingroup$ which book did you read? In fact no metric is a catch-all metric, that is why different metrics exist, there is not one super-metric. So it depends on the algorithm and what one wants to capture $\endgroup$
    – Nikos M.
    Jan 2 at 11:08
  • $\begingroup$ Please, consider upvoting the answer if you found it useful, and marking it as correct if deemed so. Alternatively, please considering describing what the answer is lacking or why you think it is not correct, so that it can be improved. $\endgroup$
    – noe
    Jan 9 at 16:03

First, the term "loss" does not define a specific computation, but just the measure you are trying to optimize. Depending on the loss you choose, it may or may not be appropriate for the problem you are addressing.

About accuracy, you do not need a scientific work to understand why it may not be appropriate: imagine a binary classification problem where 99% of the data belongs to class A and 1% of the data belongs to class B. A classifier that classifies the candidate data ALWAYS as class A, would have an accuracy of 0.99, but it clearly is not a good classifier, as it blindly chooses class A regardless of the input data.

The kind of data we described, with a 99%-1% class distribution, is said to be "imbalanced". This kind of data is the typical example where accuracy is not a good choice to evaluate your model. Instead, other measures like the RoC curve or the AUC are better for imbalanced data.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.