1
$\begingroup$

I used classification_report in sklearn library

And, the picture below shows evaluation on my model (anomaly detector)

enter image description here

In general, what are precision, recall, F1 that are reported in papers ?

I think it's reasonable to use precision and recall with macro avg (in my case, 0.5001, 0.7000)

So, when writing a paper, can I report these values?

Otherwise, what are precision, recall, F1 that are reported in papers ?

$\endgroup$
1
$\begingroup$

There's no standard range of values because evaluation scores are never good or bad in absolute, they are relevant with respect to a reference. The standard way to report evaluation scores in a paper is to present them in the context of other methods for the same task:

  • If there are other results about the same task (or a similar task) in the literature, compare to these. If the data is different, then you should ideally apply the state of the art methods to your data as well for an accurate comparison (either directly using the software if it's available or reproducing the method following the description in the paper).
  • If there's really nothing comparable, then a minimal comparison is to show the performance of a baseline classifier. A basic example is a majority class baseline, but depending on the task there can be more relevant heuristic methods.

In a binary classification problem you should only report the F1-score for the positive class, and usually the minority class (in this case anomaly).

$\endgroup$
5
  • $\begingroup$ Thank you for your advice Um.. but, I think it's also reasonable to compute and report average F1 on anomaly class and normal class. right? $\endgroup$ Jul 29 at 11:04
  • 1
    $\begingroup$ @Dae-YoungPark That's not needed, especially for a task like anomaly detection where the goal is specifically to find the positive instances. $\endgroup$
    – Erwan
    Jul 29 at 11:29
  • $\begingroup$ Um.. However, I think it's also important to find detect correct normal point I think it is crucial to consider 'false alarm problem' like anomaly detection , right? $\endgroup$ Jul 30 at 2:22
  • $\begingroup$ @Dae-YoungPark the "false alarm problem" is exactly what the precision score for the class "anomaly" represents, because it takes into account false positive cases. For example a precision value of 90% means 10% of false alarm. The other type of error is false negative, which is taken into account by recall. for example a recall of 90% means that 10% of the gold-standard positive cases are not found by the system. Thus in binary classification precision and recall for the positive class are enough to give the full picture of performance. $\endgroup$
    – Erwan
    Jul 30 at 9:48
  • $\begingroup$ Ok ! Thank you for your explanation ! $\endgroup$ Aug 2 at 8:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.