0
$\begingroup$

Hi I'm a CS graduate student

I have a question for AI or data experts. I'm writing a paper

My dataset is time-series sensor data and anomaly (positive class) ratio is between 5% and 6%

you can see the picture below. I used classification_report in sklearn library

enter image description here

But, I'm confused as to what value to report in my evaluation section ...

I think it is reasonable to report f1-score with macro avg (0.40)

Is it ok?

Thank you for your explanation !

$\endgroup$
1
0
$\begingroup$

Classic evaluation metrics are heavily affected by skewed data. In your case, since you have imbalanced data, you should definitely avoid those such as accuracy. For example,

Imagine you have a test data with 100 records. 3 of them have Class A and 97 have Class B. I create this model:

prediction = "B"

Simple as that, I always give the second class. I will get accuracy of 97%, but I fail to predict any of the records I really care.

This is important in cases like cancer prediction where early detection is important.

What you could use instead:

  • Sensitivity
  • Specificity
  • G-Mean
  • F-Measure
  • Class weighted metrics where you give more weight to the smaller one
$\endgroup$
1
  • $\begingroup$ Thank you for your explanation ! Um.. Do you think ROC-AUC is also a good measure? $\endgroup$ Jul 7 at 3:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.