I wanted to know if it makes sense to make 2 ROC curves for each of the 2 classes?
I am doing a binary classification problem but AUC is good at 82%.
But the F Score of the class labelled 1 is very poor (around 0.4).
So the AUC is very good but the F Score is very poor.
What does the ROC curve actually mean in this case?
I always thought, better the AUC score, better the classifier.
But in my case it is not able to capture the class 1 properly.
What should be done in such cases?
Any better metric to evaluate how good the classifier is?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
It does not make sense to make two separate AUC / AUROC curves for each class, these are aggregating functions, same as F1-score. If AUC is high but F-score is not then it's possible you have an imbalanced data set (classes are not equally represented in the data), since the AUC will not be able to measure this, while the F-score does. So, if you do have an imbalanced data set you should use something other then AU(RO)C, such as the F-score.
$\begingroup$
$\endgroup$
Agree with user2974951. It is very likely you have imbalanced data on the 2 classes. So check the number of samples for each classes, give different weights on the errors for each as well. For example, if you have 10 samples for class A and 1000 samples for class B, give 1 as the cost for class B errors and sqrt(100) as the cost for class A error
