0
$\begingroup$

I would like know if the accuracy has an impact on not class-imbalanced dataset ? I know that accuracy is sensitive to class-imbalance and also always good to be able to appreciate precision and recall values.

Is it also the case to not class-imbalanced dataset?

$\endgroup$
1
  • $\begingroup$ You mean for balanced sets? It should work fine. $\endgroup$
    – serali
    Sep 28 at 14:32
1
$\begingroup$

Accuracy treats all misclassifications as the same - we only care whether we got the answer right or not, but don't care about what kind of error was made. Even for class-balanced problems, this may not be a desirable feature. If misclassifications come with different "costs", accuracy is not a good measure of the overall utility of your classifier.

Suppose you are designing a medical screening test for a serious but curable disease, where patients with a negative result are given a clean bill of health and patients with a positive test are referred for a highly accurate but more expensive confirmatory test. False positives are scary for the patient but do little harm, as the only cost is the additional test which comes back negative. False negatives are a much bigger problem, as the patient goes untreated and dies a preventable death. Regardless of the actual disease incidence or class balance in the population, evaluating your classifier in terms of accuracy is not terribly useful - you don't really care about accuracy within the actual negative population, all you really care about is correctly identifying the positive cases.

Two classifiers, one of which has 80% sensitivity and 100% specificity, and the other of which has 100% sensitivity and 80% specificity would have identical accuracies in a class-balanced scenario, but would behave very differently in practice and would be suitable for totally different purposes. In any situation where false positives and false negatives have different costs, accuracy will fail to respect the different types of misclassifications and treat them all the same. These problems with accuracy are exacerbated by class imbalance (the metric is always dominated by the majority class, even if it's the class you don't care about), but still exist even with balanced classes (it will still equally weight misclassifications you don't care much about).

$\endgroup$
1
  • $\begingroup$ Thanks @Nuclear for these clear explanations $\endgroup$
    – user979974
    Sep 28 at 19:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.