1
$\begingroup$

When should I use The Area Under an ROC Curve (AUC) or the Confusion Matrix for classifier evaluation? The clasifier evaluation is for example the prediction of customers for possible future sales.

$\endgroup$
2
$\begingroup$

A confusion matrix can be used to measure the performance of a particular classifier with a fixed threshold. Given a set of input cases, the classifier scores each one, and score above the threshold are labelled Class 1 and scores below the threshold are labelled Class 2.

The ROC curve, on the other hand, examines the performance of a classifier without fixing the threshold. Given a set of input cases, the classifier scores each one. The ROC curve is then generated by testing every possible threshold and plotting each result as a point on the curve.

The ROC curve is useful when you want to test your classifier over a range of sensitivities/specificities. This may or may not be a desirable thing to do. Perhaps you want very high sensitivity and don't care much about specificity - in this case, the AUC metric will be less desirable, because it will take into account thresholds with high specificity. The confusion matrix, on the other hand, could be generated with a fixed threshold known to yield high sensitivity, and would only be evaluated for that particular threshold.

A confusion matrix evaluates one particular classifier with a fixed threshold, while the AUC evaluates that classifier over all possible thresholds.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.