0
$\begingroup$

I had input some prediction scores from a learner into the roc_auc_score() function in sklearn. I wasn't sure if I had applied a sigmoid to turn the predictions into probabilities, so I looked at the AUC score before and after applying the sigmoid function to the output of my learner. Regardless of sigmoid or not, the AUC was exactly the same. I was curious about this so I tried other things like multiplication by arbitrary numbers and applying arbitrary log or exp functions and the score was still the same.

Assuming I haven't made some other error somewhere, why is the sklearn function for ROC AUC able to work on any scale of scores?

$\endgroup$
1
$\begingroup$

The documentation says

Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by "decision_function" on some classifiers).

(My emphasis)

This is possible because the implementation only requires that the y_true can be sorted according to the y_score.

The false positive rate that is returned is given as follows:

A count of false positives, at index i being the number of negative samples assigned a score >= thresholds[i]. The total number of negative samples is equal to fps[-1] (thus true negatives are given by fps[-1] - fps).

The false positives can be calculated with only taking into account the order of the predicted scores/thresholds.

Multiplying the scores by a scalar has no effect on the order of the values, thus the resulting score remains the same!

See the sklearn GitHub page, the relevant code is under the definition of _binary_clf_curve.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.