Why is sklearn.metrics.roc_auc_score() seemingly able to accept scores on any scale?

Question

I had input some prediction scores from a learner into the roc_auc_score() function in sklearn. I wasn't sure if I had applied a sigmoid to turn the predictions into probabilities, so I looked at the AUC score before and after applying the sigmoid function to the output of my learner. Regardless of sigmoid or not, the AUC was exactly the same. I was curious about this so I tried other things like multiplication by arbitrary numbers and applying arbitrary log or exp functions and the score was still the same.

Assuming I haven't made some other error somewhere, why is the sklearn function for ROC AUC able to work on any scale of scores?

Answer 1

The documentation says

Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by "decision_function" on some classifiers).

(My emphasis)

This is possible because the implementation only requires that the y_true can be sorted according to the y_score.

The false positive rate that is returned is given as follows:

A count of false positives, at index i being the number of negative samples assigned a score >= thresholds[i]. The total number of negative samples is equal to fps[-1] (thus true negatives are given by fps[-1] - fps).

The false positives can be calculated with only taking into account the order of the predicted scores/thresholds.

Multiplying the scores by a scalar has no effect on the order of the values, thus the resulting score remains the same!

See the sklearn GitHub page, the relevant code is under the definition of _binary_clf_curve.