With scikit-learn, one is able to compute the precision values as well the predicted probability output. To compute the precision values, the sklearn precision/recall function takes the true target values as well as the predicted target probability (can be target scores or non-thresholded measure of decisions) as an input, however the computed precision array does not have the same length as the the given predicted probability (precision length = n_thresholds + 1).

Is it somehow possible to compute the precision at a given probability output?

  • $\begingroup$ Why did I get a downvote? Whats wrong with the question? $\endgroup$
    – gamma
    Oct 28 '21 at 20:42
  • $\begingroup$ Very fundamental question IMO, so I voted to leave open. Could be better worded so that someone with the same problem can find the question. $\endgroup$
    – lcrmorin
    Oct 31 '21 at 13:11
  • $\begingroup$ "To compute the precision values, the predicted probability is needed" is not true; precision is a metric of hard classifications. "...those two parameters do not have the same length" - which two parameters? What lengths do they have? $\endgroup$ Oct 31 '21 at 13:12

For everybody else, having a similar problem: I was able to solve this issue by computing the precision as follows:

import numpy as np
# test data
y_true = np.array([0,0,0,0,1,1,1]) 
predicted_prob = np.array([0.1, 0.2, 0.4, 0.8, 0.9, 0.5, 0.3])

prob_threshold = 0.7 # your arbitrary cut

n_passed = y_true[predicted_prob > prob_threshold].shape[0]
n_passed_true = y_true[predicted_prob > prob_threshold].sum()

precision = n_passed_true / n_passed

With a standard PR curve you will get not only Precision, but also Recall at various prediction probability thresholds.

See here how to do that with scikit.


Not the answer you're looking for? Browse other questions tagged or ask your own question.