Obtain precision at a certain probability value [closed]

With scikit-learn, one is able to compute the precision values as well the predicted probability output. To compute the precision values, the sklearn precision/recall function takes the true target values as well as the predicted target probability (can be target scores or non-thresholded measure of decisions) as an input, however the computed precision array does not have the same length as the the given predicted probability (precision length = n_thresholds + 1).

Is it somehow possible to compute the precision at a given probability output?

• Why did I get a downvote? Whats wrong with the question? Oct 28, 2021 at 20:42
• Very fundamental question IMO, so I voted to leave open. Could be better worded so that someone with the same problem can find the question. Oct 31, 2021 at 13:11
• "To compute the precision values, the predicted probability is needed" is not true; precision is a metric of hard classifications. "...those two parameters do not have the same length" - which two parameters? What lengths do they have? Oct 31, 2021 at 13:12

2 Answers

For everybody else, having a similar problem: I was able to solve this issue by computing the precision as follows:

import numpy as np
# test data
y_true = np.array([0,0,0,0,1,1,1])
predicted_prob = np.array([0.1, 0.2, 0.4, 0.8, 0.9, 0.5, 0.3])

prob_threshold = 0.7 # your arbitrary cut

n_passed = y_true[predicted_prob > prob_threshold].shape[0]
n_passed_true = y_true[predicted_prob > prob_threshold].sum()

precision = n_passed_true / n_passed


With a standard PR curve you will get not only Precision, but also Recall at various prediction probability thresholds.

See here how to do that with scikit.