-1
$\begingroup$

With scikit-learn, one is able to compute the precision values as well the predicted probability output. To compute the precision values, the sklearn precision/recall function takes the true target values as well as the predicted target probability (can be target scores or non-thresholded measure of decisions) as an input, however the computed precision array does not have the same length as the the given predicted probability (precision length = n_thresholds + 1).

Is it somehow possible to compute the precision at a given probability output?

Improve this question
$\endgroup$
3
  • $\begingroup$ Why did I get a downvote? Whats wrong with the question? $\endgroup$
    – gamma
    Oct 28 '21 at 20:42
  • $\begingroup$ Very fundamental question IMO, so I voted to leave open. Could be better worded so that someone with the same problem can find the question. $\endgroup$
    – lcrmorin
    Oct 31 '21 at 13:11
  • $\begingroup$ "To compute the precision values, the predicted probability is needed" is not true; precision is a metric of hard classifications. "...those two parameters do not have the same length" - which two parameters? What lengths do they have? $\endgroup$
    – Ben Reiniger
    Oct 31 '21 at 13:12
1
$\begingroup$

For everybody else, having a similar problem: I was able to solve this issue by computing the precision as follows:

import numpy as np
# test data
y_true = np.array([0,0,0,0,1,1,1]) 
predicted_prob = np.array([0.1, 0.2, 0.4, 0.8, 0.9, 0.5, 0.3])

prob_threshold = 0.7 # your arbitrary cut

n_passed = y_true[predicted_prob > prob_threshold].shape[0]
n_passed_true = y_true[predicted_prob > prob_threshold].sum()

precision = n_passed_true / n_passed
Improve this answer
$\endgroup$
0
$\begingroup$

With a standard PR curve you will get not only Precision, but also Recall at various prediction probability thresholds.

See here how to do that with scikit.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.