Choose threshold to get 90% precision classifier - ML Binary Classification problem

I have chosen threshold value with below code to get 90% precision classifier

from sklearn.model_selection import cross_val_predict
y_train_pred = cross_val_predict(sgd_clf, X_train, y_train, cv=3)

z_scores = cross_val_predict(sgd_clf, X_train, y_train, method='decision_function')

from sklearn.metrics import precision_recall_curve
precisions, recalls, thresholds = precision_recall_curve(y_train_pred, z_scores)

threshold_90_precision = thresholds[np.argmax(precisions >= 0.9)]

y_train_pred_90percent_precision = (z_scores >= threshold_90_precision)
print(precision_score(y_train, y_train_pred_90percent_precision))

I'm expecting precision_score to be 90% but it returned 95%. Is this expected? Anything incorrect with my code? If it's expected, can you please explain the reason?

• Isn’t 95% precision better than 90%?
– Dave
Jun 28 '20 at 6:37
• 95% is better than 90% but I have trained my classifier to be 90% classifier, and expect precision score as 90% Jun 28 '20 at 18:22

threshold_90_precision = thresholds[np.argmax(precisions >= 0.9)]

Above snippet is not doing what you are expecting it to do.

Try these changes

precisions[precisions < 0.9] = 1
threshold_90_precision = thresholds[np.argmin(precisions)]

Also, I am not sure whether you are calculating the accuracy properly since z_scores is decision function, not Class.

This is a working example using method='predict_proba' for 40%, you may change to 90%

model.fit(x_train, y_train)

from sklearn.model_selection import cross_val_predict
y_train_pred = cross_val_predict(model, x_train, y_train, cv=3)

z_scores = cross_val_predict(model, x_train, y_train, method='predict_proba')

from sklearn.metrics import precision_recall_curve
precisions, recalls, thresholds = precision_recall_curve(y_train_pred, z_scores[:,0])

import numpy as np
precisions[precisions < 0.4] = 1
threshold_90_precision = thresholds[np.argmin(precisions)]

y_train_pred_90percent_precision = z_scores[:,0] >= threshold_90_precision
from sklearn.metrics import precision_score
print(precision_score(y_train, y_train_pred_90percent_precision))