1
$\begingroup$

I want to recreate catboost.utils.select_threshold(desc) method for CalibratedClassifierCV model.

In Catboost I can select desired fpr value, to return the boundary at which the given FPR value is reached.

My goal is to the same logic after computing fpr, tpr and boundaries from sklearn.metrics.roc_curve

I have the following code

prob_pred = model.predict_proba(X[features_list])[:, 1]
            
fpr, tpr, thresholds = metrics.roc_curve(X['target'], prob_pred)

optimal_idx = np.argmax(tpr - fpr) # here I need to use FPR=0.1
boundary = thresholds[optimal_idx]
 
binary_pred = [1 if i >= boundary else 0 for i in prob_pred]

I guess it should be simple formula but I am not sure how to place 0.1 value here to adjust threshold.

$\endgroup$

1 Answer 1

1
$\begingroup$

I've done my research and testing and it's that simple:

def select_treshold(proba, target, fpr_max = 0.1 ):
    # calculate roc curves
    fpr, tpr, thresholds = roc_curve(target, proba)
    # get the best threshold with fpr <=0.1
    best_treshold = thresholds[fpr <= fpr_max][-1]
    
    return best_treshold
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.