0
$\begingroup$

I am given a dataset to detect fraud. Something similar like this: https://www.kaggle.com/code/imgremlin/4th-place-in-fraud-detection-from-zindi

The issue with SciKit machine learning algorithm is that it optimizes for accuracy, but I want lower its accuracy and optimize for recall so that frauds can be detected more accurately.

The issue with the dataset is that there are much more non-fraud cases, "0", than fraud cases , "1". ~ 10 to 1

Is there a way that I can tweak the SciKit algorithm so that it optimizes for recall?

$\endgroup$
2
  • $\begingroup$ Sklearn has many options what to optimize, see scikit-learn.org/stable/modules/model_evaluation.html. $\endgroup$ Oct 11, 2022 at 6:17
  • 1
    $\begingroup$ You said "SciKit machine learning algorithm [...] optimizes for accuracy", but didn't say which algorithm. You tagged logistic regression, which did not optimize for accuracy. $\endgroup$
    – Ben Reiniger
    Oct 11, 2022 at 11:31

3 Answers 3

2
$\begingroup$

It is always tough when you have extreme unbalance in you classes. If you are able to do so, consider oversampling the minority class or undersampling the dominant class to create better balance.

But beyond that you can use gridsearch to tune the model and target it on a 'recall' metric

sklearn.grid_search.GridSearchCV(clf, param_grid, scoring="recall")

You can read up on how here

$\endgroup$
2
$\begingroup$

Pursuing recall alone is not a well-defined decision rule: just classify everything as 1 and you'll get 100% recall. Actually, anything threshold-sensitive is not a very good optimization target, assuming you mean gridsearch-like routines.

If you have to work with recall, calculate sklearn.metrics.precision_recall_curve() and find a point with good enough recall and decent precision and use the respective threshold to compare your predict_proba() results to. Chances are, your model is already fine, you just need better decision making.

$\endgroup$
1
$\begingroup$

You can use the balanced accuracy metric for such cases. It is defined as the average of recall obtained on each class.

from sklearn.metrics import balanced_accuracy_score
y_true = [0, 1, 0, 0, 1, 0]
y_pred = [0, 1, 0, 0, 0, 1]
balanced_accuracy_score(y_true, y_pred)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.