0
$\begingroup$

I am currently building a binary classification model to predict order return rates. I used the GradientBoostingClassifier for training the model and also performed hyperparameter tuning using RandomizedSearchCV. Currently, the metrics for the test data are as follows:

  • Precision: 0.683
  • Recall: 0.78
  • F1 score: 0.72
  • Accuracy: 0.66
  • ROC-AUC: 0.74

The images below show the data distribution of the target label and the confusion matrix. enter image description here enter image description here

I can also see from the feature importance that the brand feature has around 70% importance. If a particular brand has many return entries than sale entries, the prediction also has more returns than sales, which is leading to false positives.

I tried assigning the sample weights to the true labels, undersampling, and SNORT techniques to balance the data, but none of them helped in improving the performance. I also tried using XGBoost, but it didn't make much of a difference.

What can I do to reduce the false positives in this case and improve the performance of the model?

$\endgroup$
2
  • $\begingroup$ I would say the main problem is class imbalance. Data are mostly about returns. Either try to get more balanced data, or maybe try oversampling like SMOTE. $\endgroup$
    – Nikos M.
    Apr 12, 2023 at 19:42
  • $\begingroup$ i balanced the data and now the false negatives increased and false positives decreased. $\endgroup$ Apr 14, 2023 at 11:42

1 Answer 1

1
$\begingroup$

Since you did not mention based on which metric your decision will be drawn (e.g. if model A has better F1 but model B is better on ROC-AUC, which would you pick?) nor much detail on your dataset, I can only give some general advice.

  1. Adjust the cost matrix: if false positive is a concern, try increasing the cost for false positives. This is a little-effort option, but you may end up having more false negative instead.

  2. Invest in feature engineering: a model can only learn what you feed in. This requires a lot of work (and creativity), but is the ultimate solution.

  3. Ensembling: train different types of models and ensemble them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.