I am currently building a binary classification model to predict order return rates. I used the GradientBoostingClassifier for training the model and also performed hyperparameter tuning using RandomizedSearchCV. Currently, the metrics for the test data are as follows:

  • Precision: 0.683
  • Recall: 0.78
  • F1 score: 0.72
  • Accuracy: 0.66
  • ROC-AUC: 0.74

The images below show the data distribution of the target label and the confusion matrix. enter image description here enter image description here

I can also see from the feature importance that the brand feature has around 70% importance. If a particular brand has many return entries than sale entries, the prediction also has more returns than sales, which is leading to false positives.

I tried assigning the sample weights to the true labels, undersampling, and SNORT techniques to balance the data, but none of them helped in improving the performance. I also tried using XGBoost, but it didn't make much of a difference.

What can I do to reduce the false positives in this case and improve the performance of the model?

  • $\begingroup$ I would say the main problem is class imbalance. Data are mostly about returns. Either try to get more balanced data, or maybe try oversampling like SMOTE. $\endgroup$
    – Nikos M.
    Apr 12 at 19:42
  • $\begingroup$ i balanced the data and now the false negatives increased and false positives decreased. $\endgroup$ Apr 14 at 11:42

1 Answer 1


Since you did not mention based on which metric your decision will be drawn (e.g. if model A has better F1 but model B is better on ROC-AUC, which would you pick?) nor much detail on your dataset, I can only give some general advice.

  1. Adjust the cost matrix: if false positive is a concern, try increasing the cost for false positives. This is a little-effort option, but you may end up having more false negative instead.

  2. Invest in feature engineering: a model can only learn what you feed in. This requires a lot of work (and creativity), but is the ultimate solution.

  3. Ensembling: train different types of models and ensemble them.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.