I am currently building a binary classification model to predict order return rates. I used the GradientBoostingClassifier for training the model and also performed hyperparameter tuning using RandomizedSearchCV. Currently, the metrics for the test data are as follows:
- Precision: 0.683
- Recall: 0.78
- F1 score: 0.72
- Accuracy: 0.66
- ROC-AUC: 0.74
I can also see from the feature importance that the brand feature has around 70% importance. If a particular brand has many return entries than sale entries, the prediction also has more returns than sales, which is leading to false positives.
I tried assigning the sample weights to the true labels, undersampling, and SNORT techniques to balance the data, but none of them helped in improving the performance. I also tried using XGBoost, but it didn't make much of a difference.
What can I do to reduce the false positives in this case and improve the performance of the model?