0
$\begingroup$

I have an xgboost model that predicts the likelihood of a sales lead to close (actually to turn into an "opportunity" which is one step before the close but that's beside the point). The dataset is very imbalanced with the minority class only making up around 15% of the train/test sets. I am getting acceptable f-1 scores (macro avg is 81% which will suffice) but because there are so few leads that close relative to the total leads in our system when I run predictions for our entire database of leads only ~2% of the leads end up getting a probability to close of 50% or greater. The sales leadership will not respond favorably if I go back to them with such a small number regardless of it it's the most accurate portrayal of reality or not. Does anyone have any ideas of what to try to make the model more "liberal" (giving higher probabilities) toward the the minority class? Thanks

$\endgroup$

1 Answer 1

2
$\begingroup$

If you have a highly imbalanced dataset, a different model architecture or loss function may be better suited to the task.

Although, if you wanna go with xgboost I think you can make your model more "liberal" with its classifications by adjusting the decision threshold for classification. By default, xgboost models use a threshold of 0.5 for binary classification, but you can adjust this threshold to be more or less conservative with your predictions. Why don't you try using a threshold of 0.4 or 0.3, this might make the model more liberal with its classifications.

You can also try oversampling the minority class in your training data to give the model more examples to learn from, in this way the model can learn to better distinguish between the minority and majority classes, and may make the model more confident in its predictions of the minority class. Use SMOTE, this might help you.

$\endgroup$
1
  • $\begingroup$ Thank you @Vic! This is great! $\endgroup$ Dec 8, 2022 at 7:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.