0
$\begingroup$

I am training a gradient boosting classifier on an imbalanced data but the model is not performing very well. These are the things I have done to improve the model's performance.

  1. Balanced the data with SMOTE
  2. Added more variables
  3. combined features
  4. polynomial feature transformation (this did not improve the model's performance)
  5. cleaned the data
  6. Scaled the data

Except number 4, the other efforts have improved the recall and precision of the model from 34% and 60% respectively to 58% and 51% respectively. Which is good but my aim is to improve the recall and precision to over 70%, is there any other method or technique I can try to get a recall and precision of of over 70?

Improve this question
$\endgroup$
4
  • $\begingroup$ Have you checked out the precision-recall curves for your models? $\endgroup$
    – Dave
    Commented Jan 18 at 13:33
  • $\begingroup$ No, I haven't. I did not check it because I am using only 1 model which is gradient boosting. I read that you can check the precision-recall curve when you are testing more than 1 model. I stand corrected though, the information I read might be wrong. $\endgroup$
    – Toluwalope Owolabi
    Commented Jan 18 at 14:00
  • $\begingroup$ I don't follow the logic of needing multiple models for a PR curve. The typical way to draw a PR curve is described here after you call a method like predict_proba on your model instead of just predict (to use sklearn terminology). While I would advocate for considering the raw probability predictions given by predict_proba, if you must do classification, you're certainly allowed to adjust the threshold. $\endgroup$
    – Dave
    Commented Jan 18 at 16:31
  • $\begingroup$ Ohh, I see, really helpful thank you. I will definitely check the PR curve. $\endgroup$
    – Toluwalope Owolabi
    Commented Jan 19 at 13:37

1 Answer 1

Reset to default
1
$\begingroup$

A little bit late, but you could try to do hyperparameter tuning on your gradient boosting classifier. For example, random search would be an efficient and effective choice (RandomizedSearchCV from sklearn in Python).

If there are missing values in your dataset, you could impute them using multiple imputation or some other type of imputation, too.

You could also try to get access to more data, if possible. I do not specifically know the size of your dataset, but increasing its size could help.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.