1
$\begingroup$

I have trained an XGBoost model (for a binary classification problem) and I have tested two scenarios:

Scenario 1 - No Monotonic Constrained applied In this case I get a Gini on the training sample of 81.1 and a Gini on the Validation sample of 76.5, throwing a red flag for overfitting. I have taken a look at the SHAP dependence plot for one char in Scenario 1 and it looks like this: enter image description here

Scenario 2 - Monotonic Constrained applied In this case I get a Gini on the training sample of 69.0 and a Gini on the Validation sample of 68.7, which looks like the model has been generalized well. I have taken a look at the SHAP dependence plot for the same char as above in Scenario 2 and it looks like this: enter image description here

I was wondering if the application of monotonic constraints to the predictive features actually prevents the XGBoost Classifier to capture the non-linear relationship between the predictive features and the binary target? Intuitively I'd say yes, but when I look at the SHAP plots I don't think so anymore. Can anyone clarify this for me please?

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.