XGBoost probability distribution tending towards the extreme

I am using an XGBoost classifier to make risk predictions, and I see that even if it has very good binary classification results, the probability outputs are mainly under $$0.05$$ or over $$0.95$$ (like 60% of them).

I have tried calibration methods (from the sklearn API) but it reduces the problem only slightly.

My dataset has 1800 training points and I test it on around 500 datapoints. It is quite balanced. I also use Bayesian optimisation to tune the hyperparameters of the model. There are 19 features for my model.

Does anyone know a solution to get more regularly distributed probabilities? Does the problem dwell in the fact I have too few datapoints? Should I set my hyperparameters differently? Do I have too many/few features?

• Where is the problem? Having big jumps in probabilities is, IMHO, related to a good discrimination of your algorithm : think about the logit function. Well, maybe that I misunderstand your question – Michael Hooreman Oct 5 '19 at 14:15
• The thing is that sometimes the values are not so extreme, and sometimes they are. On what does the phenomenon depend? – Ismalyt Oct 15 '19 at 17:45