Skip to main content
1 of 2
Ilker Kurtulus
  • 874
  • 1
  • 5
  • 13

Python XGBoost predict_proba returns very high or low probabilities

I trained my data with XGBoost in python with GridSearchCV as follows:

parameters = {'nthread':[6], 
              'objective':['binary:logistic'],
              'learning_rate': [0.01, 0.1],
              'max_depth': [5,8,13],
              'n_estimators': [200,500,1000,3000],
              'seed': [1337]}

xgb_model = xgb.XGBClassifier()

clf = GridSearchCV(xgb_model, parameters, n_jobs=-1, 
                   cv = StratifiedKFold(shuffle=True,n_splits=5), 
                   scoring='accuracy',
                   verbose=2, refit=True)

clf.fit(scaled_X_train.values, y_train)

On the test test I got 0.9 accuracy which is acceptable. However when I predict probabilities with predict_proba I saw that probabilities mostly lie between 0-0.1 and 0.9-1 ranges for 0 and 1 classes respectively.

Since I try to get scores based on the model, those dense probabilities are not so useful.

My question is what is the main reason of this dense probability distribution, is this a bad thing, how can I improve that probabilities to get more widen score range?

Thanks in advance!

Ilker Kurtulus
  • 874
  • 1
  • 5
  • 13