Skip to main content

I trained my data with XGBoost in python with GridSearchCV as follows:

parameters = {'nthread':[6], 
              'objective':['binary:logistic'],
              'learning_rate': [0.01, 0.1],
              'max_depth': [5,8,13],
              'n_estimators': [200,500,1000,3000],
              'seed': [1337]}

xgb_model = xgb.XGBClassifier()

clf = GridSearchCV(xgb_model, parameters, n_jobs=-1, 
                   cv = StratifiedKFold(shuffle=True,n_splits=5), 
                   scoring='accuracy',
                   verbose=2, refit=True)

clf.fit(scaled_X_train.values, y_train)

On the test test I got 0.9 accuracy which is acceptable. However when I predict probabilities with predict_proba I saw that probabilities mostly lie between 0-0.1 and 0.9-1 ranges for 0 and 1 classes respectively.

Since I try to get scores based on the model, those dense probabilities are not so useful.

My question isSo what is the main reason of this dense probability distribution, is? Is this a bad thing,? And how can I improve my workflow so that probabilities to get more widenwider score range?

Thanks in advance!

I trained my data with XGBoost in python with GridSearchCV as follows:

parameters = {'nthread':[6], 
              'objective':['binary:logistic'],
              'learning_rate': [0.01, 0.1],
              'max_depth': [5,8,13],
              'n_estimators': [200,500,1000,3000],
              'seed': [1337]}

xgb_model = xgb.XGBClassifier()

clf = GridSearchCV(xgb_model, parameters, n_jobs=-1, 
                   cv = StratifiedKFold(shuffle=True,n_splits=5), 
                   scoring='accuracy',
                   verbose=2, refit=True)

clf.fit(scaled_X_train.values, y_train)

On the test test I got 0.9 accuracy which is acceptable. However when I predict probabilities with predict_proba I saw that probabilities mostly lie between 0-0.1 and 0.9-1 ranges for 0 and 1 classes respectively.

Since I try to get scores based on the model, those dense probabilities are not so useful.

My question is what is the main reason of this dense probability distribution, is this a bad thing, how can I improve that probabilities to get more widen score range?

Thanks in advance!

I trained my data with XGBoost in python with GridSearchCV as follows:

parameters = {'nthread':[6], 
              'objective':['binary:logistic'],
              'learning_rate': [0.01, 0.1],
              'max_depth': [5,8,13],
              'n_estimators': [200,500,1000,3000],
              'seed': [1337]}

xgb_model = xgb.XGBClassifier()

clf = GridSearchCV(xgb_model, parameters, n_jobs=-1, 
                   cv = StratifiedKFold(shuffle=True,n_splits=5), 
                   scoring='accuracy',
                   verbose=2, refit=True)

clf.fit(scaled_X_train.values, y_train)

On the test test I got 0.9 accuracy which is acceptable. However when I predict probabilities with predict_proba I saw that probabilities mostly lie between 0-0.1 and 0.9-1 ranges for 0 and 1 classes respectively.

Since I try to get scores based on the model, those dense probabilities are not so useful.

So what is the main reason of this dense probability distribution? Is this a bad thing? And how can I improve my workflow so that probabilities get wider score range?

Thanks in advance!

Source Link
Ilker Kurtulus
  • 874
  • 1
  • 5
  • 13

Python XGBoost predict_proba returns very high or low probabilities

I trained my data with XGBoost in python with GridSearchCV as follows:

parameters = {'nthread':[6], 
              'objective':['binary:logistic'],
              'learning_rate': [0.01, 0.1],
              'max_depth': [5,8,13],
              'n_estimators': [200,500,1000,3000],
              'seed': [1337]}

xgb_model = xgb.XGBClassifier()

clf = GridSearchCV(xgb_model, parameters, n_jobs=-1, 
                   cv = StratifiedKFold(shuffle=True,n_splits=5), 
                   scoring='accuracy',
                   verbose=2, refit=True)

clf.fit(scaled_X_train.values, y_train)

On the test test I got 0.9 accuracy which is acceptable. However when I predict probabilities with predict_proba I saw that probabilities mostly lie between 0-0.1 and 0.9-1 ranges for 0 and 1 classes respectively.

Since I try to get scores based on the model, those dense probabilities are not so useful.

My question is what is the main reason of this dense probability distribution, is this a bad thing, how can I improve that probabilities to get more widen score range?

Thanks in advance!