3
$\begingroup$

I'm running xgboost and I fix the seed number
but the result is different for each time when I rerun the xgboost with same dataset

xgb = XGBClassifier(
     bagging_fraction= 0.8,
     boosting= 'gbdt',
    colsample_bytree= 0.7,
    feature_fraction= 0.9,
    learning_rate= 0.05,
    max_bin= 32,
    max_depth= 10,
    min_child_weight= 11,
    missing= -999,
    n_estimators= 400,
    nthread= 4,
    num_leaves= 100,
    predictor= 'gpu_predictor',
    seed= 1000,
    silent= 1,
    subsample= 0.8,
    tree_method= 'gpu_hist',
    verbose= True
)

This is my parameter and

# Our data is already scaled we should split our training and test sets
from sklearn.model_selection import train_test_split

# This is explicitly used for undersampling.
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
kf = StratifiedKFold(n_splits=5, shuffle=True)
# partially based on https://www.kaggle.com/c0conuts/xgb-k-folds-fastai-pca
predicts = []
for train_index, test_index in kf.split(X_train, y_train):
    print("###")
    X_train, X_val = X.iloc[train_index], X.iloc[test_index]
    y_train, y_val = y.iloc[train_index], y.iloc[test_index]
    xgb.fit(X_train, y_train, eval_set=[(X_val, y_val)], 
            early_stopping_rounds=30)
    predicts.append(xgb.predict(X_test))
$\endgroup$
1
  • $\begingroup$ I ran into the same problem as the OP described above. I got different results running xgboost() even when setting set.seed(12345) in R. I kept all the other parameters the same (nrounds, max_depth, eta, alpha, booster='dart', subsample=0.8). Does anyone know how to overcome this randomness issue? $\endgroup$
    – tle
    Commented Mar 29, 2023 at 20:22

2 Answers 2

3
$\begingroup$

You are missing some seeds there:

  • Numpy
  • Pandas
  • StratifiedKFold

I will say that StratifiedKFold(shuffle=True) without a seed can make your results change.

$\endgroup$
0
$\begingroup$

Make sure you have both a fixed random state in the train_test_split and in the initialization of the classifier. Your random_state in the train_test_split is there, however you missed the random_state in the initialization of the XGBClassifier. Add it there and it should be working as expected.

xgb = XGBClassifier(
 bagging_fraction= 0.8,
 boosting= 'gbdt',
colsample_bytree= 0.7,
feature_fraction= 0.9,
learning_rate= 0.05,
max_bin= 32,
max_depth= 10,
min_child_weight= 11,
missing= -999,
n_estimators= 400,
nthread= 4,
num_leaves= 100,
predictor= 'gpu_predictor',
seed= 1000,
silent= 1,
subsample= 0.8,
tree_method= 'gpu_hist',
verbose= True
random_state = 42
)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.