2
$\begingroup$

I'm running xgboost and I fix the seed number
but the result is different for each time when I rerun the xgboost with same dataset

xgb = XGBClassifier(
     bagging_fraction= 0.8,
     boosting= 'gbdt',
    colsample_bytree= 0.7,
    feature_fraction= 0.9,
    learning_rate= 0.05,
    max_bin= 32,
    max_depth= 10,
    min_child_weight= 11,
    missing= -999,
    n_estimators= 400,
    nthread= 4,
    num_leaves= 100,
    predictor= 'gpu_predictor',
    seed= 1000,
    silent= 1,
    subsample= 0.8,
    tree_method= 'gpu_hist',
    verbose= True
)

This is my parameter and

# Our data is already scaled we should split our training and test sets
from sklearn.model_selection import train_test_split

# This is explicitly used for undersampling.
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
kf = StratifiedKFold(n_splits=5, shuffle=True)
# partially based on https://www.kaggle.com/c0conuts/xgb-k-folds-fastai-pca
predicts = []
for train_index, test_index in kf.split(X_train, y_train):
    print("###")
    X_train, X_val = X.iloc[train_index], X.iloc[test_index]
    y_train, y_val = y.iloc[train_index], y.iloc[test_index]
    xgb.fit(X_train, y_train, eval_set=[(X_val, y_val)], 
            early_stopping_rounds=30)
    predicts.append(xgb.predict(X_test))
$\endgroup$
2
$\begingroup$

You are missing some seeds there:

  • Numpy
  • Pandas
  • StratifiedKFold

I will say that StratifiedKFold(shuffle=True) without a seed can make your results change.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.