I'm running a regression XGBoost model and trying to prevent over-fitting by watching the train and test error using this code:
eval_set = [(X_train, y_train), (X_test, y_test)] xg_reg = xgb.XGBRegressor(booster='gbtree', objective ='reg:squarederror', max_depth = 6, n_estimators = 100, min_child_weight = 1, learning_rate = 0.05, seed = 1,early_stopping_rounds = 10) xg_reg.fit(X_train,y_train,eval_metric="rmse", eval_set = eval_set, verbose = True)
This prints out as follows:
 validation_0-rmse:0.233752 validation_1-rmse:0.373165  validation_0-rmse:0.2334 validation_1-rmse:0.37314  validation_0-rmse:0.232194 validation_1-rmse:0.372643  validation_0-rmse:0.231809 validation_1-rmse:0.372675  validation_0-rmse:0.231392 validation_1-rmse:0.372702  validation_0-rmse:0.230033 validation_1-rmse:0.372244  validation_0-rmse:0.228548 validation_1-rmse:0.372253
However, I've noticed the number of training rounds printed out and in the evals_results always equals the n_estimators.
In : len(results['validation_0']['rmse']) Out: 100
If I change the number of trees to 600, the # of rounds goes up to 600, etc. I was under the impression that what's being printed is the metric result from each round of training, which includes training all the trees at once.
What is going on here? Is each layer of trees considered a separate training round?