I'm using xgboost package in R with early stopping at 75 rounds. To monitor the progress the algorithm I print the F1 score from the training and test set after each round. After the algorithm has done 75 rounds, xgboost returns the model with the highest score on the test set, not the training set. My guess is that this has to do with the monitoring functionality and the watchlist parameter of xgboost
My intention of giving the algorithm access to the test set during training (using the watchlist parameter) was to monitor the training progress, and not to select the best performing classifier with respect to the test set. That would be cheating right? My question is two-fold:
- Is this really cheating?
and if it is cheating,
- Why is it programmed into the xgboost package?