I'm evaluating the variability in performance (AUC) in the test set of a machine learning model with an intrinsic random component (xgboost). How many sources of variation should I use?
- Just replicate the analysis harnessing the model intrinsic randomness?
- Bootstrap the train set?
- Bootstrap the test set?
- Bootstrap both train and test?