XGBRegressor hyperparameter optimization using xgb cv function

Question

I am trying to optimize hyper parameters of XGBRegressor using xgb's cv function and bayesian optimization (using hyperopt package). Here is the piece of code I am using for the cv part.

dtrain = xgb.DMatrix(X_train, label=y_train)
cv_results = xgb.cv(params,dtrain,num_boost_round = 1000, folds= cv_folds, 
                        stratified = False, early_stopping_rounds = 100, metrics="rmse", seed = 44)

However, I am getting the following error within the xgb.cv function (part of the Trace):

414     cvfolds = mknfold(dtrain, nfold, params, seed, metrics, fpreproc,
--> 415                       stratified, folds, shuffle)
    416 
    417     # setup callbacks

/anaconda3/envs/py36/lib/python3.6/site-packages/xgboost/training.py in mknfold(dall, nfold, param, seed, evals, fpreproc, stratified, folds, shuffle)
    261         except TypeError:
    262             # Custom stratification using Sklearn KFoldSplit object
--> 263             splits = list(folds.split(X=dall.get_label(), y=dall.get_label()))
    264             in_idset = [x[0] for x in splits]
    265             out_idset = [x[1] for x in splits]

AttributeError: 'int' object has no attribute 'split' 

I can't figure out why am i getting this error. The documentation for xgboost is also not very clear and sparse. So any help would be greatly appreciated.

Thanks

Answer 1

It seems to be complaining about the folds.split part, indicating that folds is an integer. If your cv_folds object is just the number of folds, pass that as nfolds instead; folds expects the actual sklearn KFold object rather than a number.