I have a matrix with each row corresponds to a hyperparameter for the XGBoost model. There are seven parameters to tune in XGBoost (as shown below: nrounds/iterations, max_depth, eta, gamma, colsample_byTree, min_child_weight, and subsample). I did a literature review to specify the range and interval of values for each parameter. Using those ranges and intervals, the parameter space generated around 62,500 parameter combinations. I am using R caret::train function to generate the best hyperparameter combination for my dataset. However, the amount of simulations (62,500) is too much. I read about the Latin hypercube sampling (LHS) and I think that is what I need to reduce the number of simulations by applying initial selection of hyperparameters using LHS. But I am having trouble implementing the approach in my dataset. My goal is to generate a manageable number of hyperparameter combinations (i.e., ~500) using LHS, and then use caret::train function to select best parameters. I would like to ask for help in implementing LHS using my parameter space.
nrounds <- seq(from = 200, to = 1000, by = 200) maxdepth <- seq(from = 2, to = 10, by = 2) eta <- c(0.01, 0.05, 0.1, 0.2, 0.3) gamma <- seq(from = 0, to = 0.4, by = 0.1) colsample_bytree <- seq(from = 0.4, to = 1, by = 0.2) min_child_weight <- seq(from = 1, to = 5, by = 1) subsample <- seq(from = 0.6, to = 1, by = 0.1) dataGrid <- expand.grid(nrounds, maxdepth, eta, gamma, colsample_bytree, min_child_weight, subsample)