# select hyperparameters using Latin hypercube sampling (LHS) from a large matrix/grid of parameter combinations

I have a matrix with each row corresponds to a hyperparameter for the XGBoost model. There are seven parameters to tune in XGBoost (as shown below: nrounds/iterations, max_depth, eta, gamma, colsample_byTree, min_child_weight, and subsample). I did a literature review to specify the range and interval of values for each parameter. Using those ranges and intervals, the parameter space generated around 62,500 parameter combinations. I am using R caret::train function to generate the best hyperparameter combination for my dataset. However, the amount of simulations (62,500) is too much. I read about the Latin hypercube sampling (LHS) and I think that is what I need to reduce the number of simulations by applying initial selection of hyperparameters using LHS. But I am having trouble implementing the approach in my dataset. My goal is to generate a manageable number of hyperparameter combinations (i.e., ~500) using LHS, and then use caret::train function to select best parameters. I would like to ask for help in implementing LHS using my parameter space.

nrounds <- seq(from = 200, to = 1000, by = 200)
maxdepth <- seq(from = 2, to = 10, by = 2)
eta <- c(0.01, 0.05, 0.1, 0.2, 0.3)
gamma <- seq(from = 0, to = 0.4, by = 0.1)
colsample_bytree <- seq(from = 0.4, to = 1, by = 0.2)
min_child_weight <- seq(from = 1, to = 5, by = 1)
subsample <- seq(from = 0.6, to = 1, by = 0.1)
dataGrid <- expand.grid(nrounds, maxdepth, eta, gamma, colsample_bytree, min_child_weight, subsample)


dials from Tidymodels has a grid_latin_hypercube function you can use for this https://dials.tidymodels.org/reference/grid_max_entropy.html

library(tidymodels)

xgboost_set <- param_set(list(learn_rate(range = c(0.01,0.3), trans = NULL),
trees(range = c(200,1000), trans = NULL), #trees(): The number of trees contained in a random forest or boosted ensemble. In the latter case, this is equal to the number of boosting iterations
loss_reduction(range = c(0,0.4), trans = NULL), #This corresponds to gamma in xgboost
tree_depth(range = c(2,10), trans = NULL),
min_n(range = c(1,5), trans = NULL), # assume is same with min_child_weight parameter in boosting trees
sample_prop(range = c(0.4,1), trans = NULL) # assume is same with min_child_weight parameter in boosting trees
))

# regularization_factor(range = c(0,0.4), trans = NULL),
set.seed(463)
me_grid <- grid_max_entropy(xgboost_set, size = 200) %>% mutate(type = "max entropy")
ls_grid <- grid_latin_hypercube(xgboost_set, size = 200) %>% mutate(type = "latin hypercube")
rn_grid <- grid_random(xgboost_set, size = 200) %>% mutate(type = "random")


thanks