XGBoost have been doing a great job, when it comes to dealing with both categorical and continuous dependant variables. But, how do I select the optimized parameters for an XGBoost problem?
This is how I applied the parameters for a recent Kaggle problem:
param <- list( objective = "reg:linear",
booster = "gbtree",
eta = 0.02, # 0.06, #0.01,
max_depth = 10, #changed from default of 8
subsample = 0.5, # 0.7
colsample_bytree = 0.7, # 0.7
num_parallel_tree = 5
# alpha = 0.0001,
# lambda = 1
)
clf <- xgb.train( params = param,
data = dtrain,
nrounds = 3000, #300, #280, #125, #250, # changed from 300
verbose = 0,
early.stop.round = 100,
watchlist = watchlist,
maximize = FALSE,
feval=RMPSE
)
All I do to experiment is randomly select (with intuition) another set of parameters for improving on the result.
Is there anyway I automate the selection of optimized(best) set of parameters?
(Answers can be in any language. I'm just looking for the technique)