0
$\begingroup$

I have a general question regarding XGboost and especially the n_rounds parameter, regarding small datasets.

Normally I tune the n_rounds parameters by cross-validation, but what if you have too less observations to do proper CV? For example if I have 30 variables and 4000 observations in my training data, how can I find a nice value for n_round which is not over/underfitting the training data?

Are there any "best practices" for parameter tuning (also max_depth etc.) having small datasets?

$\endgroup$
  • 4
    $\begingroup$ I'm not sure whether you'd consider 4000 rows of 30 variables as too little data for 10 fold CV. $\endgroup$ – jkyh Jan 8 '17 at 5:16
1
$\begingroup$

You can use grid search or xgb.cv for find the best iteration. Run xgb.cv for example 500 trees and add early stopping criteria. Then you can use the best iteration for xgb.train. Have a look at these links.

Hypertuning XGBoost parameters

https://stats.stackexchange.com/questions/171043/how-to-tune-hyperparameters-of-xgboost-trees

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.