# Dynamic learning rates in XGBoost cross-validation

XGBoost's xgb.train() method takes a learning_rates parameter, which can take a custom function to apply a dynamic learning rate, depending on the current training round.

I recently posted a paper explaining how I'm using it to both speed up training in the beginning, and making more precise towards the end.

However, there's a problem with this method: it tends to overfit on the eval set because there's no cross-validation.

I noticed that xgb.cv() method has no learning_rates parameter and therefore appears to not allow for a dynamic learning rate while doing CV.

So my questions:

• Is there a possibility to use dynamic learning rates with xgb.cv()?
• If not, is there a rationale behind this or is this just not implemented yet?
• Have you tried setting in the eta Hyper-Parameter, responsible for the learning rate. Have a look here github. Moreover, have tried uploading your question at the dmlc XGBoost github issues page? Furthermore, regarding your approach, I would be interested to see it in more use cases, I might try it. However, Gradient Descent is a complicated thing and the fact that XGBoost can be trained in different sub-samples, keeps me wondering how consistent this methodology would be. – 20roso Nov 14 '16 at 14:49