I have a very large dataset (~7 million rows) for which I have extracted ~500 features during feature engineering phase. I have trained an XGBoost which has a fairly good predictive capability (based on the metrics on the test set by cross-validation for time series).

Now I would like to reduce the number of input variables to the model to speed up the training, but I don't know what method to use or how to select them. (When exploring the weight, gain and coverage values, I see that some variables with high weight have very low gain. Which method to choose? Are there others?)

  • $\begingroup$ You could do some initial filtering to reduce the correlation between features maybe. Because if the features are highly correlated the calculations of the gain of the first feature will be the same as the second feature so makes the second feature kind of redundant. Using information could be a way to ""maximise"" the entropy/ graidient/hessian values for the gain, but removal of correlated features would be to maximise the gain of each feature involved. $\endgroup$
    – timmy1691
    Feb 21 at 9:33


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.