19
$\begingroup$

Is there a way to add more importance to points which are more recent when analyzing data with xgboost?

$\endgroup$
8
$\begingroup$

You could try building multiple xgboost models, with some of them being limited to more recent data, then weighting those results together. Another idea would be to make a customized evaluation metric that penalizes recent points more heavily which would give them more importance.

$\endgroup$
  • 3
    $\begingroup$ The OP can simply give higher sample weights to more recent observations. Most packages allow this, as does xgboost. $\endgroup$ – Ricardo Cruz Aug 11 '17 at 8:55
27
$\begingroup$

Just add weights based on your time labels to your xgb.DMatrix. The following example is written in R but the same principle applies to xgboost on Python or Julia.

data <- data.frame(feature = rep(5, 5),
                   year = seq(2011, 2015), 
                   target = c(1, 0, 1, 0, 0))
weightsData <- 1 + (data$year - max(data$year)) * 5 * 0.01

#Now create the xgboost matrix with your data and weights
xgbMatrix <- xgb.DMatrix(as.matrix(data$feature), 
                         label = data$target, 
                         weight = weightsData)
$\endgroup$
  • $\begingroup$ Thanks for your answer - its really helpful to see a coded example. How does the magnitude of the weighting function coefficients affect the model? I looked through xgboost docs, but I can't find information about the significance of these numerical values. $\endgroup$ – kilojoules Dec 23 '15 at 19:29
  • $\begingroup$ didn't know this trick, nice. there's a little tidbit in the xgboost doc under the function setinfo(), though its not very descriptive $\endgroup$ – TBSRounder Dec 24 '15 at 15:39
11
$\begingroup$

On Python you have a nice scikit-learn wrapper, so you can write just like this:

import xgboost as xgb
exgb_classifier = xgb.XGBClassifier()
exgb_classifier.fit(X, y, sample_weight=sample_weights_data)

More information you can receive from this: http://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.XGBClassifier.fit

$\endgroup$
  • $\begingroup$ Wish for R caret had this built in too.. $\endgroup$ – pauljeba Jul 3 '17 at 13:16
  • 1
    $\begingroup$ that should be xgb.XGBClassifier() in the second line of code but stackexchange does not allow edits of less than six characters... $\endgroup$ – Andre Holzner Jul 18 '17 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.