35
$\begingroup$

Is there a way to add more importance to points which are more recent when analyzing data with xgboost?

$\endgroup$
1

3 Answers 3

15
$\begingroup$

You could try building multiple xgboost models, with some of them being limited to more recent data, then weighting those results together. Another idea would be to make a customized evaluation metric that penalizes recent points more heavily which would give them more importance.

$\endgroup$
1
  • 9
    $\begingroup$ The OP can simply give higher sample weights to more recent observations. Most packages allow this, as does xgboost. $\endgroup$ Aug 11, 2017 at 8:55
43
$\begingroup$

Just add weights based on your time labels to your xgb.DMatrix. The following example is written in R but the same principle applies to xgboost on Python or Julia.

data <- data.frame(feature = rep(5, 5),
                   year = seq(2011, 2015), 
                   target = c(1, 0, 1, 0, 0))
weightsData <- 1 + (data$year - max(data$year)) * 5 * 0.01

#Now create the xgboost matrix with your data and weights
xgbMatrix <- xgb.DMatrix(as.matrix(data$feature), 
                         label = data$target, 
                         weight = weightsData)
$\endgroup$
2
  • $\begingroup$ Thanks for your answer - its really helpful to see a coded example. How does the magnitude of the weighting function coefficients affect the model? I looked through xgboost docs, but I can't find information about the significance of these numerical values. $\endgroup$
    – kilojoules
    Dec 23, 2015 at 19:29
  • $\begingroup$ didn't know this trick, nice. there's a little tidbit in the xgboost doc under the function setinfo(), though its not very descriptive $\endgroup$
    – TBSRounder
    Dec 24, 2015 at 15:39
23
$\begingroup$

On Python you have a nice scikit-learn wrapper, so you can write just like this:

import xgboost as xgb
exgb_classifier = xgb.XGBClassifier()
exgb_classifier.fit(X, y, sample_weight=sample_weights_data)

More information you can receive from this: http://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.XGBClassifier.fit

$\endgroup$
2
  • $\begingroup$ Wish for R caret had this built in too.. $\endgroup$
    – pauljeba
    Jul 3, 2017 at 13:16
  • 1
    $\begingroup$ that should be xgb.XGBClassifier() in the second line of code but stackexchange does not allow edits of less than six characters... $\endgroup$ Jul 18, 2017 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.