Is there a way to add more importance to points which are more recent when analyzing data with xgboost?
$\begingroup$
$\endgroup$
1
3 Answers
$\begingroup$
$\endgroup$
1
You could try building multiple xgboost models, with some of them being limited to more recent data, then weighting those results together. Another idea would be to make a customized evaluation metric that penalizes recent points more heavily which would give them more importance.
-
9$\begingroup$ The OP can simply give higher sample weights to more recent observations. Most packages allow this, as does xgboost. $\endgroup$ Commented Aug 11, 2017 at 8:55
$\begingroup$
$\endgroup$
2
Just add weights based on your time labels to your xgb.DMatrix. The following example is written in R but the same principle applies to xgboost on Python or Julia.
data <- data.frame(feature = rep(5, 5),
year = seq(2011, 2015),
target = c(1, 0, 1, 0, 0))
weightsData <- 1 + (data$year - max(data$year)) * 5 * 0.01
#Now create the xgboost matrix with your data and weights
xgbMatrix <- xgb.DMatrix(as.matrix(data$feature),
label = data$target,
weight = weightsData)
-
$\begingroup$ Thanks for your answer - its really helpful to see a coded example. How does the magnitude of the weighting function coefficients affect the model? I looked through xgboost docs, but I can't find information about the significance of these numerical values. $\endgroup$ Commented Dec 23, 2015 at 19:29
-
$\begingroup$ didn't know this trick, nice. there's a little tidbit in the xgboost doc under the function
setinfo(), though its not very descriptive $\endgroup$ Commented Dec 24, 2015 at 15:39
$\begingroup$
$\endgroup$
2
On Python you have a nice scikit-learn wrapper, so you can write just like this:
import xgboost as xgb
exgb_classifier = xgb.XGBClassifier()
exgb_classifier.fit(X, y, sample_weight=sample_weights_data)
More information you can receive from this: http://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.XGBClassifier.fit
-
$\begingroup$ Wish for R caret had this built in too.. $\endgroup$– pauljebaCommented Jul 3, 2017 at 13:16
-
1$\begingroup$ that should be
xgb.XGBClassifier()in the second line of code but stackexchange does not allow edits of less than six characters... $\endgroup$ Commented Jul 18, 2017 at 10:05
sample_weightparameter, often used for imbalanced classification, but you can apply the weighting for any reason you need, such as data recency, e.g.: xgboosting.com/xgboost-configure-fit-sample_weight-parameter $\endgroup$