2
$\begingroup$

This is a really simple example where my training data has a single feature vector (1,2,3) and an equivalent target vector (1,2,3). I can get xgboost to build a regression tree that perfectly fits the data, but when I use it to make predictions, the predictions against the training data, the predictions don't match the target. What gives?

train <- matrix(c(1,2,3))
target <- matrix(c(1,2,3))
bst <- xgboost(data=train, nrounds=1, label=target, eval_metric="rmse", max.depth=3, min_child_weight=1, gamma=0, lambda=0, alpha=0)
predict(bst, train)  # returns 0.65 0.95 1.25

xgb.plot.tree("f1", model=bst) enter image description here

xgb.model.dt.tree("f1", model=bst)
    ID Feature Split Yes  No Missing Quality Cover Tree Yes.Feature Yes.Cover Yes.Quality No.Feature No.Cover No.Quality
1: 0-0      f1   2.5 0-1 0-2     0-1    1.50     3    0          f1         2        0.50       Leaf        1       0.75
2: 0-1      f1   1.5 0-3 0-4     0-3    0.50     2    0        Leaf         1        0.15       Leaf        1       0.45
3: 0-2    Leaf    NA  NA  NA      NA    0.75     1    0          NA        NA          NA         NA       NA         NA
4: 0-3    Leaf    NA  NA  NA      NA    0.15     1    0          NA        NA          NA         NA       NA         NA
5: 0-4    Leaf    NA  NA  NA      NA    0.45     1    0          NA        NA          NA         NA       NA         NA
$\endgroup$
  • $\begingroup$ Derp. I just needed to specify eta=1 to remove the effect of shrinkage. $\endgroup$ – Ben Jul 11 '16 at 5:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.