XGboost predict

Question

I am trying to understand this XGboost example.

After training:

ptrain = bst.predict(dtrain, output_margin=True)

they make prediction on test data, but the problem is the test data already has all the label. If my test data only has features and no label, how can I modify that example to make prediction?

Also, what I observe from their datasets: agaricus.txt.train and agaricus.txt.test is that they do not need to have the same features, and even each training data has different features. I have done linear regression before, I thought training and test datasets should have the same set of features?

Answer 1

The example cited is an advanced XGBoost concept. Notice the comment -

###
# advanced: start from a initial base prediction
#

Recall that each iteration in a GBM is built off the residuals of the previous iteration. There is an initial "guess" for the first iteration. That initial guess is often either 0.5 or the mean target (event rate in classification). This example shows that xgboost can start from a user supplied initial guess (set_base_margin).

I question if your data without a label is actually a test. How you will you test if the model is performing well on that data? It is difficult to know how the test worked if there isn't an answer key.

For the data, I have not looked at those files so I do not know the answer. Typically if data is missing, a GBM can treat these as missing values (null, sparse). Different GBM packages may handle missing data differently. xgboost handles sparse data so it handles missing values. Section 3.4 and an issue are 2 references but an internet search will get you more. Logistic regression steps are often taken to avoid missing data. Some of those steps still may be viable with GBMs - indicator variables for example.