I have an issue with xgboost custom objectives: I do not manage to get consistent forecasts. In other words, the scale of my forecasts is not in line with the values I would like to predict. I tried many custom loss, but I always get the same issue.

import numpy as np
import pandas as pd
import xgboost as xgb
from sklearn.datasets import make_regression

n_samples_train = 500
n_samples_test = 100
n_features = 200

X, y = make_regression(n_samples_train, n_features,noise=10)
X_test, y_test = make_regression(n_samples_test, n_features,noise=10)

param = {'verbosity' : 1,
      'max_depth' : 12,
      'learning_rate' : 0.01,
      'nthread' : 3,

dtrain = xgb.DMatrix(X, y)

best_nrounds = 50

bst_reglinear = xgb.train(param, 

def reg_obj(preds,dtrain):
    y = dtrain.get_label()
    N = len(y)
    #residual = (preds-y).astype("float")
    grad = 2*preds-y
    hess = 2*N*np.ones(len(y))
    return grad, hess

bst_custom = xgb.train(param,
                   obj = reg_obj)

dtest = xgb.DMatrix(X_test)

pred = bst_reglinear.predict(dtest)

pred_custom = bst_custom.predict(dtest)
  • $\begingroup$ Compare default xgboost objective function implementation with what you implemented, if they diverge your scores could be diverging $\endgroup$ – vienna_kaggling Dec 17 '19 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.