Do you know why my custom_eval_metric doesn't work? I get the error:

XGBoostError: [07:56:32] C:\Users\Administrator\workspace\xgboost-win64_release_1.4.0\src\metric\metric.cc:49: Unknown metric function custom_eval_metric
def custom_eval_metric(preds, dtrain):
    labels = dtrain.get_label()
    preds = preds.reshape(-1, 3)
    preds_binary = []
    for element in range(0,len(preds)):
        tmp = []
        tmp = preds[element][2]
    labels_adj = [0 if x == 1 else x for x in labels]
    labels_adj = [1 if x == 2 else x for x in labels_adj]
    preds_binary = np.asarray([preds_binary])
    labels_adj = np.asarray([labels_adj])
    return 'ndcg score', metrics.ndcg_score(new_items, preds)
clf_xgb_out_of_sample = xgb.XGBClassifier(objective = "multi:softmax",
                           seed = 42,
                           max_depth = 8,
                            learning_rate = 0.1,
                            gamma = 0.25,
                            colsample_bytree = 0.8,
           verbose = True,
           early_stopping_rounds = 10,
           eval_set = [(X_test, y_test['Target'])])

1 Answer 1


From the documentation:

If a str, should be a built-in evaluation metric to use. See doc/parameter.rst.


If callable, a custom evaluation metric. The call signature is func(y_predicted, y_true) where y_true will be a DMatrix object such that you may need to call the get_label method. It must return a str, value pair where the str is a name for the evaluation and value is the value of the evaluation function. The callable custom objective is always minimized.

So you need to pass the function itself instead of a string of the function name


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.