I am trying to write a custom loss function for XGBRegressor that needs to punish predicted values that are under some arbitrary threshold. The code I came up with does not affect the results at all, and I was wondering if I was doing something way off, or just the model does not perform well with the function I am trying to use.

def f(y_true, y_pred):
    residual = (y_pred - y_true).astype("float")
    grad = np.where(y_pred < 2, -0.2*residual, residual)
    hess = 0.01 + np.repeat(0, len(y_pred))
    return grad, hess

Also, if you could clarify how to use negative and positive signs for grad and hess I would appreciate it. I know that hessian should be the first derivative of gradient, but I have no idea why my results are changing dramatically when I play around with the signs of grad and hess. I also don't know why I should use 0.01 for hessian when in fact it should be 0? Anyway, if you could nudge me in the correct direction that would be great and would save me a lot of time.

New contributor
mctasar is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.


Your Answer

mctasar is a new contributor. Be nice, and check out our Code of Conduct.

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy