1
$\begingroup$

I tried to use monotonic constraints in LGBM, but if I use mean absolute error as an objective, it gives a warning that monotonic constraints cannot be done in l1.

What is the reason? Thanks!

$\endgroup$
1
  • $\begingroup$ Could you share the error traceback, so we can see where in the code the warning is produced? $\endgroup$
    – Ben Reiniger
    Jan 23, 2023 at 14:36

1 Answer 1

2
$\begingroup$

MAE is an odd loss function for GBMs: the gradient is constant ($\pm1$), and the hessian all zeros, so the usual tree-training target of $-G/H$ (possibly with additional terms for regularization) doesn't work.

Monotone constraints are implemented by rejecting splits that would produce node values (a.k.a. "weights" or "scores", the average target value among observations at that node) that would violate the specified directionality; see this blog post for details.

In LightGBM, MAE is handled by overriding the leaf scores at the end (see this comment on their github). But that means those adjusted scores are not available during tree construction, and so the monotone constraint requirements cannot be checked. They could probably add this alternative score calculation throughout; I'm not sure offhand if it would preserve monotonicity, but it even if so, it would be some effort for the PR, and increased training time.

$\endgroup$
1
  • 1
    $\begingroup$ Thanks! It seems like Hubert loss can be used with monotonic constraint $\endgroup$
    – morqueatsz
    Jan 25, 2023 at 0:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.