1
$\begingroup$

From the paper, lightGBM does a subsampling according to sorted $|g_i|$, where $g_i$ is the gradient (for the loss function) at a data instance.

My question is that, when the objective is L1 loss/regularization, $loss_i = |y_i - \hat{y_i}|$, hence $g_i$ is either 1 or -1 (assume no zero loss). Therefore the absolute value of gradient is 1 for any data instance. How to sort then and select instances for the subsample? Or does lightGBM skip the subsampling process if L1 regularization is selected?

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.