From the paper, lightGBM does a subsampling according to sorted $|g_i|$, where $g_i$ is the gradient (for the loss function) at a data instance.

My question is that, when the objective is L1 loss/regularization, $loss_i = |y_i - \hat{y_i}|$, hence $g_i$ is either 1 or -1 (assume no zero loss). Therefore the absolute value of gradient is 1 for any data instance. How to sort then and select instances for the subsample? Or does lightGBM skip the subsampling process if L1 regularization is selected?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.