I am curious on what differences in implementation allow speed up of lightGBM over XGBoost, some times up to magnitude of orders.


First of all both the GBM methods are great and superiority of each algorithm is dependent on the data.

Major Differences between the two is that LightGBM uses a novel technique of Gradient-based One-Side Sampling (GOSS) to filter out the data instances for finding a split value while XGBoost uses pre-sorted algorithm & Histogram-based algorithm for computing the best split.

GOSS assumes that data points with small gradients tend to be more well trained. This means that it is more efficient to concentrate on data points with larger gradients.To attenuate the problem of biased sample it also randomly samples from data with small gradients.

To read more you can refer:

  • $\begingroup$ LightGBM doesn't turn on GOSS by default. LightGBM is also Histogram-based algorithm. $\endgroup$ – Louis Yang May 2 at 23:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.