44
$\begingroup$

I'm trying to understand which is better (more accurate, especially in classification problems)

I've been searching articles comparing LightGBM and XGBoost but found only two:

  1. https://medium.com/implodinggradients/benchmarking-lightgbm-how-fast-is-lightgbm-vs-xgboost-15d224568031 - which is only about speed but not accuracy.
  2. https://lightgbm.readthedocs.io/en/latest/Experiments.html - which is from the authors of LightGBM and no surprise LightGBM wins there.

In my tests I get pretty the same AUC for both algorithms, but LightGBM runs form 2 to 5 times faster.

If LGBM is so cool, why don't I hear so much about it here and on Kaggle :)

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Thanks, but LightGBM also has packages for R and Python used by the majority of kagglers. I'm using it with Python. On my data and internet researches LGBM seems too perfect: very fast and not less accurate. But maybe I'm missing something here if it is not so widely used yet :) $\endgroup$
    – Sergey Nizhevyasov
    Commented May 12, 2017 at 4:34
  • $\begingroup$ Generally the skill and training time of (modern) XGB and LightGBM models are equivalent for comparable configurations, often lightgbm can be slightly faster, for example: xgboosting.com/xgbclassifier-faster-than-lgbmclassifier $\endgroup$
    – jasonb
    Commented May 16 at 23:18

2 Answers 2

Reset to default
30
$\begingroup$

LightGBM is a great implementation that is similar to XGBoost but varies in a few specific ways, especially in how it creates the trees.

It offers some different parameters but most of them are very similar to their XGBoost counterparts.

If you use the same parameters, you almost always get a very close score. In most cases, the training will be 2-10 times faster though.

Why don't more people use it then?

XGBoost has been around longer and is already installed on many machines. LightGBM is rather new and didn't have a Python wrapper at first. The current version is easier to install and use so no obstacles here.

Many of the more advanced users on Kaggle and similar sites already use LightGBM and for each new competition, it gets more and more coverage. Still, the starter scripts are often based around XGBoost as people just reuse their old code and adjust a few parameters. I'm sure this will increase once there are a few more tutorials and guides on how to use it (most of the non-ScikitLearn guides currently focus on XGBoost or neural networks).

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks, that makes sense. Maybe for top kagglers computation power is not a big problem, and it's easier to keep the scripts. $\endgroup$
    – Sergey Nizhevyasov
    Commented Jun 2, 2017 at 6:35
14
$\begingroup$

XGBoost now has a histogram binning option for tree growth similar to the one LightGBM uses. It provides about the same level of speedup and similar accuracy characteristics, although the algorithms are still not exactly the same.

There are some plots and tables here showing how they are right on top of each other now. https://github.com/dmlc/xgboost/issues/1950

To be fair, LightGBM cites their own performance tests showing them still edging out XGBoost (hist), though not by an order of magnitude any more. https://github.com/Microsoft/LightGBM/blob/master/docs/Experiments.rst#comparison-experiment

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.