1
$\begingroup$

I want to compute a confidence interval for each sample for a lightgbm model I've trained.

If the model was a random forest, it'd be quite easy, just take all the trees and compute the standard deviation of the predictions.

I wonder if a similar think can be done with lightgbm. In particular:

  • Does it make sense to compute the standard deviation of a boosting model by taking the predictions of all trees, or something similar?
  • Can it be done in scikit-learn?
  • Are there alternatives to obtain confidence intervals for lightgbm predictions, appart from the alternative that would be training quantile models?
$\endgroup$
2
$\begingroup$

If you are looking for a statistical trick, I don't know, but

Recently Andrew NG team recently published about NGBoost.

NGBoost is a new boosting algorithm, which uses Natural Gradient Boosting, a modular boosting algorithm for probabilistic predictions.

In this Towards Data Science toy example you can see how to use the Python API:

Quoting the TDS author:

NGBoost’s one of the biggest differences from other boosting algorithms is can return probabilistic distribution of each prediction. This can be visualized by using pred_dist function.

https://towardsdatascience.com/ngboost-explained-comparison-to-lightgbm-and-xgboost-fda510903e53

In their web they say "NGBoost can be used with any base learner, any family of distributions with continuous parameters, and any scoring rule." So it seems pretty flexible to do whatever you consider.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.