4
$\begingroup$

I have a dataset containing a column of trials, a column of successes and other features; and, obviously, I can generate a probability column. I would like to use gradient boosting methods (like xgboost or lightgbm) to model the success probability. Which parameter shall I set to handle this in lightgbm or xgboost?

$\endgroup$
0
$\begingroup$

I do not think this is possible with the main (as I can tell) xgboost library, because this essentially sounds like a regression on rates/proportions situation (a dependent variable bounded in [0,1]).

An easy way to have implemented this would have been to allow the Beta distribution (see how it is bounded) as an objective function in xgboost, but it is not an option at this point.

| improve this answer | |
$\endgroup$
0
$\begingroup$

There are beta regression models for R and Python. They are designed to handle values between 0 and 1. However, as far as I'm aware, there is no direct beta regression implementation with LightGBM, XGboost etc. Some folks seem to expand beta regression into tree-based approaches, but this is not equivalent to boosting.

Also see this post for a very similar problem. As mentionned there, a naive approach could be to simply run a normal regression in a boosting setup and see how "bad" the results are (i.e. values $<0$ and $>1$). Alternatively you could try to predict the continuous features and calculate the success probability from predicted values.

| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.