# xgboost: Is there a way to perform regression on rates/percentages data?

I have a dependent variable, $$Y$$, that is made up of rates/percentages data, so each value is between $$0$$ and $$1$$. I was attracted to the xgboost library because it allows focusing in on specific subsets of the data in training itself, but I am stumped as how to perform regression on the data I have.

Normal OLS regression will produce outputs that will be over and under the [0,1] range if you do not change the likelihood to be the Beta distribution, or something else bounded in the same range, but will xgboost suffer the same mistake?

Any advice on trying to get this to work would be greatly appreciated by me.

In principle: yes, you will have the same problem as with OLS. However, since xgboost is tree-based (and by that non-parametric), you may get relatively accurate estimates, meaning that values which are below zero or above one would be rare (at least the problem should be less severe than with OLS). In this case you could simply restrict results to $$\hat{y} \in [0,1]$$. An alternative would be to do a multiclass classification task where you have 100 classes $$y=[1\%,2\%,...,100\%]$$. Boosting usually performs well on classification tasks.

Just a hint: xgboost can be somewhat "heavy" in data handling. Boosting tools such as lightgbm are a good alternative. I prefer lightgbm over xgboost.

• +1 for suggesting lightgbm - for speed & memory usage. Aug 6, 2019 at 8:08

You could use the reg:logistic objective function. https://xgboost.readthedocs.io/en/latest/parameter.html#learning-task-parameters

Edit: You need to use either the internal API for this, or use the sklearn API's XGBRegressor with objective='reg:logistic' (or binary:logistic).
Do not use XGBClassifier, as it will label-encode the target!

• But this command only takes classes as input and not floats, right? Or am I mistaken? Aug 6, 2019 at 15:17
• No, the "reg" indicates it's trying to fit regression, just with the logistic link. It turns out that even binary:logistic will happily work with inputs between 0 and 1, and in fact is the exact same fitting objective. The difference between the two is just the default evaluation metric. datascience.stackexchange.com/questions/9802/… github.com/bmreiniger/datascience.stackexchange/blob/master/… Aug 6, 2019 at 15:37
• Ah, okay... this might be a good solution for the problem. Aug 6, 2019 at 15:59

I agree with all those answers, it is a regression case. You can convert it into classification case if you are able to split your target values into several classes.

But I would also try to transform target values this way:

old_target = function(new_target)
new_target = inverse_function(old_target)


For example, softmax looks promising. It will give you another range of target values:

(-inf, inf)


Then, you train XGBRegressor on this transformed target.

In general, the type of function() depends on the target values distribution.

• So... what you're saying is a softmax transformation on my bounded [-1,1] values would spread it over the real number line (-inf, inf)? Wouldn't that cause problems with the values being spread out so wide that some of them will simply be too ungainly/too big/small for calculations? Aug 10, 2019 at 20:39
• > what you're saying is a softmax transformation on my bounded [-1,1] values would spread it over the real number line (-inf, inf)? Inverse softmax transformation. Aug 11, 2019 at 4:01
• > Wouldn't that cause problems with the values being spread out so wide that some of them will simply be too ungainly/too big/small for calculations? >> That's why I said "the type of function() depends on the target values distribution". Take 'wider softmax' in range (-2; 2). Aug 11, 2019 at 4:03
• Also, check lightgbm: > binary, binary log loss classification (or logistic regression). Requires labels in {0, 1}; see cross-entropy application for general probability labels in [0, 1] multi-class classification application It has the option you need: lightgbm.readthedocs.io/en/latest/Parameters.html In this case you may need to convert your [-1, 1] to [0, 1] and then convert to classification case. Aug 11, 2019 at 4:06