# XGBoost non-linear regression

Is it possible to use XGBoost regressor to do non-linear regressions?

I know of the objectives linear and logistic.
The linear objective works very good with the gblinear booster.
This made me wonder if it is possible to use XGBoost for non-linear regressions like logarithmic or polynomial regression.

a) Is it generally possible to make polynomial regression like in CNN where XGBoost approximates the data by generating n-polynomial function?
b) If a) is generally not possible, would it be possible to declare a curve with its parameters and let XGBoost figure out the values of the parameters? (To give an example) Assume we guess that the curve can be approximated with:

$$10^{a\log_{k}({x})-b}$$

XGBoost would have to figure out $$a$$, $$k$$, and $$b$$. $$x$$ would be a given feature.

• As a heads up, polynomial regression is a type of linear regression. You might want to watch this video by MathematicalMonk (Jeff Miller).
– Dave
Commented Dec 4, 2021 at 15:08

Boosting is just a special way to fit some model by trying to successively/repeatedly "explain" the residual. See a minimal example for a linear booster here. So essentially the xgboost model with gblinear will be a "normal" linear model.
• Use boosting with "tree based" (gbtree). This will fit a model which is essentially "non-parametric". However, the success of this strategy will depend on the explanatory power of your "x" variables (which you did not mention in the question).
• Thank you for your answer Peter. I asked because we got an assignment in University. Everybody got an ML task. I got XGBoost where I used it successfully for classification. I've talked with a college that has LSTM for a Timeseries problem. Then I saw that XGBoost can do regression and gave it a shot with simple functions (kx+d style). After some success with those, I tried to get into more complicated ones and hit a wall. So that was my motivation for this question. I'll upload a notebook somewhere to make my question more detailed. Your answer is still highly appreciated. Commented Dec 5, 2021 at 9:14