Imagine a dataset having five predictor variables and a target variable, through scatter plot I observed three predictor variables having a linear relationship with the target variable and the other two having a nonlinear relationship.

How can I build a Generalized Linear Regression model in such a way that the non-linearity of the two variables is explained along with linear relationship of the other three variables?


I suggest using "Generalised Additive Models". These type of models are linear but can treat wild non-linearity. The idea is - e.g. with regression splines - that a number of linear regressions are "stacked", so that they can jointly account for highly non-linear effects.

Here is a Python implementation: https://pygam.readthedocs.io/en/latest/

When you are bound to linear regression (OLS), you can add polynomials to the regression. In this case you simply generate a new "column" in your data frame, containing e.g. $x^2$. You can add this variable to the regression directly because linear regression is additive:


$$ y = \beta_0 + \beta_1 x + u $$

...can be augmented by a squared term for $x$...

$$ y = \beta_0 + \beta_1 x + \beta_2 x^2 + u $$

... and this also works for $x^3$ (and so on) or you can take $log()$ etc.

With GAM you don't have to decide on how to model non-linearity. That is the great advantage of GAMs. When you stick to OLS, you need to check if the non-linearity (imposed by you) really helps to improve fit and/or prediction.

GAM are very well explained in "Introduction to Statistical Learning" have a look at Chapter 7. There also is Python code for the Labs in the book.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.