Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Imagine a dataset having five predictor variables and a target variable, through scatter plot I observed three predictor variables having a linear relationship with the target variable and the other two having a nonlinear relationship.
How can I build a Generalized Linear Regression model in such a way that the non-linearity of the two variables is explained along with linear relationship of the other three variables?
I suggest using "Generalised Additive Models". These type of models are linear but can treat wild non-linearity. The idea is - e.g. with regression splines - that a number of linear regressions are "stacked", so that they can jointly account for highly non-linear effects.
Here is a Python implementation: https://pygam.readthedocs.io/en/latest/
When you are bound to linear regression (OLS), you can add polynomials to the regression. In this case you simply generate a new "column" in your data frame, containing e.g. $x^2$. You can add this variable to the regression directly because linear regression is additive:
Example:
$$ y = \beta_0 + \beta_1 x + u $$
...can be augmented by a squared term for $x$...
$$ y = \beta_0 + \beta_1 x + \beta_2 x^2 + u $$
... and this also works for $x^3$ (and so on) or you can take $log()$ etc.
With GAM you don't have to decide on how to model non-linearity. That is the great advantage of GAMs. When you stick to OLS, you need to check if the non-linearity (imposed by you) really helps to improve fit and/or prediction.
GAM are very well explained in "Introduction to Statistical Learning" have a look at Chapter 7. There also is Python code for the Labs in the book.
