I have been reading a couple of articles regarding polynomial regression vs non-linear regression, but they say that both are a different concept. I mean when you say polynomial regression, in fact, it implies that its Nonlinear right. Then why there is a difference in the Data Science world regarding both the concepts?
2 Answers
the difference is probably easily seen with an example. Linear regression assumes a form $$f(x, \beta) = \beta_0 + \beta_1 x_1 + \cdots + \beta_n x_n$$ with some covariates $x_i$ and some parameters to estimate $\beta_i$. An example of non-linear regression would be something like $$f(x,\beta) = \frac{\beta_1 x_1 + \beta_2 x_2}{\beta_3 x_3 + \cdots + \beta_n x_n}.$$ Essentially you are assuming your model to be of a nonlinear form. Polynomial regression on the other hand is a fixed type of regression where the model follows a fixed form $$f(x, \beta) = \beta_0 + \beta_1 x + \beta_2 x^2 + \cdots + \beta_n x^n$$ which is a nonlinear function, however it is still linear in the parameters $\beta$ you are trying to estimate.
That is to say,
Polynomial regression is non-linear in the way that $x$ is not linearly correlated with $f(x, \beta)$; the equation itself is still linear.
In the other hand, non-linear regression is both non-linear in equation and $x$ not linearly correlated with $f(x, \beta)$.
-
$\begingroup$ Nice answer, could you also tell if it's preferable to use non linear regression using RF or Boosting etc. than polynomial regression due to difficulty in finding right polynomial degree. $\endgroup$ Oct 10, 2019 at 21:00
-
$\begingroup$ can we say that polynomial is a special case (form) of non-linear regression? $\endgroup$ Dec 22, 2021 at 4:43
-
$\begingroup$ @AbdulkarimKanaan Linearity refers to linearity in the parameters: a linear combination of features. Since each polynomial term is its own feature, polynomial regression is a linear regression. $\endgroup$– DaveDec 3, 2022 at 19:59
Well elaborated by Marvin,To add on that-We define linear model as "A model that expresses the target output value in terms of a sum of weighted input variables" taking this definition into consideration for polymonial regression- the features are just numbers within a weighted sum