Skip to main content
Bounty Ended with 50 reputation awarded by CommunityBot
added 566 characters in body
Source Link
oW_
  • 6.4k
  • 4
  • 29
  • 47

I think it is helpful to distinguish between linear functions (representing the relationship between independent and dependent variables) and linear models (representing the relationship between the model parameters and the outcome).

A linear model can be represented by a non-linear function. A linear regression model is any model that is represented by a linear function in the parameters. A polynomial can be represented as such.

Likewise a nonlinear regression model is any model that is represented by a nonlinear combination of parameters.

To quote from Wikipedia:

Although polynomial regression fits a nonlinear model to the data, as a statistical estimation problem it is linear, in the sense that the regression function E(y | x) is linear in the unknown parameters that are estimated from the data.

Let the model be represented by the function $f(x,w)$ with inputs $x$ and parameter vector $w$. Polynomial regression has the form $f(x,w) = w_0 + w_1*x + w_2*x^2 \ldots$ It is linear in the parameters because $f(x,a+b) = (a_0+b_0) + (a_1+b_1)*x + (a_2+b_2)*x^2 \ldots = a_0 + a_1*x + a_2*x^2 \ldots + b_0 + b_1*x + b_2*x^2 \ldots = f(x,a) + f(x,b)$. But it is not linear in the inputs because $f(x+y,w) \neq f(x,w)+f(y,w)$.

Often people think about the relationship $y=f(x)$ between $x$ and $y$, but linear in linear regression refers to linearity in $w$.

I think it is helpful to distinguish between linear functions (representing the relationship between independent and dependent variables) and linear models (representing the relationship between the model parameters and the outcome).

A linear model can be represented by a non-linear function. A linear regression model is any model that is represented by a linear function in the parameters. A polynomial can be represented as such.

Likewise a nonlinear regression model is any model that is represented by a nonlinear combination of parameters.

To quote from Wikipedia:

Although polynomial regression fits a nonlinear model to the data, as a statistical estimation problem it is linear, in the sense that the regression function E(y | x) is linear in the unknown parameters that are estimated from the data.

I think it is helpful to distinguish between linear functions (representing the relationship between independent and dependent variables) and linear models (representing the relationship between the model parameters and the outcome).

A linear model can be represented by a non-linear function. A linear regression model is any model that is represented by a linear function in the parameters. A polynomial can be represented as such.

Likewise a nonlinear regression model is any model that is represented by a nonlinear combination of parameters.

To quote from Wikipedia:

Although polynomial regression fits a nonlinear model to the data, as a statistical estimation problem it is linear, in the sense that the regression function E(y | x) is linear in the unknown parameters that are estimated from the data.

Let the model be represented by the function $f(x,w)$ with inputs $x$ and parameter vector $w$. Polynomial regression has the form $f(x,w) = w_0 + w_1*x + w_2*x^2 \ldots$ It is linear in the parameters because $f(x,a+b) = (a_0+b_0) + (a_1+b_1)*x + (a_2+b_2)*x^2 \ldots = a_0 + a_1*x + a_2*x^2 \ldots + b_0 + b_1*x + b_2*x^2 \ldots = f(x,a) + f(x,b)$. But it is not linear in the inputs because $f(x+y,w) \neq f(x,w)+f(y,w)$.

Often people think about the relationship $y=f(x)$ between $x$ and $y$, but linear in linear regression refers to linearity in $w$.

Source Link
oW_
  • 6.4k
  • 4
  • 29
  • 47

I think it is helpful to distinguish between linear functions (representing the relationship between independent and dependent variables) and linear models (representing the relationship between the model parameters and the outcome).

A linear model can be represented by a non-linear function. A linear regression model is any model that is represented by a linear function in the parameters. A polynomial can be represented as such.

Likewise a nonlinear regression model is any model that is represented by a nonlinear combination of parameters.

To quote from Wikipedia:

Although polynomial regression fits a nonlinear model to the data, as a statistical estimation problem it is linear, in the sense that the regression function E(y | x) is linear in the unknown parameters that are estimated from the data.