8
$\begingroup$

In linear regression, we are fitting a polynomial to a set of data points. In Bishop's book of Pattern Recognition & Machine Learning, there are a few examples where the fit is a curve or a straight line. I am a bit confused if a curve is linear or not. The term linear means that the fit should be a linear function or a polynomial of degree 1 i.e., a straight line. But in many resources, examples are shown where the fit can be a polynomial of degree 3,9 etc. So, are these higher order polynomials linear?

$\endgroup$
8
$\begingroup$

Polynomial regression (for nth degree polynomial) in statistics is a special case of linear regression. Lets give an example for square function:

1. y = w*x           

This is linear in terms of both weight (w) and data (x).

2. y = w*(x^2)    OR        y = w*z ; where z = x^2     

This is still linear in terms of weight (w) and still treated as a linear regression for the transformed data (z). While the modeled relationship between y and x is certainly non-linear.

As you can notice above: The commonality in (1) and (2) is the linearity with the weight/ coefficient of linear regression.

$\endgroup$
1
$\begingroup$

Linear in linear regression means linear in parameters.

It refers to the relationship between the parameters that you are estimating (e.g., $\beta$) and the dependent variable (e.g., $y_i$). Hence, $y=e^x\beta+\epsilon$ is linear, but $y=e^\beta x + \epsilon$ is not.

This has nothing to do with the powers of the independent variables.

there are a few examples where the fit is a curve or a straight line.

The fit can be a curve and can incorporate higher powers of independent variables and be linear in parameters -- the betas.

$\endgroup$
0
$\begingroup$

If instead of using feature x, you use its square, you get a curve. It is a linear function of its variables, but you may enter the square or a cube of a variable, therefore making the graph appear as a curve. In this sense it is still linear while in essence it is a polynomial curve.

$\endgroup$
  • $\begingroup$ This does not answer the question. You might want to elaborate further. $\endgroup$ – naive Feb 23 at 8:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.