In linear regression, we are fitting a polynomial to a set of data points. In Bishop's book of Pattern Recognition & Machine Learning, there are a few examples where the fit is a curve or a straight line. I am a bit confused if a curve is linear or not. The term linear means that the fit should be a linear function or a polynomial of degree 1 i.e., a straight line. But in many resources, examples are shown where the fit can be a polynomial of degree 3,9 etc. So, are these higher order polynomials linear?
Polynomial regression (for nth degree polynomial) in statistics is a special case of linear regression. Lets give an example for square function:
1. y = w*x
This is linear in terms of both weight (w) and data (x).
2. y = w*(x^2) OR y = w*z ; where z = x^2
This is still linear in terms of weight (w) and still treated as a linear regression for the transformed data (z). While the modeled relationship between y and x is certainly non-linear.
As you can notice above: The commonality in (1) and (2) is the linearity with the weight/ coefficient of linear regression.
Linear in linear regression means linear in parameters.
It refers to the relationship between the parameters that you are estimating (e.g., $\beta$) and the dependent variable (e.g., $y_i$). Hence, $y=e^x\beta+\epsilon$ is linear, but $y=e^\beta x + \epsilon$ is not.
This has nothing to do with the powers of the independent variables.
there are a few examples where the fit is a curve or a straight line.
The fit can be a curve and can incorporate higher powers of independent variables and be linear in parameters -- the betas.
If instead of using feature x, you use its square, you get a curve. It is a linear function of its variables, but you may enter the square or a cube of a variable, therefore making the graph appear as a curve. In this sense it is still linear while in essence it is a polynomial curve.