# Why does reducing polynomial regression to linear regression work?

Getting into machine learning, have a reasonable background in statistics and understand the basic principles of linear algebra (matrix multiplication etc.) - but am having a damn hard time figuring out why reducing a polynomial regression works.

For example, say we have this function:

$$y =$$ $$\beta_0$$ + $$\beta_1x_1$$ + $$\beta_2x_2^2$$

From what I've seen on 4 videos and 6 articles, we can use the following substitutions:

• $$x_2$$ = 1
• $$x_3$$ = $$x$$
• $$x_4$$ = $$x^2$$

To create the following model: $$y =$$ $$\beta_0$$ + $$\beta_1x_1$$ + ($$\beta_2x_2$$ + $$\beta_3x_3$$ + $$\beta_4x_4$$)

And then, fine - we can solve that as a normal multiple linear regression, and all is well.

But why, why does this work? I really cannot find an explanation for this.

Thanks!

• I don't understand the way you index the $x$. Why is there no index in your three bullets above? – Peter Feb 15 '20 at 11:15

I don't really understand your problem statement. You can simply add squared terms to any linear model. Say you have $$y$$ and $$x$$ and you want to model a polynomial function, you can write a model like:

$$y=\beta_0 + \beta_1 x_1 + \beta_2 x_1^2 + u$$.

In matrix form this would look like

$$y=X\beta + u$$.

An example with some numbers would write:

$$\left( \begin{array}{rrrr} 5 \\ 8 \\ 7 \\ 6 \\ \end{array}\right)$$ = $$\left( \begin{array}{rrrr} 1 & 8& 8^2 \\ 1 & 4 & 4^2 \\ 1 & 3 & 3^2 \\ 1 & 6 & 6^2 \\ \end{array}\right) \beta + u.$$

You can solve this like $$\hat{\beta} = (X'X)^{-1} X'y$$, where $$\hat{\beta}$$ are the coefficients of the linear regression model.